You set the premise of a somewhat unsafe storage location. There are multiple ways to mitigate this.
The first and obvious:
It shouldn't be too hard to hide a small piece of paper with the mnemonic recovery words in such a way that it would be very hard to discover and find for an intruder.
You can even use an english book and hide your recovery words therein by inconspicuous marking of letters or whole words (use multiple copies of the same book at different places of family members, friends, workplace, you name it).
OK, if it would be the only english book in your household, then it's time to buy more of them.

Second:
Secure your mnemonic recovery words with an additional (optional) mnemonic passphrase which creates a whole new different wallet. The mnemonic recovery words alone then don't reveal your hidden wallet, only maybe a sacrificial one.
It's mandatory to backup both recovery words and the additional mnemonic passphrase very well and not together!
Third:
Paper backups can burn away should fire strike your fate, so steel or titanium metal backups are advisable which are usually larger and less easy to hide. But I'd argue that you can still hide a metal backup well enough that an intruder will have a hard time to find it.
Do not rely solely on your wet brain memory. It will fail you sooner or later.
Anybody can memorize 12 or even 24 words, it's only a matter of proper technique. But you have to keep practicing, recalling and refreshing your memory. To continuously verify your memory you need a reliable written backup otherwise you're soon likely screwed. The need for a reliable verification backup makes memorizing kind of pointless, in my opinion. It's just too dangerous without.
I'll repeat: human memory is a tricky thing, better not rely on it solely. You've been warned.