You are giving yourself too much work and stressing your brain at the same time. It's a bad practice for you to memorize your seed phrase because if there's any accident that affects you head, you might lose your memory and your seed phrase is gone likewise your funds.
The best way is to write down your seed phrase on a metal steel or on a paper and make three duplicates of it and store them in three different locations far from each other. I hope that you haven't burnt your seed phrase backup which you wrote down on paper like you said.
There is nothing bad in memorizing the seed phrase, but it shouldn't be the only option for recording that one needs to rely on. In life, one can lose memory at anything due to stress, illness, or even an accident.
The seed phrase needs to be stored in a good way that it can be accessed at any time, but very secure and private from strangers. Instead of memorizing the seed phrase, there are different ways the seed phrase can be stored for the future.