Most people don't know that bitcoin even exist, the people that do know about bitcoin could be not using bitcoin for several reasons:
- they don't understand the concept of bitcoin
- they are afraid of losing money when they start to invest into it
- they don't know anyone who are into bitcoin, so if something goes wrong, they won't know who to turn to
- or they just don't believe bitcoin would be a benefit to them
You're right, those are indeed the most common reasons as to why people don't get into bitcoin.
Some people know but they just simply don't believe how much potential bitcoin has