So you weren't screaming, there were no people around, they just took you to a car in broad daylight and drove away? Unless it was 4 am...
I had .55 BTC (yes I know about self custody)
That's a lot of money to be carrying around on your phone. You either are very rich and $20k is nothing, or you're very careless.
I own much more bitcoin than that and I'd never carry even .1 BTC with me. There's just nothing that I'd need to buy in person that costs this much. For larger transactions I'd send bitcoin from home.
When you gave them the phone and password, what happened next? Did they just leave, or drop you off somewhere?
I hope the police finds them, good luck.
All this could be avoided if only people learn not to carry a lot of money with them and not tell others they're bitcoin investors. The attackers knew that you own bitcoin from someone.
These people knew you had bitcoin and how you look like. They recognized you, "called you to a job" so they had your number. It wasn't a random robbery.