(This could probably be fixed with a soft fork. If two blocks arrive at nearly the same time, take the one with the more accurate timestamp. But that's not the current rule.)
It would be more complicated than that: How do you objectively what is the "right" time to compare blocks timestamps to? You would need to define a reference; which would introduce a single point of failure, and a single trusted entity, both of which are against the goals of bitcoin.
As far as how you objectively define the right time in a way which is not controlled by a single entity:
https://en.wikipedia.org/wiki/Coordinated_Universal_Time and
https://en.wikipedia.org/wiki/International_Atomic_TimeTAI as a time scale is a weighted average of the time kept by over 200 atomic clocks in over 50 national laboratories worldwide.
So, 200 points of failure, and 50 trusted entities. And I think we'd discover it pretty quickly if someone managed to hack into all the atomic clocks in the world and mess with their clocks.
As far as how you distribute that time, NTP does not have a single point of failure, nor is it controlled by a single trusted entity.
If two blocks arrive within 5 seconds of each other, choose the one with the more accurate time. Where's the single point of failure? Where's the single trusted entity? What's the attack scenario in the first place? Worst case scenario the fix is imperfect; it doesn't introduce any new vulnerabilities. If the two blocks don't arrive within 5 seconds of each other, everything goes along as normally.
And yes, NTP isn't perfect. There are security enhancements from the old days of NTP (including from the time when Satoshi mentioned it in the code as a future improvement), though, and even more that can be made.
When two blocks arrive within 2 seconds of each other, and one is timestamped 2 seconds ago, and the other is timestamped 2 minutes ago, you build on the one with the more accurate timestamp, because it's much less likely that that's the one which is employing "selfish mining" or whatever you want to call it. It's not a perfect solution, but it's much better than what we have now.
The current implementation rejects blocks that are more than a few minutes off according to the miners, and that appear to be "before" the latest block, because even without a central autority "built-in" the protocal, you can be pretty sure that all "good faithed" miners will agree on the current time within a few minutes. Such checks prevents an attack by which a miner would "fake" the timestamp to make the block appear having been far in the future on in the past to trick the difficulty retargetting algorythm to do stupid things, but it is not accurate enough to know if a block is "better" than another.
That's a start, but NTP accuracy is much better than "a few minutes". Hell, a clock synced via NTP once per day is almost certainly going to be a whole lot more accurate than "a few minutes". We can't close the gap completely. But being able to delay a block by only a few seconds makes an attack a lot less likely to be successful than if you can delay a block by a few minutes.
"Good faithed" miners should be able to agree on the current time within a few seconds. There's no good excuse for a miner's clock being off by dozens of seconds.