I suspect the same reason it's fairly difficult to get high quality sushi fish as a regular consumer, the high end markets buy it all up.
But you also have the problem that OP described where people literally have no idea what quality actually taste like so they can get away with selling mediocre fish. Nearly all fruit in the United States is another example. My cousin by marriage was born abroad and he absolutely refuses to eat his favorite fruit, mango, because so much of it is outright trash and even a "good"mango sucks in comparison
You can most certainly find them if you want them and it's a fruit that is at least somewhat cultivated locally. Try peaches when they're really in season, which only seems to be a few weeks a year. I've had them right after picking from a farm in the Bay Area but I'm sure Georgia and other peach-growing areas have access to this as well. I don't know if I'll ever find peaches that give me the experience I had of biting into peaches as juicy and flavorful as those.
Or, try apples in a place that actually grows a large amount and diversity of apples. In season, you can walk into some grocery stores and find 20+ varieties that all actually taste unique.
Wait what? I grew up in Miami and ate mangos off the tree. I've also had store bought mango and Ataulfo (Champagne) mangos from Mexico. I don't think I'm missing anything with the store bought mangos. So I think that's a bad example.
But, there's a variety of pineapple you can only get only in Hawaii that's way better. And I've only ever sourced really good key limes from friends and family trees in Florida.
Store bought tomatoes are also usually shit, but you can get ones from the farmers markets and road-side stands.
I can't specifically speak to mangoes since I haven't tried them in that many places, but melons in the United States are outright terrible in comparison to anything I had when I lived in Japan. They cost more but you're pretty much guaranteed quality. A relatively inexpensive melon in Japan might be about 10 bucks but its going to be as good as the best melon you can get in the crap shoot at the grocery store. When you start moving up in price the melons basically become so good that they are an out and out replacement for any kind of high end dessert. Theres no need for ice cream, yogurt, honey, etc because the melon is just that good.
What kind of melon? We have fantastic watermelon here if you get it in-season and local and I've been pretty happy with cantaloupe and honeydew. A lot of problem is Americans have no clue what in-season fruit is and don't buy local.
NC has a bounty of fruit and vegetables available locally grown:
I miss fruits in general in japan. It’s one of those things I’m willing to pay a premium for because they are just that much better. I can’t eat honey dew and cantaloupe in the states anymore.
Eat a mango from India or Pakistan (in the summer) before you die. Hard to find in the US unfortunately, but you'll realise that the mangoes you're familiar with taste like tree bark.