Bugs accumulate until people give up Yes - all those additonal (seldom used) features have costs ... more bugs, harder to use, slower app, more storage & memory.
I submit that these costs are mostly insignificant stacked up next to "the alternative lacks that one niche feature you personally absolutely cannot live without."
They might be insignificant costs to the user who wants that niche feature, but they affect all users. So millions of people are pay costs associated with something used by a tiny niche.
The problem is that everyone is part of one niche or another. Each group, naturally, sees their own needs are the core feature set the software should have and thinks the other stuff is irrelevant junk that can be removed. But the reason this doesn't happen is a lack of broad consensus on which features belong in which categories.