I don't think search engines are promoting spammy $#!+ content per se; what they're doing is heavily promoting newer content that's relevant to the most common search queries, as this gives them the only real hope of staying ahead of the spam. Of course the "small", long-lasting, independent Web is heavily disadvantaged by this shift.
One development that would be good for small web sites to look into is schema.org linked-data formats. Those might simply be too effort-intensive for the spammers to adopt (at a high level of detail) and perhaps too much of a commitment to quality and transparency (they would have to actively forge the info, which would leave them open to bans given the lack of plausible deniability), so they might become a viable signal of quality and lead to higher visibility in SERP.
(Similar for things like proper separation of style from content, that have always been advocated for in the web-standards community but are not really commercially viable.)
I'm not quite sure if others have experimented with this stuff already, but it seems worth trying.
One development that would be good for small web sites to look into is schema.org linked-data formats. Those might simply be too effort-intensive for the spammers to adopt (at a high level of detail) and perhaps too much of a commitment to quality and transparency (they would have to actively forge the info, which would leave them open to bans given the lack of plausible deniability), so they might become a viable signal of quality and lead to higher visibility in SERP.
(Similar for things like proper separation of style from content, that have always been advocated for in the web-standards community but are not really commercially viable.)
I'm not quite sure if others have experimented with this stuff already, but it seems worth trying.