Devil's advocate, body shape's going to affect them while swimming even if they're not wearing anything, so the interaction between the "regulation apparel" and body shape could be considered just another form of that. (I can see how it might not be in the spirit of the sport, though.)