It's not wasted. Maybe I'm more sensitive than average, but I can absolutely tell the difference between different beans and roasts even after cream/sugar has been added.
Working as a barista I was always amused when a down to earth tradesman who added sugar to their coffee would pick up on subtle blend changes while others who assumed coffee lover or foodie as part of their identity would be surprised when we casually mentioned that the house blend was seasonal.