No, but many workers get paid meager (but slightly livable for their area) wages we would likely see as/refer to as modern "slave labor"
Is it a pretty thing? Not even slightly. However, this lets them/their family survive. Take it away, native workers will die/turn to even shittier methods of survival.
I think the EU was set to ban the use of palm oil for use in biofuels this year. Also, biofuel is/was "only" 50% of the EU's total palm oil consumption. And for other coutries, it's a lot less.
Also palm oil feeds many families in SE Asia. Boycott is not a solution.