Yeah I think it typically only gets slower, not faster. But it's a good point - if you're going to run your physics loop in the update loop (not ideal) then your movements should be time based.