Modern JVMs do support SIMD vectorisation now and I've seen at least one example where JVM outperformed GCC 4.5 in loop vectorisation by 2x. There is no reason JIT compilers cannot do all the same SIMD optimisations that static compilers do.

If you want really top performance for non-trivial cases, e.g. matrix multiplication, I guess you need to probably vectorise by hand and go down right to assembly level.