Threads are bad for high concurrency. Specifically when you need to call out to another service that has some latency.
Say you have 1000 threads. To handle a request each one needs to make 50ms of external or DB calls. In one second, each thread can handle 20 calls. So you can handle 20k requests/second with 1000 threads. But Rust is so fast it can serve 500k requests a second. So with regular threads, you need ~25,000 threads. The OS isn't going to like that.
With async you can run a single thread per core, with no concurrency limits. So you get your 500k requests without overhead. With fibers you just run 20k fibers which is a little bit of overhead but easy to do.
This is the core reason everyone is pushing async and fibers in fast languages. When you can push a ton of requests/second but each one has latency you can't control, regular threads will kneecap performance.
In "slow" languages like Python, Ruby, etc, async/fibers don't really matter because you can't handle enough requests to saturate a huge thread pool anyways.
Let's be real here, its not just the memory requirements, because context switching and the associated nuking of cpu caches are not free. You can go very far with it nowadays, but you can go much farther with async code, if you really need to.
We have some pretty vanilla file upload code that needs async. S3 latency is fairly high. If you're uploading a few tiny files per user per second, thread usage gets out of hand real fast.
With a simulated load of ~20 users we were running over 1000 threads.
Several posts in the chain say that 20k+ threads is "fine". Not unless you have a ton of cores. The memory and context switching overhead is gigantic. Eventually your server is doing little besides switching between threads.
We had to rewrite our s3 code to use async, now we can do many thousands of concurrent uploads no problem.
Other places we've had to use async is a proxy that intercepts certain HTTP calls and user stats uploader that calls third party analytics service.
Just sayin it's not that unusual to need async code because threading overhead is too high
> And IMHO the added code complexity is not worth the trouble.
The thing is, this is just that - your opinion, generalized as The Truth. But engineering is about making the right trade-offs. Often threading will be fine, you'll win simplicity, and all is good. But sometimes you really need the performance, or your field is crowded and its a competitive advantage. Think large-scale infrastructure at AWS, central load-balancers, or high-freq-trading.
It goes deeper than that. There is plenty of research showing that shared memory multithreading is not even a viable concurrency model. The premise that threads are fine and simple is just false.
I'm not sure what you mean. One of Rust's major research contributions is to show that shared memory multithreading is a perfectly viable concurrency model, as long as you enforce ownership discipline to statically eliminate data races.
Say you have 1000 threads. To handle a request each one needs to make 50ms of external or DB calls. In one second, each thread can handle 20 calls. So you can handle 20k requests/second with 1000 threads. But Rust is so fast it can serve 500k requests a second. So with regular threads, you need ~25,000 threads. The OS isn't going to like that.
With async you can run a single thread per core, with no concurrency limits. So you get your 500k requests without overhead. With fibers you just run 20k fibers which is a little bit of overhead but easy to do.
This is the core reason everyone is pushing async and fibers in fast languages. When you can push a ton of requests/second but each one has latency you can't control, regular threads will kneecap performance.
In "slow" languages like Python, Ruby, etc, async/fibers don't really matter because you can't handle enough requests to saturate a huge thread pool anyways.