> ~25X greater throughput than Google Protocol Buffers (GPB) with very low and predictable latency
Seems like it might take more than structure tweaking to overcome the quoted delta. I'd still like to see a general comparison too though, especially to Cap'n Proto on the one side and HDF5 on the other.
Looking at their benchmarks -- specifically, in C++ -- I immediately see a coding error that probably makes the protobufs version run half as fast as it should. Namely, they make multiple redundant calls to message.ByteSize(). ByteSize() is not free -- it has to walk the whole message to compute the size, doing many of the same expensive branches that would be needed when actually serializing. Had they profiled their benchmark, they would have noticed this.
This is just from one minute of looking. There are probably more problems. Unfortunately, I find this is typical of benchmarks that claim something is massively faster than Protobufs. Admittedly, this may be indicative of the Protobuf interface being unintuitive.
Not that I doubt that SBE is faster; the design should obviously make it so. But it's probably not 25x faster.
Good point, although it's not like the stock protobuf implementation (rather than protocol) is the best that can be had. For example, this implementation of protobuf deserialization is quite a bit faster:
Hi there, upb author here, thanks for the shout-out!
The stock protobuf implementation is pretty close to optimal given what it does. Where upb is faster, it is faster by doing less.
For example, I can beat protobuf if you don't write the parsed data into a tree structure (or only write a few fields). I can beat it if your input has unknown fields and you don't care about preserving them. But if what you need is a tree structure that contains 100% of the input data, protobuf is hard to beat speed-wise (though there are still a few tricks, like arena allocation, that can beat it given some usage patterns).
The design principles between SBE & Cap'N Proto are very similar, so in that some part the lessons are already learned.
I haven't bench marked either of those 2 against each other but would not be surprised if they were very close to each other in the C++ implementations. That said, SBE does provide a non-JNI java implementation which as far as I know is not available for Capnproto yet. SBE also provides a C# implementation which Cap'n proto doesn't. That in and of itself may be enough for many people (Cap'N seems to have a lot of languages under development that SBE doesn't fwiw).
Seems like it might take more than structure tweaking to overcome the quoted delta. I'd still like to see a general comparison too though, especially to Cap'n Proto on the one side and HDF5 on the other.