By using gRPC, we get much more without thinking too much. Also, bi-directional streaming is instantaneous and it is not like afterthought long polling.
It is pretty straightforward and not too hard to understand the API. Designing APIs with protobuf makes things convenient and it brings lots of already written tools with it.
Since gRPC built on top of HTTP/2, we thought gRPC as easiest and performant way of writing a HTTP API with good defaults.
"we get much more without thinking too much" is precisely one of the reasons for the original push to REST (and against RPC at that time SOAP and CORBA and RMI etc.) was made in our industry 10-15 years ago. By falsely representing a _remote_ resource as if it was a _local_ one we open a whole can of worms; not just performance, but resilience, infrastructure issues, etc. Transferring documents over HTTP with a common language of HTTP's verbs was supposed to get programmers to model their applications as the sets of resources that they are, discourage them from making excessive round trips, make debugging easier, and make use of standard HTTP load balancing hardware.
Yes it's easy to make that RPC call. But should you?
I've been off doing other things in the intervening period, but while I had my back turned the industry seems to have turned its back on REST and gone whole hog on RPC. Again.
At first I was thinking this was just internally here at Google, where protobufs and gRPC reign supreme. But it seems to have taken hold everywhere.
What did I miss? Why have we swung this way. Again. Is the pendulum going to go back?
> By falsely representing a _remote_ resource as if it was a _local_ one we open a whole can of worms
I've heard this argument before, but how does gRPC itself cause this issue to manifest? I'm curious to hear what your opinions are on a better alternative, and how not to represent a remote resource as a local one.
But the fact that it presents the remote resource in an API which resembles a local object means that programmers often get lazy in the manner of which they think about these things. The REST semantic is supposed to make this more explicit.
A remote object is not an object in your program or even your computer. It's something you're taking from something that is computationally miles and miles away. Compared to the microseconds it takes to dispatch a local call, it's an eternity away, even on a local network of the highest speed.
Accumulate those latencies over thousands of dispatches, and trouble can ensue.
I am reminded of an observation from when I first joined Google, coming out of their acquisition of the scrappy awesome ads company I worked at (Admeld).
We had a little service that kept track of ad impression caps / budgeting. We didn't want to serve an ad a single time more than the customer wanted us to, etc. Serving many thousands of ads per second, a process distributed across multiple machines in multiple data centres, this is a bit of a tricky shared state problem. The people who came before me had designed a rather clever solution which used a form of backoff to trickle down the number of ads served as they got closer and closer to budget cap, and to synchronize this state across clusters (this was before there were Rafty services to make this kind of thing easier, BTW).
We had a bit of a show and tell with Google when we first joined. I wanted to know how they were handling this problem, since their scale was many times ours, so I asked the question and got a puzzled/annoyed look:
"Oh we just make an RPC call to our budgeting server."
Summary: if you're at Google you don't have to worry as much about these problems. You still do, but there's an insane amount of infrastructure and horsepower and an army of SREs to help make it happen.
So, yeah, my point is -- just because Google does something or has invented something doesn't mean it's the best way to do it, especially in a smaller more cost conscious organization.
Thank you for taking the time to write this up, and the interesting anecdote.
What I don't see though, is how is making a (g)RPC call any different from making a REST call? Like you said, the REST call is supposedly more explicit, but at the end of the day, it seems more of a convention and some hard underlying difference. What's the difference between `httpClient.get("...")` and `grpcClient.foo(...)`?
The latter encourages you to model things as remote procedure calls, the former, well, in my experience it's open to incompetence and abuse, too, so... ehh... but done properly... well, go read the Roy Fielding paper :-)
I mean, internally at Google we have load balancing for grpc (I'm sure the outside world does now, too, but it was new to me when I joined) -- but load balancing HTTP requests containing readable JSON or XML documents, that's far more sysadmin friendly, wouldn't you say? Off the shelf infrastructure, nothing exotic, easier to monitor. Same goes for caching, for proxies, etc.
Being able to just stick a URL for a given resource in your browser, or hit it with wget/curl to read it, that's a serious bonus.
In general URLs follow conventions similar to those down by our Unix forefathers, when they designed the filesystem API. We are all familiar with this model. And in some ways REST done right is very Unix philosophy -- provide a consistent model upon which a bunch of little tools can interoperate.
I could go on... have to go put my daughter to bed tho
I don't think that's accurate... as the article claims (and it seems believable) it has only appeared in etcd because of Xooglers interfering with the project in the name of Kubernetes (also from Google). Or do you actually have examples of gRPC being used in many other non-Google-related projects?
For example, whole etcd API is located at https://github.com/etcd-io/etcd/blob/4c6881ffe4b3bae257c0720...
It is pretty straightforward and not too hard to understand the API. Designing APIs with protobuf makes things convenient and it brings lots of already written tools with it.
Since gRPC built on top of HTTP/2, we thought gRPC as easiest and performant way of writing a HTTP API with good defaults.