HTTP/2 Theory and Practice in NGINX, Part 3
This transcription blog post is adapted from a live presentation at nginx.conf 2016 by Nathan Moore. This is the third of three parts of the adaptation. In Part 1, Nathan described SPDY and HTTP/2, discussed proxying under HTTP/2, and summarized HTTP/2’s key features and requirements. In Part 2, Nathan talked about NPN/ALPN, NGINX particulars, benchmarks, and more. This Part includes the conclusions, criticism of HTTP/2, closing thoughts, and a Q&A.
td {
padding-right: 10px;
}
Table of Contents
| 22:15 | Conclusions |
| 24:05 | Second Systems Effect |
| 24:43 | Criticism of HTTP/2 |
| 25:32 | W. Edwards Deming |
| 25:40 | Theory of Constraints |
| 26:40 | W. Edwards Deming, continued |
| 26:57 | Theory of Constraints, continued |
| 28:07 | Buzzword Compliance |
22:15 Conclusions
One thing I really want to emphasize here is that engineering is always done with numbers, right? The moment someone comes out, “I think this, I think that,” well, that’s great, but that’s opinion.
You need some sort of numbers. You really need to take the time to understand what your app is doing. You need to take the time to benchmark, make sure this actually works for you.
You really, really have to optimize. You really have to make sure that you’re playing to its strengths. If you can play to its strengths like header compression, it’s there, it will give it to you, but you better make sure that’s the use case that you’re using it for and it actually works for you.
If you look a little more globally in terms of future proofing of this, H2 is not going away, right? Now it’s been ratified as a standard. It’s now been pulled into NGINX and other main lines. It’s very common. It’s pulled into Apache.
Support for this is basically uniform. It sits across pretty much everywhere, so the protocol itself isn’t going to disappear. You need to make sure that it’s working correctly for you. The last thing I wanted to mention in terms of future proofing is: you noticed that it reduced the number of connections?
If you are an app developer, you probably don’t care so much about this because this is really your SRE’s responsibility, making sure there’s enough computer resources available for it. That’s not really your problem. But if you’re in the SRE role – I, as a cache, I adore this. I’ll run a server with 50,000 simultaneous connections, 100,000 simultaneous connections, and not break a sweat.
If I have a protocol which reduces that down to 25,000 to 50,000, well, that’s a major benefit to me, right? Maybe not to you, but it certainly benefits me. A lot of pros and cons are associated with it. You have to be very, very careful in evaluating it, you cannot just push the button. That’s sort of the technical point I promised.
24:05 Second Systems Effect
Good, I still have a little bit of time here, because what I wanted to do is go into some of the criticisms. In the academic world, also, there have been some criticisms of the protocol itself. There really is a thing called the Second Systems Effect, and this has been noted by a lot of other guys – I’m just stealing, ripping off from Wikipedia here, right? – the tendency of a small, elegant and successful system to have elephantine feature-laden monstrosities as their successors due to inflated expectations. This is unfortunately very common. IPv4 versus IPv6 is left as an exercise to the reader.
24:43 Criticism of HTTP/2
This is absolutely something that we’ve seen in the past. And there was a gentleman by the name of Poul-Henning Kamp. He is one of the developers behind Varnish, one of the competitor products to NGINX, and he wrote this letter and it got published in the journal of the ACM.
This is very high up in the academic world. It’s about as high up as you can go publicly in the academic world. And he has this wonderful, wonderful quote which I’ve just ripped off shamelessly, “HTTP/2 – IETF is phoning it in, bad protocol, bad politics.” The part I love is that last sentence: “Uh-oh, Second Systems Syndrome.”
If that sounds underwhelming, it’s because it is. There are very legitimate complaints about what’s going on with this and again, I have to go back to my earlier comment that you have to benchmark. You have to know it’s right for you. You have to know that the use case works for you.
25:32 W. Edwards Deming
I’m not that cynical. And the reason why I’m not that cynical about it is: I want to go back to an older concept I’m going to rip off shamelessly from Deming which is his definition of a system.
And his system is a network of independent components, interdependent components that work together to try to accomplish the aim of the system. So Linux, NGINX, your network card, well, those are your network of interdependent components and they all work together to try to accomplish the aim of the system which is to serve out a web page, to serve out an HTTP object, right? That sure looks like a system to me.
25:40 Theory of Constraints
And if I have a system, then I can apply the Theory of Constraints. What’s Theory of Constraints?
Well, a guy named Dr. Goldratt came up with this, wrote a book about it in the 80s and spent a long time with it. But it’s a methodology for identifying the bottleneck. What Theory of Constraints says is that given a system, and you’re interested in performance, you’re always looking for the bottleneck, you’re looking for the slowest part of the system.
Then you go alleviate the bottleneck, you rerun your benchmarks, find out where the new bottleneck is, lather, rinse, repeat. And you continue this iterative cycle until you get something that meets your minimum specified criteria.
26:40 W. Edwards Deming, continued
Also known as the Shewhart cycle, right?: Plan Do Study Act. You plan: I think the bottleneck is here. You do: you alleviate that. You study: did it actually work? And act: If it did? Great. Move on to the next one. If it didn’t, then I have to review and go back and replan how I’m going to affect this particular problem.
26:57 Theory of Constraints, continued
Is H2 the bottleneck in your system? And that’s where you’re stuck, you have to do the benchmarking work. You can’t just take my word for it, you can’t take anybody else’s word for it, you have to make sure that you are playing to its strengths, that it works for you.
You have to put the leg work in on this one, you cannot just phone it in. So to go back to the very first story that I started out with: as I said, I was supposed to give this talk with a couple of other gentlemen who unfortunately bailed on me.
What they were really interested in doing when we sat down to talk about it was: we talked about how H2 was probably a feature that a lot of customers were asking for, but in our own benchmark and testing, we didn’t really see a major benefit to doing it for most people.
On the other hand, it didn’t seem to cost us an awful lot in order to offer it, which is one of the reasons why we went ahead and did it, right? We were responding to the customers, but we were kind of indifferent, right? It didn’t really do much of anything for us.
And the problem is: what do you call it when you have the situation and sales and marketing wants you to do something, wants you to add a feature, but it doesn’t really do anything, but it doesn’t really hurt. What do you call that?
28:07 Buzzword Compliance
Well, that’s kind of the definition of buzzword compliance which was the original topic of the talk. All right, and I think I have left enough time for a reasonable amount of questions.
28:21 Questions?
My expectation is that this is going to start a holy war because stuff like this always does, at least among the engineers in the crowd.
Yes, sir. Ah, no worries. Well, when Mike gets here, let me know. Hi Mike.
Q. So is StackPath implementing server push something you guys are gonna hack into your offering or you just sticking with the upstream?
A. We were talking about it for the initial product launch. The answer is no.
Q. So do you think in a reasonable time you could have come up with a benchmark that would have actually demonstrated the performance improvement of HTTP/2, let’s say, by interleaving?
A. Yes, yes sir. That’s an excellent point. You’re absolutely right. Through interleaving and the ability to do multiple requests over one connection, you’re absolutely right, it’s totally possible to come up with a benchmark that does that.
I didn’t, for the obvious reason, I was trying to show the difference there, but you’re absolutely right, there are such benchmarks. There are use cases where H2 outperforms the older protocols especially in encrypted scenarios.
Q. (Continued) Actually, a clarification: In your benchmarks, was it a single SSL connection and you were just putting multiple requests on it, or was it like multiple sessions as well?
A. You’re absolutely right. Just one connection, and this particular benchmark is a very, very stupid one because I’m asking for the same object over and over.
Even if I tried to interleave these, it didn’t matter because it was the exact same object. It looked the same if I was requesting 1234 or 1324. What? Nobody wants to argue the criticism of the protocol?
I can talk more about it if you like. I mean, it’s not just Mr. Kamp who has brought up criticisms of it.
Yes, sir.
Q. So for the batching of multiple requests into one connection, it feels like it would have a significant improvement on the performance for the SSL side.
Is that what you would expect in an actual deployment, in a CDN deployment, or would you still expect that the number of connections required would still, for other reasons, have to go large?
A. My expectation is that the number of connections is going to decrease and that therefore I should have more CPU and other resources available to handle the encryption overhead. But a lot of this comes down to systems design because the moment the connection bottleneck is alleviated, I can start to afford to spend CPU resources on other things.
Ideally, I can offload even encryption from the CPU that will result in lower latencies, which is ultimately what I’m very, very deeply interested in. So I want to get to the point where I can run incredibly high clock speed CPUs with a minimal number of cores and I can do that if I can offload the SSL operations to a secondary card.
So TCP, the packets come in, it’s basically a queue, and the faster I can pull things off of that queue, the faster I can satisfy the request, the lower my total object latency time is.
And unfortunately, that correlates very, very tightly with CPU speed, not so much with the number, of course, that I happen to have. That’s sort of the area I want to go into. CDNs largely compete on latencies. That’s sort of why we were expecting people to use us. That’s our value proposition. Anything I can do to increase even a millisecond is a major, major win.
Yes, sir.
Q. Yesterday it was mentioned that one of the criticisms was: you can still have Head-of-Line blocking in HB2, and I’m wondering if you can explain the difference or what the, what he was mentioning there versus no Head-of-Line blocking that the protocol is supposed to –
A. That was an excellent, excellent point, and one of it is: NGINX has worker processes, so I’m sure Valentin can fill in more details if I get something wrong. But you have multiple worker processes. That worker process now has to handle the HP2 pipeline.
Within that one pipeline, those HTTP/2 requests, those can be interleaved. There’s no Head-of-Line blocking within that pipeline, but the pipeline itself can block. So if it’s stalled out on something, then forget about. It doesn’t matter how many things are queued up within the H2 connection, those are all now blocked because the process itself is blocked.
Q. Maybe one small question.
A. Yes, sir. One small answer.
Q. (Continued) Why is many connections a performance problem?
A. Each connection chews up a fair amount of memory. One of the things that we found, especially looking at comparing different SSL terminators, was that the amount of memory each connection gets reserved – if you have too many connections, this winds up blowing past a reasonable amount of memory assigned to it.
In a given caching server, we have to reserve some amount of the memory for the actual objects we’re interested in, and we have to use some amount for the networking stack. We tend to push our buffers really, really big because again, we’re really interested in performance and we want to make sure that our server itself, and certainly our network, is never the bottleneck. And as a result, we allow connections.
We allow our congestion windows to run up really, really high and we allow our TCP windows to go incredibly, incredibly big, and unfortunately that just chews up constant, huge amounts of memory. And it correlates with a number of connections. So the fewer connections we have, the less memory we use, the more connections we have, the more memory we need.
Q. (Continued) So it’s about memory or it’s about cache?
A. Well, those are the same answer, right? Because we use it for both purposes.
Q. (Continued) And maybe another small question here is: when you talk about encryption, you talk about symmetric encryption. So modern processors, they do have specialized instructions for that. Why do you think the specialized card will do better?
A. So one of the things you’re referring to is Intel’s AES-NI instructions – a little quick side story about that is: I’ve actually taken receipt of servers which, for some bizarre reason, had the AES-NI disabled in the BIOS.
I’ve actually made the mistake of putting in servers which I thought were configured correctly but came from the manufacturer incorrectly configured, and I was wondering why is my CPU load so high? What’s wrong with these servers?
And it took a little bit of, diving down into it to realize that the reason why was the AES-NI instructions had been disabled, and enabling that I very luckily was able to bring our CPU load back down and to what we were expecting to see.
And you can play some other games with these things, and the idea being: once again, we have a queue where the content request comes in, it goes into a queue. It has to get satisfied.
It goes back out and anything that we can do to decrease the cycle time of this, to increase our throughputs through that queue is of tremendous interest. So that’s sort of a very broad answer to why we would be interested in looking at a card. Sorry, there’s one over here, then we’ll come back over here.
Q. You guys use the H2 to deliver videos and if so, what’s the performance like?
A. Excellent question. So I’m going to talk from the MaxCDN side from the legacy business that’s used to deliver – largely, I negotiate either HLS or HTS content, so we’re not using it for a lot of others.
Little quick side note is: at one point, we took advantage of inter Nexus slicer module and wrote a little package we call Ranger which is designed to assist with video delivery by taking up a very large video mp4 file and slicing it into smaller chunks which we can then deliver and taking advantage of range request.
So from that perspective, we haven’t really noticed a major – we’re not getting complaints about it. When you actually start hitting it with benchmarks, it performs well enough that it’s not the bottleneck. So again, the protocol here, at least for us, is not yet the bottleneck for this. We have bottleneck cells for within the system, so we haven’t really noticed a difference.
Yes, sir.
Q. So obviously it’s trivial to look at the H1 wire protocol through just Netcat or TCP dump or something like that like – really ASCII protocol – and see what’s going on. You can’t do that with H2 because it’s binary and because it’s encrypted because browsers are forcing encryption.
Have you guys done any work on building Wireshark plugins or doing any custom work to do any wire debugging for your inbound traffic?
A. Excellent, excellent question. We haven’t monkeyed much with a Wireshark for that particular purpose. Obviously we use it elsewhere tremendously. However, there are ways – if you have the key, then you can go ahead and there are ways to use Wireshark to use that key to go ahead and decrypt the contents that are contained within the packet.
In practice, I – this a little side note: when I’m using Wireshark I tend to – if I’m doing a TCP dump, I tend to set the snap links really, really tiny because I actually don’t want the content. I’m trying to find out: what’s the metadata? How is it actually performing? Am I dropping packets? Where is the delay actually occurring? Am I getting timeouts? And so this is – fortunately, metadata is not encrypted, and so that stuff I can actually read. But no, no, we haven’t played much with it yet. Yes.
I believe I have one minute left, so I think that’s enough time for one or two more questions, if I can hold myself to a fast answer.
Q. Hello. A lot of HTTP queries go through transparent proxies on the Internet from various services. Do you know which percentage of those might support HTTP/2 or whether that’s a problem. Will it result in a lot of downgrading to HTTP/1?
A. Excellent, excellent question. You’re absolutely right. That’s the way that, in practice, this is what we have to do with NGINX. Yes, if NGINX is that transparent proxy, you can only do H2 on the upstream side my mod_proxies wanted.
There will be a lot of these games going where we have to switch the protocol version, but luckily since the object is the same, we can bounce it across the protocols. At least the end user still gets the exact same content, he gets what he requests.
But, you’re absolutely right, it’s going to be stuck passing through multiple protocol versions which means that you have to now optimize for those middle layers as well as for the edge layer.
And so when you start looking at how do I want my topology to look like and how can I take advantage of things like keep lives or, you know, again very big TCP buffers that I can get very, very fast responses between my mid-tier level and the edge, then you know, maybe you can ameliorate some of the issues.
But you’re absolutely right, you are going to see protocol renegotiations going on and you’re going to see different connections.
Q. (Continued) Do you have any numbers to dish out, like: if you had 10 million unique HTTP users, you know, which percentage would actually manage to get HTTP/2 from end-to-end and which percentage would need to be downgraded?
A. End-to-end: anybody who’s talking directly to an NGINX origin can negotiate the full HQ connection. So you’ve actually kind of asked two questions which you are going after, one of which is what percentage of websites actually offer H2 negotiation right now?
And that’s surprisingly small. 20, 30, 40 percent, if that. This is from Google Statistics. However, the other question you’re asking is if I can go end-to-end because CD ends are so common and served so much of the world’s traffic.
In practice, the answer is a very, very tiny percentage, is actually fully end-to-end, because most guys are flowing through a transparent proxy and there you can only do, or for most guys, you’re only gonna be able to do the HQ termination from the edge on. Everything else internally is going to be H1.1.
I’m sorry, my time is out. I’m very happy to continue questions. I hope that you’ll find me after this and I hope that you’ll have a wonderful day at this conference. Please, please, please continue the argument. Very, very happy to keep talking and we’ll kind of go from there. Thank you.
This transcription blog post was adapted from a live presentation at nginx.conf 2016 by Nathan Moore. This was the third of three parts of the adaptation. In Part 1, Nathan described SPDY and HTTP/2, discussed proxying under HTTP/2, and summarized HTTP/2’s key features and requirements. In Part 2, Nathan talked about NPN/ALPN, NGINX particulars, benchmarks, and more. This Part included the conclusions, criticism of HTTP/2, closing thoughts, and a Q&A.
The post HTTP/2 Theory and Practice in NGINX, Part 3 appeared first on NGINX.
Leave a Reply