HTTP/2 - The New IoT Protocol?

On today's Internet, HTTP is the most relevant  application layer protocol - as it is the foundation of the World-Wide Web, and therefore the basis of most of the Internet's economic value.

For the Internet of Things, there is an ongoing discussion about the possible need for other application-layer protocols. Typically, proposals for such protocols are all based on either UDP/IP or TCP/IP, the core network and transport layer protocols of the Internet. But let us first take a look at HTTP.

HTTP/1.1 is the protocol that allows browsers to request Web pages from a Web server, usually represented as HTML documents. This is the Web as we all know from direct experience.

However, the design of HTTP is in no way tied to HTML documents, nor to presentation purposes in general, nor to clients that interact with humans. In fact, HTTP is also the standard protocol for the "programmable Web": Web services that typically operate with non-visual data represented in XML-based, JSON-based, or other formats. This decoupling from specific UI-oriented formats makes HTTP applicable to many IoT applications as well (or Web of Things applications, if you will).

HTTP supports the REST architecture principles. This means that

• A service provides a set of resources, which can stand for any kind of "thing". For example, a device service may expose resources for the configuration of the device, a sensor of the device, the most recent temperature sample taken by the sensor, the average of the samples taken by the sensor over the last hour, a subscription where other services may register for getting temperature updates, etc.

• When transferred via a request or response message, a resource is encoded in a representation. Multiple representations may be supported for a resource. For example, the current temperature value may be represented in a textual JSON format or in a compact binary representation.

• A resource is identified by a uniform resource identifier (URI). For example, such URIs may look like this:

• example.com/reactor-17/configuration
• example.com/reactor-17/sensors/sensor-3
• example.com/reactor-17/feeds/current-temperature
• example.com/reactor-17/feeds/average-temperature
• example.com/reactor-17/subscriptions/temperature
• example.com/plant-overview/feeds/max-temperature
  

• HTTP defines a small set of request operations (verbs). The most important are:

• GET e.g. for reading the current temperature
• PUT e.g. for updating the device´s configuration parameters
• POST e.g. for adding a subscriber to a subscription resource
• DELETE e.g. for removing such a subscription again
  

• Metadata, in the form of HTTP headers, is used to tell the message receiver how the message content is to be interpreted. HTTP headers are used to determine the best representation that both client and server understand for a particular resource. This protocol negotiation feature is important for avoiding a tight coupling of client and server, by allowing each of them to evolve their support for new representations over time, without breaking compatibility. Another important use of HTTP headers is for indicating how long a message may be cached somewhere between client and server. For example, a temperature sample may indicate in its metadata how many seconds the sample can still be considered valid and can thus remain cached. This kind of caching is a key reason why the Web has been able to scale so incredibly, even though many resources are frequently polled by large numbers of clients.

• Servers are stateless in that they do not store any client-specific application state between client requests. This means that a server need not keep track of clients, and in particular it does not have to worry about failed clients. On the other hand, if a client's request to a server fails, the client can simply repeat the failed operation, at least for the idempotent operations GET, PUT and DELETE. In this way, the REST principles make it easy to avoid inconsistent states, and thus achieve robustness, even when clients and servers may fail anytime.

• Resources may be hyperlinked. For example, a representation of a temperature subscription may contain an URI for a resource specifying the schema of the temperature data that is published by the device. Hyperlinks can be used to enable discovery of a device's resources, starting with some well-known service resource (e.g. example.com/). In practice, the set of provided resources, along with the supported representations and HTTP verbs, is often documented and published (REST APIs), whether or not the resources are also discoverable at runtime.

Most modern cloud services expose REST APIs based on HTTP - either to other cloud services or to apps running on "smart screens" like PCs or smartphones (the green edge in our IoT Triangle).

Facebook, Twitter etc. have shown that the REST architecture and HTTP can scale to hundreds of millions of users for a service. Moreover, HTTP has proven suitable for an incredibly broad spectrum of applications. Therefore it is only natural to reuse the existing Web know-how and infrastructure also for IoT applications, as long as there is no strong reason for using some other protocol.

Whether HTTP can further scale to IoT applications with potentially billions of devices, or whether other protocols will be needed, is hotly debated today. Usually, the arguments boil down to performance and scalability for an expected future with many billions of Internet-connected devices.

Other considerations are the buffering of data when the receiving end-points are offline (e.g. a device that is sleeping most of the time to conserve power), security, support for complex information models, etc. Different industries may favor different protocols (e.g., OPC UA in industrial automation), and even vendor-specific protocols may have a market impact (Apple's HAP, i.e. HomeKit Accessory Protocol is a candidate, for home automation applications in the Apple ecosystem).

So inevitably, other protocols have been positioned to compete with HTTP (I am not talking about edge network protocols here, just about protocols for the link between edge networks and cloud services - the red link in the IoT Triangle). These are e.g. MQTT, AMQP, XMPP, OPC UA, MTConnect or DDS.

In most applications where HTTP is ruled out for IoT today, this is because of performance considerations. Some aspects of HTTP can indeed be drawbacks in terms of performance. Frequent opening and closing of TCP connections is expensive, especially if transport layer security is used. Keeping open multiple TCP connections is expensive. Waiting for a response after each request before sending the next request often unnecessarily penalizes performance. The textual encoding of HTTP headers incurs unnecessary overhead for parsing. While the current version 1.1 of HTTP allows keeping open a connection and reusing it for multiple request/response round-trips, the other issues remain.

As the REST architecture principles are attractive for any kind of distributed system, not just global large-scale systems like the Internet, it is no surprise that a light-weight alternative to HTTP for embedded systems and sensor networks has been proposed. It is called CoAP. It can be regarded as a more efficient encoding of a subset of HTTP, with a few added features, such as the OBSERVE verb that allows to subscribe to notifications from a resource. As it has been designed explicitly with conversions to and from HTTP in mind, it is possible to develop generic gateways.

CoAP appears attractive in particular for use between smart objects, within low-bandwidth edge networks. It is less clear how much sense it would make to use CoAP even across the Internet at large (the red edge in the IoT Triangle). The necessary infrastructure investments into CoAP proxies, JavaScript libraries, browser support etc. may make it more attractive to keep CoAP traffic within edge networks, and convert such traffic to and from HTTP in the gateways.

While CoAP was developed with embedded systems as targets, the mentioned overhead of HTTP has also been motivating another group to look at alternatives: large cloud companies such as Google, Twitter and Facebook. It is in their best economic interest to minimize the load on their servers. The result of this effort is a specification for HTTP/2, based on a Google proposal called SPDY.

As of today, HTTP/2 is considered "done" and has been formally approved, so that it can become an official IETF RFC! From now on, there should only be editorial changes of the specification.

HTTP/2

• is fully compatible with HTTP/1.1, in the sense that an HTTP/1.1 request can be converted in a "lossless" way into an HTTP/2 request and vice versa,
• includes a multiplexing protocol that allows multiplexing multiple connections onto a single TCP connection,
• uses an optimized header encoding, which reduces both the necessary network bandwidth and also the processor cycles needed for parsing,
• has a core specification that allows for small-footprint implementations on  microcontrollers, while the more advanced capabilities (e.g., request priorities) are optional,
• will be broadly supported by all major cloud and browser vendors, with test implementations already available.
  

For these reasons, I think that HTTP/2 may quickly become a very attractive choice for an IoT application layer protocol for the red edge of the IoT Triangle, at least whenever a client / server approach is considered appropriate. It retains all the architecture benefits of REST, fits in well with the existing HTTP ecosystem, and reduces overhead both regarding bandwidth and processor cycles.

But most importantly: the potential customers for Limmat we have talked to all have existing HTTP-based REST backend services. HTTP/2 will by far be the most natural and painless upgrade path for them if and when HTTP/1.1 should become a problem.

This is why we are focusing on HTTP/2 as our default protocol for IoT in the Limmat project (along with HTTP/1.1). Of course, specific requirements, desired architecture qualities and technical constraints need to be taken into account for each project individually, and may lead to different decisions. For this reason, we would also consider supporting other protocols for Limmat gateways if and when this is needed. But as a general-purpose IoT protocol, our bet is on HTTP/2 for the coming years.

Here, another developer's take on HTTP/2 as an IoT protocol. So the ball has started rolling... And when will your device speak HTTP/2?

Cuno Pfister, Oberon microsystems AG

PS

Next week, I'll give a talk (in Session 24/II) at the largest embedded systems trade fair in the world, Embedded World in Nürnberg, Germany. Guess what IoT protocol I'll be talking about?

UPDATE

I'll collect links here that refer to HTTP/2 and the IoT. Please send me an email to pfister@oberon.ch if I am missing a good one. Thanks!

Already mentioned above, here the first such reference I've seen:
http://robbysimpson.com/2015/01/26/http2-and-the-internet-of-things/

Tim Kellog's take on how MQTT could be replaced by HTTP/2: http://timkellogg.me/blog/2015/02/20/can-http2-replace-mqtt/

"HTTP/2 lends itself surprisingly well to the pub/sub pattern, despite being designed for request/response."

Dominique Guinard's first blog post on HTTP/2 for the Web of Things: http://webofthings.org/2015/10/25/http2-for-internet-of-things-1/