Use the Accept Header in Your XHRs
An example of a common scenario: You are building a browser-based app for a grocery store. The app shows a list of all the valid coupons this week. Your browser-based app needs to load all of the store’s coupons by requesting the coupon data from the server. The server is capable of returning the coupons in JSON or XML format. Your browser app only knows how to deal with a JSON response and you need to indicate that in your request. You consider several options to indicate the desired response format.
Your first option, and probably the most popular option in use today, is to use a format extension in the request URL.
var xhr = new XMLHttpRequest();
xhr.open('GET', '/coupons.json', true);
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
if (xhr.status == 200) {
var data = JSON.parse(xhr.responseText);
// do something with data
} else {
//handle errors
}
}
};
xhr.send();The resulting HTTP request
GET /coupons.json HTTP/1.1
Accept: */*
Host: api.nestorsmarket.com(Note that the above Accept header says the browser can handle any response format. That’s a lie as you know your app can only handle a JSON response.)
Your second option, which is very similar to the first option, is to use a format parameter in the query string of the request URL.
var xhr = new XMLHttpRequest();
xhr.open('GET', '/coupons?format=json', true);
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
if (xhr.status == 200) {
var data = JSON.parse(xhr.responseText);
// do something with data
} else {
//handle errors
}
}
};
xhr.send();The resulting HTTP request
GET /coupons?format=json HTTP/1.1
Accept: */*
Host: api.nestorsmarket.com(More lying.)
Your third option, and probably the least commonly used option, is to use the Accept header.
var xhr = new XMLHttpRequest();
xhr.open('GET', '/coupons', true);
xhr.setRequestHeader('Accept', 'application/json');
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
if (xhr.status == 200) {
var data = JSON.parse(xhr.responseText);
// do something with data
} else {
//handle errors
}
}
};
xhr.send();The resulting HTTP request
GET /coupons HTTP/1.1
Accept: application/json
Host: api.nestorsmarket.comThe first and second options are ugly hacks. These hacks were only ever used because, in the past, the third option didn’t work or server-side programmers did not know how to deal with it. You may be asking, “what is so terribly wrong with the first and second options?”
Intermediary Cache
Imagine you have two types of client apps that want to access the same data. The first client type is a browser-based app that can only handle JSON responses. The second client type is a native iOS app that can handle JSON or XML responses. Also imagine there is an intermediary caching server somewhere between the clients and your web server. A browser-based client makes an XHR request for the JSON format using the format extension technique.
GET /coupons.json HTTP/1.1
Accept: */*
Host: api.nestorsmarket.comThe cache on the intermediary server is empty so the request is passed on to the web server. The web server responds with
HTTP/1.1 200 OK
Cache-Control: max-age=86400
Content-Type: application/json
[
{
"upc": 827454737,
"description": "20% OFF Rib Eye Steak!"
},
{
"upc": 546373635,
"description": "Save $3.18 on Rocket Fuel french roast coffee!"
}
]The caching server sees from the Cache-Control header that this response is cacheable for one more day so it caches it to speed up responses for the remainder of the day. It passes the response back to the client.
Soon after, the iOS app makes a request. It uses the Accept header technique indicating it can handle both data formats.
GET /coupons HTTP/1.1
Accept: application/json, application/xml
Host: api.nestorsmarket.comWhen the request arrives to the caching server, the server checks to see if it has a response cached for /coupons. There is nothing matching in the cache. What a pity. There is a match for /coupons.json but that does not match /coupons. The caching server has no idea how to interpret these strings. It checks for exact matches only.
Because the caching server cannot make a match, the caching server must again pass the request to the web server. The web server must again generate the response. The caching server will again cache the response. This wastes end user time, bandwidth, CPU time on the web server, and storage space on the caching server.
Suppose that instead, the browser-based app had originally made this request indicating only one acceptable response format.
GET /coupons HTTP/1.1
Accept: application/json
Host: api.nestorsmarket.comThen when the iOS app made its request, the caching server would see it has a match in the cache and could respond quickly to the client without bothering the web server.
The intermediary cache is only one compelling example demonstrating that if we use HTTP as it was designed, many things are improved.
Opera Bug
My exploration into this topic started way back in 2008 with the discovery described in the email to the authors of the XMLHttpResponse specification.
From: "Peter Michaux" <petermichaux@gmail.com>
To: public-webapi@w3.org
Date: Wed, 16 Apr 2008 16:49:44 -0700
Subject: XHR setting headers
The XMLHttpRequest spec says "The setRequestHeader() method appends a
value if the HTTP header given as argument is already part of the list
of request headers."
This is fine but what is a problem is whether or not a new
XHMHttpRequest object has any default headers. I was trying to use the
Accept header a few days ago and I wanted to have only
Accept: application/json
but Opera has a default header
Accept: text/html, text/xhtml, etc
so my application/json was appended to the front of that list which
makes my Accept header useless as part of the client-server
communication. The server thinks that the client knows what to do with
text/html. My JavaScript certainly does NOT know what to do with
text/html. My JavaScript only knows how to handle application/json.
I think all XMLHttpRequest headers should be specified as blank when
the object is created. Then the JavaScript can add any headers it
needs to add. If, when the call to send() occurs, some essential
header(s) is missing the XHMLHttpRequest object should add these
automatically but only according to specified behavior.
PeterTo make the problem described in my email above more clear, in Opera 9.27, I would do this.
var xhr = new XMLHttpRequest();
xhr.open('GET', '/coupons', true);
xhr.setRequestHeader('Accept', 'application/json');
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
if (xhr.status == 200) {
var data = JSON.parse(xhr.responseText);
// do something with data
} else {
//handle errors
}
}
};
xhr.send();The resulting HTTP request would be this.
GET /coupons HTTP/1.1
Accept: application/json, text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg,
Host: api.nestorsmarket.comYou can download old versions of Opera to try for yourself.
The final email in the thread of discussion was from the author of the XMLHttpRequest specification (who’s email address happened to end in opera.com.)
From: "Anne van Kesteren" <annevk@opera.com>
To: "Julian Reschke" <xxxx.xxxx@xxx.de>
Cc: "Peter Michaux" <petermichaux@gmail.com>, public-webapi@w3.org
Date: Tue, 27 May 2008 14:14:02 +0200
Subject: Re: XHR setting headers
On Tue, 13 May 2008 09:24:42 +0200, Julian Reschke <xxxx.xxxx@xxx.de> wrote:
> Anne van Kesteren wrote:
> > On Mon, 12 May 2008 17:24:16 +0200, Julian Reschke <xxxx.xxxx@xxx.de> wrote:
> > > Well, we just heard from people complaining about XHR implementations
> > > pre-filling request headers, and thus causing clients to create broken
> > > content-type headers (because of the append functionality).
> >
> > Actually, that was specifically about Accept and it seems to be simply
> > an Opera bug per the specification.
>
> So does the specification say which headers may be prefilled by the
> implementation, and when?
It is now made clear that user agents can only control the headers that
authors can not set and can control a limited set of headers if the author
has not set them (e.g. the specification allows the author to control
caching himself, but if he does not do that the user agent can do it).You can read the whole email thread in the mailing list archives.
The related parts of the XMLHttpRequest specification are in The setRequestHeader() method and The send() method sections where the "author request headers" are discussed.
I was happy with the outcome. The specification was improved. Of course, at that time, it was still 2008 and the buggy versions of Opera were still out there in the wild being used. I thought, "Great! In 10 years I can start using the Accept header!!" Well, it is almost 8 years now and I think it is time to start.
The first version of Opera to ship with this bug fixed was Opera 10.50 beta 1 which was released on February 11, 2010. That is only 6 years ago but Opera has released new versions and has fallen in popularity. For one site I monitor, in the year prior to the release of Opera 10.50 beta 1, 1 out of 125 sessions was from the buggy versions of Opera. Now, only 1 out of 17241 sessions comes from the old buggy versions of Opera. Low enough for me.
I’m not aware of any other browser that has had this bug or any other bug preventing the proper use of the Accept header...and I’ve been looking. All of the following browsers worked.
| Operating System | Browser |
|---|---|
| Windows XP SP2 | Internet Explorer 6.0.2900.2180.xpsp_sp2_rtm.040803-2158 |
| Internet Explorer 7.0.5720.13 | |
| Firefox 42.0 | |
| Chrome 48.0.2564.11 m | |
| Opera 10.50 Beta 1 | |
| Windows 7 | Internet Explorer 8.0.7600.16385 |
| Firefox 14.0.1 | |
| Chrome 47.0.2526.111 m | |
| Internet Explorer 9.0.8112.16421 | |
| Firefox 35.0.1 | |
| Windows 10 | Internet Explorer 11.63.10586.0 |
| OS X 10.4.11 | Safari 4.0.5 (4531.22.7) |
| Firefox 3.6.28 | |
| OS X 10.11.3 | Firefox 44.0.2 |
| Chrome 48.0.2564.116 | |
| Safari 9.0.3 (11601.4.4) | |
| Opera 10.50 Beta 1 | |
| Opera 11.11 | |
| Opera 35.0.2066.68 | |
| Debian 7.4 | Iceweasel 17.0.10 |
| Ubuntu 15.10 | Firefox 44.0.2 |
| iOS 6.1.6 | Safari 6.0 |
| iOS 9.2.1 | Safari 9.0 |
| Chrome 48.0.2564.87 | |
| Android 4.4.2 | Internet 4.4.2.I747MVLUFFOB3 |
| Chrome 47.0.2526.83 | |
| Windows Phone 8.1 | Internet Explorer Mobile 11.0 |
Even good-old Internet Explorer 6 allowed successful control and setting of the Accept header in an XHR!
Conclusion
In the browser world, it takes a long time for old bugs to go away. I think it is now time to ditch the format hacks and take advantage of the proper use of HTTP Accept header.
Do you already use the Accept for your XHR requests from browser-based apps?
Have you tried to use Accept for your XHR requests from browser-based apps and had problems?
I’d like to know about your experiences.
Comments
Have something to write? Comment on this article.