The AJAX response: XML, HTML, or JSON?

Since my last AJAX project I've increasingly been wondering about the "ideal" output format for the AJAX response. Once you've succesfully fired an AJAX request, what sort of response should the server give? An XML document? An HTML snippet? A JSON string which is converted to a JavaScript object? Or something else? In this entry I'd like to discuss the three formats, with examples, and ask you which format you've used in your practical AJAX applications.

(This article has been translated into Spanish.)
When you receive an additional bit of data for your AJAX application, you should start up a script that incorporates this extra data into your single-page HTML interface. Of course the form of the script heavily depends on the format of the data you've received. Should you search an XML document for specific nodes and copy their text to the HTML? Or did you receive an HTML snippet that should be added to the page "as is"?
In my last project I received some data as XML documents and some as HTML snippets, and they needed different kinds of scripts to write the data to the page. Both formats, and both kinds of scripts, have their advantages and disadvantages.
After I'd finished the application I delved a bit deeper into JavaScript Object Notation, invented by Douglas Crockford and recently chosen as the default output format for most Yahoo services, and I think I like it, although I've never yet used it.
I'm left wondering: which format is the best? Which format do you think is best, or at least most useful in a practical AJAX environment?

Example

As an example of the three formats, let's take an AJAX-driven online bookstore. We ask for the JavaScript books they have in store, and by a staggering coincidence they have exactly those three JavaScript books I keep lying around on my desk. An AJAX request returns these three results, and you have to incorporate them in the HTML of your single-page interface.
I'll give examples of all the three formats, and a simple script to show the results in a <div id="writeroot">.

XML documents

The first and most obvious choice for an output format is the XML document. The original idea behind the XMLHTTP object was the importing of XML documents, so it's no surprise that most of the attention went to XML and it's still considered the default output format.

Example

The server returns this XML document:

<books>
 <book>
  <title>JavaScript, the Definitive Guide</title>
  <publisher>O'Reilly</publisher>
  <author>David Flanagan</author>
  <cover src="/images/cover_defguide.jpg" />
  <blurb>Lorem ipsum dolor sit amet, consectetuer adipiscing elit.</blurb>
 </book>
 <book>
  <title>DOM Scripting</title>
  <publisher>Friends of Ed</publisher>
  <author>Jeremy Keith</author>
  <cover src="/images/cover_domscripting.jpg" />
  <blurb>Praesent et diam a ligula facilisis venenatis.</blurb>
 </book>
 <book>
  <title>DHTML Utopia: Modern Web Design using JavaScript &amp; DOM</title>
  <publisher>Sitepoint</publisher>
  <author>Stuart Langridge</author>
  <cover src="/images/cover_utopia.jpg" />
  <blurb>Lorem ipsum dolor sit amet, consectetuer adipiscing elit.</blurb>
 </book>
</books>

We need this script to show the results in our <div>.

function setDataXML(req)
{
 var books = req.responseXML.getElementsByTagName('book');
 for (var i=0;i<books.length;i++)
 {
  var x = document.createElement('div');
  x.className = 'book';
  var y = document.createElement('h3');
  y.appendChild(document.createTextNode(getNodeValue(books[i],'title')));
  x.appendChild(y);
  var z = document.createElement('p');
  z.className = 'moreInfo';
  z.appendChild(document.createTextNode('By ' + getNodeValue(books[i],'author') + ', ' + getNodeValue(books[i],'publisher')));
  x.appendChild(z);
  var a = document.createElement('img');
  a.src = books[i].getElementsByTagName('cover')[0].getAttribute('src');
  x.appendChild(a);
  var b = document.createElement('p');
  b.appendChild(document.createTextNode(getNodeValue(books[i],'blurb')));
  x.appendChild(b);
  document.getElementById('writeroot').appendChild(x);
 }
}

function getNodeValue(obj,tag)
{
 return obj.getElementsByTagName(tag)[0].firstChild.nodeValue;
}

Rather a lot of code, as you see. Although the W3C DOM gives us full access to both the XML document from the server and the HTML document the data should be shown in, it doesn't give us an elegant, simple way of extracting exactly that data we need: we have to delve into the XML document time and again.
It's here that XSLT would come in handily, since this language is meant precisely to convert an XML document to another kind of XML, and since XHTML is XML, we can also use it to create Web page snippets. I haven't studied XSL(T) since 1999, though, and doubtless there are many minor compatibility issues that I'd have to solve before getting a workable demo. We'll leave XSLT for another time.

Advantages

The most important advantage of XML is that it's the most easily readable format for other humans.
A secondary advantage is that XML has been around for quite a while and that many developers are already accustomed to it. Saying "I'd like your server side script to return an XML document" won't cause raised eyebrows, while saying "I'd like the script to return a JSON object" might.

Disadvantages

The JavaScript required to insert the data into the HTML page is quite verbose. I wrote a little convenience function getNodeValue() to get rid of the most verbose and boring part of the script: reading out the text in an XML tag. Nonetheless the script won't ever win a beauty contest.

HTML snippets

The second, and maybe the most interesting, output format is an HTML snippet. Note that I call it a snippet, since we do not receive a complete HTML page. Instead, we get exactly that HTML that has to be inserted into our <div>.

Example

The server returns this HTML snippet:

<div class="book">
 <h3>JavaScript, the Definitive Guide</h3>
 <p class="moreInfo">By David Flanagan, O'Reilly</p>
 <img src="/images/cover_defguide.jpg" />
 <p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit.</p>
</div>
<div class="book">
 <h3>DOM Scripting</h3>
 <p class="moreInfo">By Jeremy Keith, Friends of Ed</p>
 <img src="/images/cover_domscripting.jpg" />
 <p>Praesent et diam a ligula facilisis venenatis.</p>
</div>
<div class="book">
 <h3>HTML Utopia: Modern Web Design using JavaScript & DOM</h3>
 <p class="moreInfo">By Stuart Langridge, Sitepoint</p>
 <img src="/images/cover_utopia.jpg" />
 <p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit.</p>
</div>

The script is extremely simple: just put the responseText in the innerHTML of the correct object and you're ready.

function setDataHTML(req)
{
 document.getElementById('writeroot').innerHTML = req.responseText;
}

Advantages

The script's simplicity is the most important advantage of this method.
In addition, this format offers interesting accessibility options. We could cleverly write the server side script to build a complete, accessible HTML page that can be shown to any device. If the request happens to be made by an AJAX script, the server side script would discard all HTML except for the search results, or the AJAX script itself would search for the results.
It's of course perfectly possible to create similar accessibility features when you're working with XML or JSON, but the HTML snippet format is the easiest one for this job.

Disadvantages

If the HTML snippet contains forms, or if the receiving HTML element is a form, this method give horrific errors in Explorer.
In addition, HTML snippets may become quite complicated. The example above isn't, but as soon as you want to use advanced CSS techniques that require more elements than strictly necessary, the snippet would have to contain extra <span>s or whichever elements you need. Thus the server side script that generates the HTML may become quite complicated.

JSON

The third method is JSON, JavaScript Object Notation. Personally I pronounce it as "Jason", so that yet another ancient Greek hero enters modern JavaScript development. (And please remember that Ajax's father Telamon accompanied Jason as an Argonaut. Jason was older, and on the whole more succesful, than Ajax)
The general idea is to deliver a bit of text (a string, really) which can be interpreted as a JavaScript object. Once it has arrived, you use JavaScript's eval() method to convert the string into a real JavaScript object, which you then read out.

Example

The server returns this JSON string:

{"books":[{"book":
  {
  "title":"JavaScript, the Definitive Guide",
  "publisher":"O'Reilly",
  "author":"David Flanagan",
  "cover":"/images/cover_defguide.jpg",
  "blurb":"Lorem ipsum dolor sit amet, consectetuer adipiscing elit."
  }
 },
 {"book":
  {
  "title":"DOM Scripting",
  "publisher":"Friends of Ed",
  "author":"Jeremy Keith",
  "cover":"/images/cover_domscripting.jpg",
  "blurb":"Praesent et diam a ligula facilisis venenatis."
  }
 },
 {"book":
  {
  "title":"DHTML Utopia: Modern Web Design using JavaScript & DOM",
  "publisher":"Sitepoint",
  "author":"Stuart Langridge",
  "cover":"/images/cover_utopia.jpg",
  "blurb":"Lorem ipsum dolor sit amet, consectetuer adipiscing elit."
  }
 }
]}

The script looks rather a lot like the XML script. It does the same things, it just reads out the data from another format. Here, too, XSLT might come in handy.

function setDataJSON(req)
{
 var data = eval('(' + req.responseText + ')');
 for (var i=0;i<data.books.length;i++)
 {
  var x = document.createElement('div');
  x.className = 'book';
  var y = document.createElement('h3');
  y.appendChild(document.createTextNode(data.books[i].book.title));
  x.appendChild(y);
  var z = document.createElement('p');
  z.className = 'moreInfo';
  z.appendChild(document.createTextNode('By ' + data.books[i].book.author + ', ' + data.books[i].book.publisher));
  x.appendChild(z);
  var a = document.createElement('img');
  a.src = data.books[i].book.cover;
  x.appendChild(a);
  var b = document.createElement('p');
  b.appendChild(document.createTextNode(data.books[i].book.blurb));
  x.appendChild(b);
  document.getElementById('writeroot').appendChild(x);
 }
}

Advantages

The most important advantage is that JSON circumvents JavaScript's same-source policy, if you import the JSON file as a new <script> tag. See Simon Willison's example for the gory details.
JavaScript does not allow you to access documents (be they XML or HTML) that come from another server. However, if you import a JSON file as a script tag you circumvent this problem, and any JSON data can be imported into any website. It depends on your business goals whether this is a Good or a Bad Thing, but right now it's the only data format that allows unrestricted access.
A secondary advantage is that scripts for JSON data are slightly simpler and slightly more in line with the rest of the JavaScript language than scripts for XML data.

Disadvantages

The most important disadvantage of JSON is that the format is very hard to read for humans, and that, of course, every single comma, quote, and bracket should be in exactly the correct place. While this is also true of XML, JSON's welter of complicated-looking syntax, like the }}]} at the end of the data snippet, may frighten the newbies and make for complicated debugging.

Your choice?

These are the three output formats you can use for getting AJAX data. Although I'd love to be able to say that one of them is "the best", I think choosing the right format depends on the circumstances, and not on any theoretical musings.
Nonetheless, let's take a stab at finding "the best" format. I have four questions for you:

1. Can you think of another output format?
2. Which output format did you use in a practical, commercial AJAX application? (Demos and the like don't count)
3. Will you switch to another output format in the future? If so, which one and why?
4. Can you think of other advantages or disadvantages of the three formats?

My own answers are:

1. No.
2. Mainly XML documents, a few HTML snippets.
3. I'm going to study JSON carefully and might switch to it for an unrestricted access application I have in mind. Nonetheless I feel that XML remains the best overall format for the time being, mainly because people are used to it.
4. I wrote down all advantages and disadvantages I could think of.