Building the YUI Browser History Manager

By Julien LecomteFebruary 22, 2007

Julien LecomteAbout the Author: Julien Lecomte is an engineer on Yahoo!'s DHTML Evangelist team, a group that provides architectural assistance to Yahoo! developers on the design and implementation of rich interactions in the browser. He has worked extensively on Yahoo!'s new Mail application. Julien is the author of a new, experimental YUI component, the Browser History Manager, which was released earlier this week. In this article, he explores the broad problems associated with controlling history and state in the browser during rich interactions and he lays out the multifaceted approach he took to building a YUI history manager with support for all A-Grade browsers. -Eric Miraglia

Most Ajax Applications Break Standard Browser Functionality

Our industry has made significant progress in the past few years. Web pages have become vastly more sophisticated in terms of appearance and responsiveness. Some companies have pushed the envelope so far that their web page could easily be mistaken for a native application (check out Scalix Web Access or the new Yahoo! Mail, to name just two examples). In the meantime, that same rush to produce the most stunning web experience has led us to forget some of the most basic principles of the web. Our pages run inside a web browser, and even if modern web browsers can now be used to support very complex applications they are mostly used for what they were originally intended to do: browsing the web.

This dual purpose creates a lot of confusion among users. Some of the basic features of browsing, such as the use of back and forward buttons, or the ability to bookmark a page, rarely achieve the desired result in a dynamic web application because dynamic changes to a page do not naturally register themselves with the browser's native history engine. Moreover, there is a difference between "backing out changes" in an application — the undo command, ctrl-z — and retracing one's steps in a navigation context. Many web-application developers find themselves puzzling over what role the back button should play when they make the transition from writing web sites to writing web applications.

Ignorance Is Bliss...

Only recently did I find myself impacted in my own work by the absence of a good solution for managing navigation history and session state in a rich application. At the time, I was working on a very simple web site for my father's new venture. After "ajaxifying" the site, the navigation became a lot faster. However, a coworker of mine pointed out to me that the back and forward buttons were not working anymore, and that individual sections of the web site could not be bookmarked. In my case, this problem was a deal-breaker; I absolutely had to find a solution. After a bit of time digging around on Yahoo! Search, I discovered a long and impressive library of articles from developers attempting to solve this problem.

Currently Available Solutions

One of the most important articles in this collection is Brad Neuberg's piece on how to handle bookmarks and the back button in Ajax applications. Neuberg's Really Simple History (RSH) framework works well on most browsers, but unfortunately it does not work on Safari — considered an A-Grade browser in Yahoo!'s Graded Browser Support approach. Another disadvantage of the RSH framework is the need for a server round-trip every time a new entry is added to the browser history on Internet Explorer. Nevertheless, Neuberg's work, which pushed back the frontier on this difficult issue, made me believe that my quest was not hopeless.

I then read Mark Schiefelbein's article on developing Ajax applications that preserve standard browser functionality. His article mainly looks at the Backbase Ajax engine and how it provides support for the back/forward buttons. The Backbase solution works well, but I was once again confronted by the issue of browser support; their demos do not appear to work on Opera and Safari. Moreover, Backbase is a commercial product, and I wanted a solution that would play well with the open-source tools I was already using, and in particular one that could be included as part of YUI.

Creating a Browser History Manager for YUI

Given that background, I decided to tackle the problem on my own. After a little bit of tinkering, I was able to put together a demo that worked on Firefox and Opera using the URL fragment identifier. The idea is very simple: Change the URL fragment identifier, using the window.location.hash property, to add a new entry to the browser history and periodically check the value of the hash to detect history changes. In addition to solving the issue with the back/forward buttons, this also allows the application to be bookmarked in a specific state. This approach provides support for Firefox (and other Gecko-derived browsers), Opera, and for recent nightly builds of WebKit (the open-source rendering engine underlying Apple's Safari browser). Unfortunately, the window.location.hash approach does not work fully on current versions of Safari — and, more importantly, it does not work at all on Internet Explorer. It might serve as part of the solution, but it obviously couldn't constitute a unified approach across browsers.

One of my colleagues at Yahoo!, Dav Glass, is an accomplished Mac geek who has a great deal of experience solving hard problems on Safari; Dav volunteered to help tackle browser history in the current versions of Safari. With Safari, there were two main issues to overcome:

  1. We needed to identify when back forward interactions occurred;
  2. When those interactions occurred, we needed to identify where we were in our stack of saved states.

After reading David Bloom's post on using document.body.scrollTop, we started thinking about what other properties might change during back/forward navigation. So we wrote a script to dump all the window.* variables and found out that, in Safari, window.history.length changed each time the browser history changed — not that it should change, but it clearly did. Armed with this knowledge, we began by setting up our own "history" array to track the changes keyed off of the window.history.length change. This approach gave us the two pieces of information we needed: We could tell when the user hit "back" or "forward", and we could tell where they were when they did so. (Note: The fact that window.history.length changes when the user hits the back button is a bug that has been fixed in WebKit. Therefore, this approach requires "forward compatible code" for Safari by using the URL fragment identifier as the preferred, long-term method of detecting history changes; we only fall back to relying on the history object itself when it's necessary to do so. Over time, the window.history.length hack will become less and less important.)

All of that still left us in need of a solution for Internet Explorer. On IE, we had to adopt a radically different approach. We create a hidden IFrame that we initially point to an existing document on the web server. Writing to the IFrame using document.write adds a new entry to the browser history. The HTML we write looks like this ("current_state" is replaced by the actual state of the application):

<html>
 <body>
  <div id="state">current_state</div>
 </body>
</html>

We then periodically (every 50ms) look at the content of the IFrame — especially the text inside the inner <div> element — to detect history changes. Once we detect a change to the history, we change the URL fragment identifier using top.location.replace(...); this technique allows the application to be bookmarked in a specific state without impacting IE's history stack.

In sum, the YUI Browser History Manager uses three different techniques to provide support across the A-Grade browsers:

Firefox (and Gecko Browsers), Opera and Nightly WebKit Add history entries and detect history changes by using the URL fragment identifier (window.location.hash)
Safari (current) Add history entries by using the URL fragment identifier (window.location.hash), and detect history changes by checking window.history.length
Internet Explorer Add history entries by writing to an IFrame, detect history changes by checking the content of the IFrame, and add bookmark support by setting the URL fragment identifier using top.location.replace(...)

The Problem Of The Initial State

Another concern in creating the Browser History Manager was to provide a mechanism for identifying the initial state of the application when the page/application loads via the back button (eg, the user has navigated away from the application to another site, then returns to our application using the back button). Indeed, from a programmatic standpoint, returning to a page using the back button or the forward button, or simply loading the page for the first time, are identical scenarios. However, the state of the application may need to be different depending on the situation. For example, if you return to a page using the back button, the state of the application should be the last state visited by the application before you left the page; this will likely be different from the initial state when the page first loaded during this browser session.

We approach this problem by storing state information in a hidden form field because the value of form fields persists across pages during the entire browser session. If we set the value of our form field to the current state of the application every time the application changes state, we can then initialize our page appropriately when the user returns to our application after having interacted with it beyond its initial state.

Intended Use: "Back" vs. "Undo"

The main role of the YUI Browser History Manager is to persist the history of states visited by a web application within a single browser session; in this week's initial (and experimental) release we believe that it handles this problem well. However, there are limits to what these techniques allow you to do. The simple fact that URLs have a maximum length limits the amount of state data that can be stored (that limit is browser dependent — it is 2083 bytes on Internet Explorer, a bit more on other browsers). Imagine for example a TreeView Control containing thousands of nodes located at several levels of depth. If you wanted to represent the collapsed/expanded state of each node as a string, you would quickly run out of room; the information space provided by the URL doesn't have the capacity to support that level of detail. More complex web applications can have a theoretically infinite number of states.

Given these limitations, and given the user's expectation of what a "back button does" as compared to an "undo" command, we recommend thinking about the Browser History Manager as a tool for managing high-level navigational states: what screen is displayed, what tab is selected, what month is being displayed in a dynamically-generated calendar, etc. Because we cannot encode an infinite number of states, the back button cannot act as an "undo" command that takes snapshots of session states at high resolution throughout an interaction — and it shouldn't try to do so. This sometimes subtle difference should drive the way you plan to make use of the YUI Browser History Manager.

Feedback

Due to the complexity of this problem space as it applies to the A-Grade browser family, we're releasing the YUI Browser History Manager as an experimental utility at this point. We look forward to your feedback (here, or in our forums), and we hope to improve this implementation over the coming months to make it a genuinely robust, reliable, and cross-browser utility that solves one of the most vexing and problematic challenges involved in building rich internet applications.