Performance » Yahoo! User Interface Blog (YUIBlog)

Next-Gen YSlow powered by YUI

July 18, 2011 at 9:17 pm by Marcel Duran | In Development, Performance | 4 Comments

A couple of weeks ago, Yahoo! announced YSlow for Mobile at Velocity 2011, bringing the power of YSlow performance analysis to the mobile world.

YSlow for Mobile works as a bookmarklet, making it possible to run on browsers other than Firefox (as an add-on) or Chrome (as an extension).

The YSlow architecture was partially redesigned to work cross-platform and YUI was the essential factor in make sandboxing, cross-browser abstraction and simple YQL access possible.

Sandboxing

In order to be embedded on a page without interfering with performance analysis and without messing with the page itself, YSlow is a bookmarklet that injects JavaScript and CSS into any page by leveraging the power of YUI sandboxing:

Bookmarklet URL:

javascript:(function (y, p, o) {
    p = y.body.appendChild(y.createElement('iframe'));
    p.id = 'YSLOW-bookmarklet';
    p.style.cssText = 'display:none';
    o = p.contentWindow.document;
    o.open().write('
        <head>
        <body onload = "
            YUI_config = {
                win: window.parent,
                doc: window.parent.document
            };
            var d = document;
            d.getElementsByTagName(\'head\')[0]
                .appendChild(
                    d.createElement(\'script\')
                ).src = \'http://d.yimg.com/jc/yslow-bookmarklet.js\'"
        >
    ');
    o.close()
}(document))

The code above:

creates an empty iframe;
appends it to the page body;
hides the iframe*;
gets its window handler;
writes into its content the body of the iframe;
this body is empty but has an onload event
the onload event defines how to inject YSlow JS:
- sets YUI_config, so win and doc points to the page being analyzed window and document respectively
- dynamically injects YSlow URL by creating a script element into iframe’s head

* the iframe is displayed by the time all YSlow presentation assets are loaded

This will place an iframe into the page being analyzed. This iframe will act as a sandboxed environment and YSlow will reside within it. Since the iframe is dynamically created without the src attribute, it will have access to its parent (the page being analyzed) because there’s no same origin policy violation happening there.

The YUI_config object is handy because it sets win and doc to the iframe’s parent (the page being analyzed), thus any new YUI instance will be bound to the parent document by default, wiring any call to Y.all and Y.one through Y.config.win or Y.config.doc from the YUI use callback.

YSlow’s presentation is handled by the iframe window and document references, allowing the YSlow main script to render the markup as well as fetch the external CSS within this iframe without conflicting with the parent page’s styles. YSlow scans the parent page in order to get all the components (images, scripts, links, background-images, flash, etc.) required for later performance analysis. This is done by accessing Y.config.win and Y.config.doc, since they refer to the parent page.

Cross-browser abstraction

Being a bookmarklet, YSlow for Mobile is supposed to work on any browser*. YUI abstracts cross-browser issues by normalizing browser differences, resulting in a clean, easy-to-read and maintainable codebase.

YSlow was not fully ported to YUI 3 — only the controller layer (from the upcoming App component) for now — but still, all DOM manipulation and event handling are done by the node and event modules. In future releases we plan to port more YSlow features to YUI 3.

* not all browsers are currently supported

YQL

YSlow analyzes pages by checking the HTTP headers for the components found on the page. HTTP response headers are not available in the page, hence those components need to be requested again in order for YSlow get the response header information. This could be achieved by requesting the list of component URLs through XmlHttpRequest (AJAX) but unfortunately due to same origin policy restriction, this is not possible unless all components are in the same domain as the page which is very unlikely.

A common workaround for same origin policy restriction is using JSONP, where an external server works as a proxy requesting the list of components URLs and retrieving their HTTP response headers on behalf of YSlow. Due to YSlow’s popularity and recent mobile performance analysis efforts, we’re expecting quite heavy traffic for the YSlow for Mobile bookmarklet. In order to support such traffic, YQL was the scalable solution adopted by YSlow through an open data table named data.headers, which retrieves the response headers and content for a given list of URLs while impersonating the user-agent to ensure the expected content is retrieved.

The YQL Query component does all the work of managing YQL queries while managing JSONP requests under the hood, making the YSlow controller code much simpler and easy-to-maintain.

Future enhancements: New YSlow for Mobile friendly interface

Currently the YSlow for Mobile user experience is the same as the desktop experience. Dealing with a long list of performance analysis data is not the best experience on small smart-phone screens. Since YUI also abstracts cross-device gestures, YSlow for Mobile will get a brand new mobile-friendly interface in future releases.

Performance of performance tool

YSlow for Mobile deployment was made carefully considering its performance impact on the load time of the page being analyzed. The YUI 3 modules used on YSlow were scrutinized to include only the modules needed to load YSlow as fast as possible. The YUI seed file and Loader were not included since all necessary modules and submodules were combined together following Ryan Grove’s Performance Zen tips, which made it possible to load everything together into a single small single request: yslow-bookmarklet.js: 204KB, 66KB (gzip) where:

YUI: 75KB, 27KB (gzip)
YSlow: 129KB, 39KB (gzip)

More about YSlow

Keep up-to-date with the latest YSlow announcements by:

Visiting the redesigned YSlow page at getyslow.com
Liking YSlow on Facebook: facebook.com/getyslow
Following YSlow on Twitter: twitter.com/getyslow

About the author: Marcel Duran is the Front End Lead for Yahoo!’s Exceptional Performance Team. He has been into web performance optimization on high traffic sites in Yahoo! Front Page and Search teams where he applied and researched web performance best practices making pages even faster. He is now dedicated to YSlow and other performance tools development, researches and evangelism. His goal is to make the web even faster than it can be and believes there is no such thing as “just a few milliseconds won’t hurt”.

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

YUI and Loader changes for 3.4.0

July 1, 2011 at 6:34 am by Dav Glass | In Development, Performance | 10 Comments

In 3.4.0 we started the process of shifting some of Loader’s logic around, to not only make it more performant, but to make it more robust and easier to use in other places (like on the server). We will be rolling out more changes in future revisions, but I wanted to take some time and explain what was changed, why it was changed and how it may impact developers. For the majority of use-cases, developers will notice nothing different, except that things are a little faster and their requirement downloads are a little smaller.

Seed File

The first thing I want to address is the YUI seed file. In previous versions of YUI, our seed file was very tiny and did not contain Loader or any of its meta-data. We’ve found that in the 90% use-case this was not as performant as we had hoped. The normal user includes the seed file then requests their modules, which in turn means that the seed needs to first fetch Loader, then calculate all of its dependencies, then fetch them all. We now feel that this extra http request is the wrong thing to do, so the new standard seed file contains Loader and its meta-data. Yes, this will make the initial request a little larger, but it will make the loading of modules that much faster since all of its meta-data requirements are now already on the page.

If you wish to use it the old way, you can just include the yui-base seed file instead. It contains everything that is needed to make YUI run in stand-alone mode plus it contains the ability to fetch Loader on demand. If you require even finer-grained dependencies, we have created a yui-core seed file that is exactly what the old yui-base seed was.

    /build/yui/yui-min.js //YUI Seed + Loader
    /build/yui-base/yui-base-min.js //Old YUI Seed with Loader fetch support
    /build/yui-core/yui-core-min.js //Old yui-base without Loader fetch support

It should be noted that these URLs are different than the previous URLs. Anyone that was using the yui/yui-base.js files need to repoint them to yui-core/yui-core.js. If you want the older way of loading the seed and fetching Loader, you would use the yui-base/yui-base.js seed file.

The other reasoning for this change is our roadmap for making YUI run in as many places as possible. The old seed file plus Loader in a single combo server request is all well and good if you have a combo server available in your application. But what about on the server? Or in an offline app on a mobile device? These places need to minimize file access while still getting the information they need.

Rollups

The next thing that we changed was removing rollups from the system and defaulting allowRollup to false in the Loader config. What does this mean to you? Well, hopefully nothing at all. Before I explain the impact of the change, let me explain the reasoning behind it. The primary reason is, again, performance, along with payload delivery. Take this example:

     Module A: requires event-a, event-b
     Module B: requires event-c, event-d

When you request both, the rollup logic prior to 3.4.0 used to determine that you should get the event rollup. Which actually meant that you were getting:

     event.a, event.b, event.c, event.d, event.e, event.f, event.g, event.h

You ended up with more on your page than you actually needed. By turning off the rollup support, YUI will now ask for only what you actually requested and nothing more. In most cases, you will not notice this. Module developers, may run into a situation where things that worked in the past may not work now. The reason for this is that they actually worked by accident before. Let me use a real world example: Dial.

In 3.3.0, Dial required this:

    requires: [
        'widget',
        'dd-drag',
        'substitute',
        'event-mouseenter', 
        'transition',
        'intl'
    ]

For the most part, Dial worked in 3.4.0, however keyboard support did not work. After doing some simple investigating, it turned out that the rollup support was actually requesting the entire Event rollup (which includes event-move and event-key). Without the rollup logic pulling in all of event, 3.4.0 Dial no longer had all of its requirements. Making Dial’s requirements more specific and defining all of its actual dependencies properly makes it work as expected.

    requires: [
        'widget',
        'dd-drag',
        'substitute',
        'event-mouseenter',
        'event-move',
        'event-key',
        'transition',
        'intl'
    ]

For module developers, it is a best practice to make sure that your module requires exactly what it needs to function. Do not assume that an upstream module requirement is there. It’s always better to make sure that you ask for what you need.

This also means that module requirements are more well defined. For example, datatype-date has Intl support built in. In previous versions you would access the Intl like this:

    Y.Intl.getAvailableLangs('datatype-date');

But since this module doesn’t actually have a language (the datatype-date-format module does), this will fail. It needs to be more specific and actually ask for languages for the correct module:

    Y.Intl.getAvailableLangs('datatype-date-format');

Build File Explosion and Submodule Removal

After making this change, the next change we made was exploding the build directory and removing submodules from the core system. Submodule logic was not removed, only our meta-data structure was changed. This will provide backward compatibility for current installations.

Submodules in the core system caused a couple of issues that we needed to resolve. The first reason was performance. Each time Loader needed to calculate dependencies, it needed to walk the submodule/plugin structure of each module. Doing this thousands of times was hurting our performance during the Loader calculate routine. By removing support for submodules in the core system we saved tens of thousands of function calls and iterations.

Loader was changed so that if a use property in a module’s meta-data defined more modules, it will use those modules instead of trying to load the original module. So, if you requested “dd” Loader would inspect “dd“‘s meta-data and see a use property that looks something like this:

    "dd-ddm-base,dd-ddm,dd-ddm-drop,dd-drag,dd-proxy,dd-constrain,dd-drop,dd-scroll,dd-drop-plugin"

In the core YUI seed file, we are also including what we are calling virtual rollups or aliases. These module definitions are exactly the same as the meta-data in Loader. This way you can include all the files exported from our Dependency Configurator and still use these rollups without having Loader present on the page. In future releases, we will be refining this approach even more.

After making this change, we then preceeded to explode our build files. In previous releases, the submodules determined the modules file path. For example:

    "dd": {
        "submodules": {
            "dd-drag": 
            // Module data
        }
    }

In 3.3.0 when you built “dd”, the file structure looked something like this:

    /build/dd/dd-drag.js
    /build/dd/dd-ddm.js
    /build/dd/dd-drop.js

With the build system exploded in 3.4.0, “dd”‘s build files now look like this:

    /build/dd-drag/dd-drag.js
    /build/dd-ddm/dd-ddm.js
    /build/dd-drop/dd-drop.js

This allowed us to remove the “path” property from all of our module meta-data as well, saving file size and reducing the logic required to assemble the modules url paths.

If you are including a pre-configured combo URL, you must recalculate your URL when you upgrade.

The downside to this change is that if you are including a combo URL of modules to “prep” your page you can not just change the version number and upgrade. You will need to revisit the Dependency Configurator and generate a new URL with new module structure.

The Future

I will be continuing to refine, refactor and maximize every aspect of our Loader and Seed strategy. These first steps were needed to aid in future changes that need to be made for not only our client-side strategy but also our server, command-line and mobile device strategies as well.

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

Announcing YUI Compressor 2.4.6

April 26, 2011 at 10:09 am by Stoyan Stefanov | In Development, Performance | 10 Comments We’re pleased to announce the immediate availability of version 2.4.6 of the YUI Compressor. This version contains mostly updates related to Compressor’s handling of CSS minification and introduces batch processing of multiple files with a single command.

CSS minification

Highlights include:

Fixed numerous bugs that break the compressor and/or the resulting minified files.
Added documentation on what exactly the minifier does and also which CSS hacks it tolerates.
There’s a JavaScript port of CSS min in case it’s more suitable for your build process. Here’s also a test web UI that uses the JavaScript port, where you can experiment with the minifier.
A significant number of new tests added (but you can add even more).
Safe handling of some CSS features that are getting more adoption such as media queries and CSS3 transforms.

Batch processing

Another welcome addition to Compressor is that it can now handle batches of files. This can significantly reduce the time your build process takes, especially if you have a great number of files to minify. For example the following commands minify all .js and .css files and write the minified files with a “-min.css” suffix.

$ java -jar yuicompressor.jar -o '.css$:-min.css' *.css
$ java -jar yuicompressor.jar -o '.js$:-min.js' *.js

Thanks go out to Stephen Woods and the Flickr team for this feature!

Links

YUI Compressor 2.4.6 is available for immediate download. Feel free to help us out by filing a bug or feature request, writing more tests, forking the code or joining the conversation.

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

Improving perceived site performance with spinners

November 11, 2010 at 9:33 am by Matt Parker | In Development, Performance | 4 Comments At the London Ajax meetup this week, Piotr (one of the creators of the rather good jsfiddle.net talked about spinners — the pretty common “I’m doing something” indicator — and how users perceive them. Apparently, people perceive Chrome to be faster in part because the little activity indicator keeps changing — it appears and disappears, and changes speed — while a page is loading. This sense of something happening persuades people that something is in fact happening, and faster, even if the actual speeds are identical. So Piotr set up a randomized survey, comparing perceptions of load speed after clicking two buttons — it’s here if you’re interested. When you click the button there’s a delay before the spinner is shown, and then a short (random) time later the results are shown. Then you click another button, and the same thing happens. And then you say which was faster. He also allowed for a “this thing seems broken option”. (If you’re going to do the survey, do it now and then come and read the conclusions — I don’t want to spoil it for you!). The results are here. The conclusion is that by delaying the display of the spinner slightly, users perceive things to be happening quicker. But wait too long and they start to think something’s broken — 0.4s seemed to be the optimal delay, from the survey results. And it may be worth thinking about other indicators if things take longer — add a “loading…” text overlay after 1 sec, perhaps.

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

Mobile Browser Cache Limits, Revisited

July 12, 2010 at 8:45 am by Ryan Grove | In Development, Performance | 9 Comments

In Mobile Browser Cache Limits: Android, iOS, and webOS, I shared the results of my attempts to determine browser cache limits on Android, iOS, and webOS devices. At the end of the article, I wrote:

Use these results as a starting point, but verify them yourself before you make major decisions based on assumptions about mobile cache limitations. The mobile browser world changes at a lightning pace, so this research will have a very short shelf life.

As it turns out, that was good advice: the day after the article was posted, Steve Souders commented that he had run tests using a different methodology that was more representative of a real-world web workflow and had gotten different results.

New Methodology

My original methodology involved navigating directly to a randomly generated page of a certain size, served with a text/html content type. The results using this methodology were reliably reproducible (except on webOS), but as Steve pointed out, users don’t navigate directly to CSS and JavaScript files. My assumption that the limits for direct navigation to an HTML resource were the same as the limits for external CSS and JavaScript was incorrect, so even though the results of my tests were valid, they weren’t widely applicable.

Over the course of many IM sessions, several emails, and a couple of phone calls, Steve and I worked out a new testing methodology. I implemented a version of it on top of my cache testing framework, then Steve implemented a version capable of publishing results to Browserscope.

In the new tests, we load an HTML page that refers to a randomly-generated CSS or JavaScript component of a certain size. Then we navigate to a second HTML page that loads the same component and checks whether or not it was loaded from the cache. To determine whether a component was loaded from the cache, we store a timestamp in a cookie on each request; if the timestamp is updated the second time we load the component, we know the request hit the server, which means the component was not loaded from the cache.

New Results

We found that all the mobile browsers we tested had significantly higher cache limits for external resources loaded by a page than they did for an HTML page itself. This is excellent news for mobile web developers.

The table below illustrates our findings:

Table: Mobile browser external resource cache characteristics
Browser/OS/Device	Single Component Limit	Survives Power Cycle
Android 2.2 (Nexus One)	2MB	Yes
Mobile Safari, iOS 3.1.3 (1st-gen iPhone)	4MB+	No
Mobile Safari, iOS 3.2 (iPad)	4MB+	No
Mobile Safari, iOS 4.0 (iPhone 3GS)	4MB+	No
Mobile Safari, iOS 4.0 (iPhone 4)	4MB+	No
webOS 1.4.1 (Palm Pre Plus)	~0.99MB (1,023KB)	Yes

Note that 4MB was the largest size we tested, and all the iOS devices cached 4MB components. The actual cache limit for those devices may be larger than 4MB. Also, webOS on the Palm Pre Plus gave consistent results in this test, whereas it had some problems in the previous test.

It’s possible that the much lower limits my previous test showed for HTML components on iOS may indicate the use of a RAM cache for those components, while the much higher limits for CSS/JS components in this test may indicate the use of a disk cache, but this is just conjecture. Android, at least, does appear to use a disk cache in both cases, since its cache survives power cycles.

New Recommendations

Based on these new results, coupled with the results from my previous tests, I offer the following updated set of recommendations:

Use far-future cache expiration headers. This will prevent the browser from having to send a conditional GET request.
Try to limit HTML pages to 25.6KB or less if you want them to be cached, since the previous tests showed that this limit—imposed by iOS 3.2 on the iPad—was the lowest HTML resource limit of the devices tested.
Keep CSS and JS components under 1MB. Of course, 1MB is enormous and your components should be much smaller than this, but don’t bother splitting a component into separate requests for the sake of cacheability unless its size approaches 1MB.
Consider using the HTML5 application cache if it’s important that your components persist in the cache for a long time, or across power cycles.
Do your own testing. I stressed the importance of this in my previous article and I’ll stress it again here. Use these results as a starting point, but verify them yourself before you make important decisions based on them.

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

Mobile Browser Cache Limits: Android, iOS, and webOS

June 28, 2010 at 8:45 am by Ryan Grove | In Development, Performance | 19 Comments

Update (July 12, 2010): While the results described in this article are accurate for HTML pages, new tests have revealed very different cache limits for CSS and JS resources. The updated results are described in Mobile Browser Cache Limits, Revisited.

In early 2008, Wayne Shea and Tenni Theurer wrote a YUI Blog post on iPhone Cacheability in which they shared the results of research into various characteristics and limitations of Mobile Safari’s cache in iPhone OS 1.x. Among other things, they found that individual components larger than 25KB were not cached, and that there was a maximum total cache size of between 475KB and 500KB.

Much has changed since then. We’ve seen two new major releases and many minor releases of the iPhone OS (now iOS), and several other mobile devices with highly capable browsers have appeared to challenge the iPhone. Stoyan Stefanov found, in late 2009, that the iPhone’s cache limits had changed (sadly, for the worse). But where do things stand now? And what about those non-iOS browsers?

Background

Browsers have two types of caches that we’re concerned with for the purposes of these tests:

The component cache, or object cache, stores individual files. HTML, CSS, JavaScript, and images all go into the component cache. Whenever it needs one of these components, the browser first checks the cache before making a network request.
The page cache, also known as the back/forward cache, stores an entire page and all of its components, as well as their current state. When you use the back or forward button, the browser will load the page from the page cache if possible.

The HTML5 application cache is another type of cache that’s widely supported by mobile browsers. Browser makers already do a good job of documenting the limits of the application cache, so I didn’t include it in my testing. More on the application cache later.

Devices Tested

I tested the following mobile browser/platform combinations:

Android 2.1 (Nexus One)
Mobile Safari on iOS 3.1.3 (1st-gen iPhone)
Mobile Safari on iOS 3.2 (iPad)
Mobile Safari on iOS 4.0 (iPhone 3GS)
Mobile Safari on iOS 4.0 (iPhone 4)
webOS 1.4.1 (Palm Pre Plus)

Note: With the exception of Mobile Safari on iOS 4.0, I tested only one device in each category. If there are variations between individual devices or differences based on installed software beyond the OS, my tests would not detect those variations. These particular devices were tested because they’re the ones I had access to, not because I consider them to be more important than other devices.

Methodology

Cache testing is tedious, but relatively simple.

I wrote a tiny Sinatra app (fork it on GitHub!) that generates a response consisting of a requested number of pseudorandom alphanumeric and whitespace bytes. The responses can be served either gzipped or uncompressed. The following far-future expiration response headers are sent to ensure that all responses are considered cacheable:

Cache-Control: max-age=315360000
Expires: Fri, 01 May 2020 03:47:24 GMT

Over my local network, I then manually performed the following steps on each device to test the component cache:

Load the cache test index page.
Tap on a link to a component of a particular size, ranging from 5KB to 20MB, and wait for it to finish loading.
Tap the back button.
Tap the same link again. Observe whether the random characters are the same, and whether the server console prints a log entry for a request, to determine whether the component was cached in step 2.
Repeat and adjust component sizes as necessary to determine the maximum component size that will be cached.

To test the page cache, I performed essentially the same steps except that instead of tapping the link again in step 4, I tapped the browser’s forward button, causing it to use the page cache rather than the component cache.

Support for ETag and Last-Modified was determined by tweaking the server to send the appropriate ETag or Last-Modified response headers (in separate tests) and to omit the far-future expiration headers. I then inspected the request headers received by the server to verify that the browser sent the expected If-None-Match or If-Modified-Since headers on step 4.

Results

I tested with gzip both enabled and disabled, but I found that gzip had no effect on cacheability on any device. The uncompressed component size is what matters in all cases, regardless of whether or not that component is served gzipped. As such, all component sizes mentioned here are uncompressed sizes.

The table below illustrates my overall findings.

Table: Mobile browser cache characteristics
Browser/OS/Device	Single Component Limit	Total Component Limit	Page Cache Size Limit	Supports Last-Modified	Supports ETag	Survives Power Cycle
Android 2.1 (Nexus One)	~2MB (~2,048,000b)	~2MB (~2,048,000b)	∞ ²	Yes	Yes	Yes
Mobile Safari, iOS 3.1.3 (1st-gen iPhone)	0b ¹	0b ¹	∞ ²	No	No	No
Mobile Safari, iOS 3.2 (iPad)	25.6KB (26,214b)	~281.6KB (~288,354b)	25.6KB (26,214b)	Yes	Yes	No
Mobile Safari, iOS 4.0 (iPhone 3GS)	51.199KB (52,428b)	~1.05MB (~1,100,988b)	∞ ²	Yes	Yes	No
Mobile Safari, iOS 4.0 (iPhone 4)	102.399KB (104,857b)	~1.9MB (~1,992,283b)	∞ ²	Yes	Yes	No
webOS 1.4.1 (Palm Pre Plus) ³	~1MB (~1,048,576)	?	~1MB (~1,048,576)	No	No	Yes

Notes:

¹ Mobile Safari on iOS 3.1.3 doesn’t appear to cache any components, regardless of size, except for the page cache. It’s unclear whether this is intentional or a bug.

² The page caches in Android 2.1, iOS 3.1.3, and iOS 4.0 (but not iOS 3.2) appear to be limited only by available RAM when it comes to individual page size. I didn’t attempt to determine exactly how many separate pages could coexist in the page cache at once.

³ webOS test results were inconsistent and at various points the cache seemed to stop working altogether until the phone was power-cycled. I don’t consider these results conclusive, or even trustworthy, but they’re listed here for the sake of comparison.

Android

The Android browser exhibited the best cache behavior of all devices tested. While it appears to impose no limit on the size of individual components, the total cache size seems to be limited to approximately 2MB, which means that individual components are effectively limited to 2MB as well.

The page cache appeared to impose no limit on the size of individual pages, happily caching every byte I threw at it until the available RAM was exhausted and the browser crashed.

I was pleasantly surprised to find that Android’s component cache survived both browser restarts and power cycles, a feat none of the iOS devices was able to match.

Possible caveat: A review of Android’s WebKit source tree leads me to believe that its cache limits may adapt based on the amount of RAM and/or flash memory available on the particular device on which it’s running. If true, these numbers may only be applicable to the Nexus One. In fact, if the cache size adapts based on unused memory rather than total memory, these numbers may only be applicable to my Nexus One.

I could be mistaken, but the differences in the iOS 4.0 test results on the iPhone 3GS and iPhone 4 support this theory. (Android and Mobile Safari are both WebKit-based browsers, so they may have this behavior in common.) If you’re familiar with the WebKit source and can shed more light on this, please get in touch with me.

iOS

Results varied wildly across the three most recent versions of iOS. Astonishingly, Mobile Safari on iOS 3.1.3 did not cache components of any size, despite having an apparently unlimited page cache size. This is troubling since it means iOS 3.1.3 users are likely getting a suboptimal browsing experience, especially if they aren’t using wifi. The unlimited page cache size does little good here, since it only comes into play for back/forward navigations. This behavior is a significant change from what others observed in previous iOS releases and I can’t imagine any good reason for it, so I suspect this may be a bug.

Mobile Safari on iOS 3.2 (which is only available on the iPad) does not exhibit this bug. Its 25.6KB component limit and ~281.6KB total cache limit are better than nothing, but they still seem paltry compared to the other devices tested. Uniquely among iOS devices, the iPad appears to limit the size of pages in the page cache to 25.6KB, the same as its component size limit.

Mobile Safari on iOS 4.0 exhibited different limits on the iPhone 3GS and on the iPhone 4, which implies that the limits adapt based on available RAM (the iPhone 3GS has 256MB while the iPhone 4 has 512MB; both devices tested had 32GB of flash memory). On the iPhone 3GS, iOS 4.0 has a 51.199KB component size limit and a ~1.05MB total component cache size.

On the iPhone 4, the component size limit was almost exactly two times the limit on the iPhone 3GS, at 102.399KB. The total component cache size was approximately 1.9MB. Perhaps because iOS 3.2 and iOS 4.0 were developed separately but branched from a common ancestor, the iOS 4.0 page cache size appears to be limited only by available RAM on both devices tested, just like iOS 3.1.3.

None of the iOS devices preserved the contents of the cache across forced browser restarts or device power cycles, although they did preserve the cache when merely switching applications without actually killing the browser.

webOS

My test results on webOS were so inconsistent that I have little confidence in them. I’ve included what little data I managed to gather purely for the sake of completeness. Please take it with a hefty grain of salt.

As near as I was able to determine, webOS might have an individual component size limit of about 1MB, with a matching page size limit in the page cache. I was unable to coax If-None-Match or If-Modified-Since request headers from webOS, which implies that it does not support ETag and Last-Modified.

On some tests, it appeared that webOS’s maximum component size was greater than 1MB, but this was inconsistent. As far as I can tell, webOS appears to have a nasty bug where, after a certain point—possibly when the maximum total cache size is reached—the cache just completely stops working altogether until the phone is power-cycled. In some cases even power cycling didn’t fix the cache breakage, so I eventually gave up trying to establish the exact cause of the problem and the exact limits of the webOS cache.

Recommendations

Based on these results, I offer the following recommendations to anyone developing web applications for the tested devices:

Use far-future cache expiration headers. This will prevent the browser from having to send a conditional GET request and will improve cacheability in webOS, which doesn’t support ETag or Last-Modified.
At least until iOS 4.0 arrives on the iPad, try to limit individual component sizes to 25.6KB or less, uncompressed. And urge your iPhone users to upgrade to iOS 4.0 as soon as possible.
If your website must support iOS 3.1.3 users (which is likely), if it requires components larger than 25.6KB, or if the total size of all your components is larger than 281.6KB, consider using the HTML5 application cache, localStorage, or database storage to store your components. Alex Kessinger’s recent YUI Blog post, An Introduction to Using YUI 3 in Offline Applications, might be of interest for YUI 3 developers considering this approach.
Do your own testing. Don’t assume that these results apply to any future version of any of the tested browsers or devices. Use these results as a starting point, but verify them yourself before you make major decisions based on assumptions about mobile cache limitations. The mobile browser world changes at a lightning pace, so this research will have a very short shelf life.

I’ve made my test code available on GitHub and I encourage you to use it, fork it, and share what you learn.

Call for Documentation

Browser makers, please consider documenting and publishing your browser’s cache limits. In the desktop world where these limits are typically so high as to be a non-issue, documentation wasn’t needed. In the mobile world, browser cache limits are vital information that web developers must have in order to create performant websites with compelling features.

The limits of new features like localStorage and the application cache are typically well-documented. Please extend this level of documentation to the component cache as well.

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

YUI Theater — Douglas Crockford: "Ajax Performance"

December 23, 2008 at 8:07 am by Eric Miraglia | In Development, Performance, YUI Theater | 6 Comments

Douglas Crockford returns to YUI Theater with another chapter in his evolving lecture series. This session, “Ajax Performance,” debunks common misconceptions about the relationship between JavaScript and performance and gives engineers a core focus for improving the performance of web apps: Reduce the value of n. Because DOM interactions are generally slow, leveraging Ajax to reduce the number of DOM operations, Douglas argues, is often the most important optmization you can make. In fact, it usually dwarfs other techniques in terms of its impact on the actual experience of using a website.

This talk joins an extensive library of Douglas’s lectures now available on YUI Theater, including his popular series on JavaScript.

Douglas Crockford: "Ajax Performance" @ Yahoo! Video

download (m4v)