Yahoo! User Interface Blog (YUIBlog)

Mobile Browser Cache Limits, Revisited

July 12, 2010 at 8:45 am by Ryan Grove | In Development, Performance | 4 Comments

In Mobile Browser Cache Limits: Android, iOS, and webOS, I shared the results of my attempts to determine browser cache limits on Android, iOS, and webOS devices. At the end of the article, I wrote:

Use these results as a starting point, but verify them yourself before you make major decisions based on assumptions about mobile cache limitations. The mobile browser world changes at a lightning pace, so this research will have a very short shelf life.

As it turns out, that was good advice: the day after the article was posted, Steve Souders commented that he had run tests using a different methodology that was more representative of a real-world web workflow and had gotten different results.

New Methodology

My original methodology involved navigating directly to a randomly generated page of a certain size, served with a text/html content type. The results using this methodology were reliably reproducible (except on webOS), but as Steve pointed out, users don’t navigate directly to CSS and JavaScript files. My assumption that the limits for direct navigation to an HTML resource were the same as the limits for external CSS and JavaScript was incorrect, so even though the results of my tests were valid, they weren’t widely applicable.

Over the course of many IM sessions, several emails, and a couple of phone calls, Steve and I worked out a new testing methodology. I implemented a version of it on top of my cache testing framework, then Steve implemented a version capable of publishing results to Browserscope.

In the new tests, we load an HTML page that refers to a randomly-generated CSS or JavaScript component of a certain size. Then we navigate to a second HTML page that loads the same component and checks whether or not it was loaded from the cache. To determine whether a component was loaded from the cache, we store a timestamp in a cookie on each request; if the timestamp is updated the second time we load the component, we know the request hit the server, which means the component was not loaded from the cache.

New Results

We found that all the mobile browsers we tested had significantly higher cache limits for external resources loaded by a page than they did for an HTML page itself. This is excellent news for mobile web developers.

The table below illustrates our findings:

Table: Mobile browser external resource cache characteristics

Browser/OS/Device	Single Component Limit	Survives Power Cycle
Android 2.2 (Nexus One)	2MB	Yes
Mobile Safari, iOS 3.1.3 (1st-gen iPhone)	4MB+	No
Mobile Safari, iOS 3.2 (iPad)	4MB+	No
Mobile Safari, iOS 4.0 (iPhone 3GS)	4MB+	No
Mobile Safari, iOS 4.0 (iPhone 4)	4MB+	No
webOS 1.4.1 (Palm Pre Plus)	~0.99MB (1,023KB)	Yes

Note that 4MB was the largest size we tested, and all the iOS devices cached 4MB components. The actual cache limit for those devices may be larger than 4MB. Also, webOS on the Palm Pre Plus gave consistent results in this test, whereas it had some problems in the previous test.

It’s possible that the much lower limits my previous test showed for HTML components on iOS may indicate the use of a RAM cache for those components, while the much higher limits for CSS/JS components in this test may indicate the use of a disk cache, but this is just conjecture. Android, at least, does appear to use a disk cache in both cases, since its cache survives power cycles.

New Recommendations

Based on these new results, coupled with the results from my previous tests, I offer the following updated set of recommendations:

• Use far-future cache expiration headers. This will prevent the browser from having to send a conditional GET request.
• Try to limit HTML pages to 25.6KB or less if you want them to be cached, since the previous tests showed that this limit—imposed by iOS 3.2 on the iPad—was the lowest HTML resource limit of the devices tested.
• Keep CSS and JS components under 1MB. Of course, 1MB is enormous and your components should be much smaller than this, but don’t bother splitting a component into separate requests for the sake of cacheability unless its size approaches 1MB.
• Consider using the HTML5 application cache if it’s important that your components persist in the cache for a long time, or across power cycles.
• Do your own testing. I stressed the importance of this in my previous article and I’ll stress it again here. Use these results as a starting point, but verify them yourself before you make important decisions based on them.

Find conversations: bloglines | technorati

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

Mobile Browser Cache Limits: Android, iOS, and webOS

June 28, 2010 at 8:45 am by Ryan Grove | In Development, Performance | 19 Comments

Update (July 12, 2010): While the results described in this article are accurate for HTML pages, new tests have revealed very different cache limits for CSS and JS resources. The updated results are described in Mobile Browser Cache Limits, Revisited.

In early 2008, Wayne Shea and Tenni Theurer wrote a YUI Blog post on iPhone Cacheability in which they shared the results of research into various characteristics and limitations of Mobile Safari’s cache in iPhone OS 1.x. Among other things, they found that individual components larger than 25KB were not cached, and that there was a maximum total cache size of between 475KB and 500KB.

Much has changed since then. We’ve seen two new major releases and many minor releases of the iPhone OS (now iOS), and several other mobile devices with highly capable browsers have appeared to challenge the iPhone. Stoyan Stefanov found, in late 2009, that the iPhone’s cache limits had changed (sadly, for the worse). But where do things stand now? And what about those non-iOS browsers?

Background

Browsers have two types of caches that we’re concerned with for the purposes of these tests:

• The component cache, or object cache, stores individual files. HTML, CSS, JavaScript, and images all go into the component cache. Whenever it needs one of these components, the browser first checks the cache before making a network request.
• The page cache, also known as the back/forward cache, stores an entire page and all of its components, as well as their current state. When you use the back or forward button, the browser will load the page from the page cache if possible.

The HTML5 application cache is another type of cache that’s widely supported by mobile browsers. Browser makers already do a good job of documenting the limits of the application cache, so I didn’t include it in my testing. More on the application cache later.

Devices Tested

I tested the following mobile browser/platform combinations:

• Android 2.1 (Nexus One)
• Mobile Safari on iOS 3.1.3 (1st-gen iPhone)
• Mobile Safari on iOS 3.2 (iPad)
• Mobile Safari on iOS 4.0 (iPhone 3GS)
• Mobile Safari on iOS 4.0 (iPhone 4)
• webOS 1.4.1 (Palm Pre Plus)

Note: With the exception of Mobile Safari on iOS 4.0, I tested only one device in each category. If there are variations between individual devices or differences based on installed software beyond the OS, my tests would not detect those variations. These particular devices were tested because they’re the ones I had access to, not because I consider them to be more important than other devices.

Methodology

Cache testing is tedious, but relatively simple.

I wrote a tiny Sinatra app (fork it on GitHub!) that generates a response consisting of a requested number of pseudorandom alphanumeric and whitespace bytes. The responses can be served either gzipped or uncompressed. The following far-future expiration response headers are sent to ensure that all responses are considered cacheable:

Cache-Control: max-age=315360000
Expires: Fri, 01 May 2020 03:47:24 GMT

Over my local network, I then manually performed the following steps on each device to test the component cache:

1. Load the cache test index page.
2. Tap on a link to a component of a particular size, ranging from 5KB to 20MB, and wait for it to finish loading.
3. Tap the back button.
4. Tap the same link again. Observe whether the random characters are the same, and whether the server console prints a log entry for a request, to determine whether the component was cached in step 2.
5. Repeat and adjust component sizes as necessary to determine the maximum component size that will be cached.

To test the page cache, I performed essentially the same steps except that instead of tapping the link again in step 4, I tapped the browser’s forward button, causing it to use the page cache rather than the component cache.

Support for ETag and Last-Modified was determined by tweaking the server to send the appropriate ETag or Last-Modified response headers (in separate tests) and to omit the far-future expiration headers. I then inspected the request headers received by the server to verify that the browser sent the expected If-None-Match or If-Modified-Since headers on step 4.

Results

I tested with gzip both enabled and disabled, but I found that gzip had no effect on cacheability on any device. The uncompressed component size is what matters in all cases, regardless of whether or not that component is served gzipped. As such, all component sizes mentioned here are uncompressed sizes.

The table below illustrates my overall findings.

Table: Mobile browser cache characteristics

Browser/OS/Device	Single Component Limit	Total Component Limit	Page Cache Size Limit	Supports Last-Modified	Supports ETag	Survives Power Cycle
Android 2.1 (Nexus One)	~2MB (~2,048,000b)	~2MB (~2,048,000b)	∞ 2	Yes	Yes	Yes
Mobile Safari, iOS 3.1.3 (1st-gen iPhone)	0b 1	0b 1	∞ 2	No	No	No
Mobile Safari, iOS 3.2 (iPad)	25.6KB (26,214b)	~281.6KB (~288,354b)	25.6KB (26,214b)	Yes	Yes	No
Mobile Safari, iOS 4.0 (iPhone 3GS)	51.199KB (52,428b)	~1.05MB (~1,100,988b)	∞ 2	Yes	Yes	No
Mobile Safari, iOS 4.0 (iPhone 4)	102.399KB (104,857b)	~1.9MB (~1,992,283b)	∞ 2	Yes	Yes	No
webOS 1.4.1 (Palm Pre Plus) 3	~1MB (~1,048,576)	?	~1MB (~1,048,576)	No	No	Yes

Android

The Android browser exhibited the best cache behavior of all devices tested. While it appears to impose no limit on the size of individual components, the total cache size seems to be limited to approximately 2MB, which means that individual components are effectively limited to 2MB as well.

The page cache appeared to impose no limit on the size of individual pages, happily caching every byte I threw at it until the available RAM was exhausted and the browser crashed.

I was pleasantly surprised to find that Android’s component cache survived both browser restarts and power cycles, a feat none of the iOS devices was able to match.

Possible caveat: A review of Android’s WebKit source tree leads me to believe that its cache limits may adapt based on the amount of RAM and/or flash memory available on the particular device on which it’s running. If true, these numbers may only be applicable to the Nexus One. In fact, if the cache size adapts based on unused memory rather than total memory, these numbers may only be applicable to my Nexus One.

I could be mistaken, but the differences in the iOS 4.0 test results on the iPhone 3GS and iPhone 4 support this theory. (Android and Mobile Safari are both WebKit-based browsers, so they may have this behavior in common.) If you’re familiar with the WebKit source and can shed more light on this, please get in touch with me.

iOS

Results varied wildly across the three most recent versions of iOS. Astonishingly, Mobile Safari on iOS 3.1.3 did not cache components of any size, despite having an apparently unlimited page cache size. This is troubling since it means iOS 3.1.3 users are likely getting a suboptimal browsing experience, especially if they aren’t using wifi. The unlimited page cache size does little good here, since it only comes into play for back/forward navigations. This behavior is a significant change from what others observed in previous iOS releases and I can’t imagine any good reason for it, so I suspect this may be a bug.

Mobile Safari on iOS 3.2 (which is only available on the iPad) does not exhibit this bug. Its 25.6KB component limit and ~281.6KB total cache limit are better than nothing, but they still seem paltry compared to the other devices tested. Uniquely among iOS devices, the iPad appears to limit the size of pages in the page cache to 25.6KB, the same as its component size limit.

Mobile Safari on iOS 4.0 exhibited different limits on the iPhone 3GS and on the iPhone 4, which implies that the limits adapt based on available RAM (the iPhone 3GS has 256MB while the iPhone 4 has 512MB; both devices tested had 32GB of flash memory). On the iPhone 3GS, iOS 4.0 has a 51.199KB component size limit and a ~1.05MB total component cache size.

On the iPhone 4, the component size limit was almost exactly two times the limit on the iPhone 3GS, at 102.399KB. The total component cache size was approximately 1.9MB. Perhaps because iOS 3.2 and iOS 4.0 were developed separately but branched from a common ancestor, the iOS 4.0 page cache size appears to be limited only by available RAM on both devices tested, just like iOS 3.1.3.

None of the iOS devices preserved the contents of the cache across forced browser restarts or device power cycles, although they did preserve the cache when merely switching applications without actually killing the browser.

webOS

My test results on webOS were so inconsistent that I have little confidence in them. I’ve included what little data I managed to gather purely for the sake of completeness. Please take it with a hefty grain of salt.

As near as I was able to determine, webOS might have an individual component size limit of about 1MB, with a matching page size limit in the page cache. I was unable to coax If-None-Match or If-Modified-Since request headers from webOS, which implies that it does not support ETag and Last-Modified.

On some tests, it appeared that webOS’s maximum component size was greater than 1MB, but this was inconsistent. As far as I can tell, webOS appears to have a nasty bug where, after a certain point—possibly when the maximum total cache size is reached—the cache just completely stops working altogether until the phone is power-cycled. In some cases even power cycling didn’t fix the cache breakage, so I eventually gave up trying to establish the exact cause of the problem and the exact limits of the webOS cache.

Recommendations

Based on these results, I offer the following recommendations to anyone developing web applications for the tested devices:

• Use far-future cache expiration headers. This will prevent the browser from having to send a conditional GET request and will improve cacheability in webOS, which doesn’t support ETag or Last-Modified.
• At least until iOS 4.0 arrives on the iPad, try to limit individual component sizes to 25.6KB or less, uncompressed. And urge your iPhone users to upgrade to iOS 4.0 as soon as possible.
• If your website must support iOS 3.1.3 users (which is likely), if it requires components larger than 25.6KB, or if the total size of all your components is larger than 281.6KB, consider using the HTML5 application cache, localStorage, or database storage to store your components. Alex Kessinger’s recent YUI Blog post, An Introduction to Using YUI 3 in Offline Applications, might be of interest for YUI 3 developers considering this approach.
• Do your own testing. Don’t assume that these results apply to any future version of any of the tested browsers or devices. Use these results as a starting point, but verify them yourself before you make major decisions based on assumptions about mobile cache limitations. The mobile browser world changes at a lightning pace, so this research will have a very short shelf life.

I’ve made my test code available on GitHub and I encourage you to use it, fork it, and share what you learn.

Call for Documentation

Browser makers, please consider documenting and publishing your browser’s cache limits. In the desktop world where these limits are typically so high as to be a non-issue, documentation wasn’t needed. In the mobile world, browser cache limits are vital information that web developers must have in order to create performant websites with compelling features.

The limits of new features like localStorage and the application cache are typically well-documented. Please extend this level of documentation to the component cache as well.

Find conversations: bloglines | technorati

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

YUI Theater — Douglas Crockford: "Ajax Performance"

December 23, 2008 at 8:07 am by Eric Miraglia | In Development, Performance, YUI Theater | 6 Comments

Douglas Crockford returns to YUI Theater with another chapter in his evolving lecture series. This session, “Ajax Performance,” debunks common misconceptions about the relationship between JavaScript and performance and gives engineers a core focus for improving the performance of web apps: Reduce the value of n. Because DOM interactions are generally slow, leveraging Ajax to reduce the number of DOM operations, Douglas argues, is often the most important optmization you can make. In fact, it usually dwarfs other techniques in terms of its impact on the actual experience of using a website.

This talk joins an extensive library of Douglas’s lectures now available on YUI Theater, including his popular series on JavaScript.

• Slides (zipped PPT)
• Full text transcript of this talk

Douglas Crockford: "Ajax Performance" @ Yahoo! Video

download (m4v)

In Case You Missed…

Some other recent videos from the YUI Theater series:

• Nicole Sullivan: "Design Fast Websites (Don’t Blame the Rounded Corners)" (Yahoo! Video | .m4v download)
• Todd Kloots: “Developing Accessible Widgets Using ARIA” (Yahoo! Video | .m4v download)
• Nicholas C. Zakas: "Test-Driven Development with YUI Test" (Yahoo! Video | .m4v download)
• Douglas Crockford: "Web Forward" (Yahoo! Video | .m4v download)

Subscribing to YUI Theater:

• YUI Theater RSS feed
• YUI Theater on iTunes

Find conversations: bloglines | technorati

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

YUI Theater — Nicole Sullivan: "Design Fast Websites (Don’t Blame the Rounded Corners)"

December 23, 2008 at 8:04 am by Eric Miraglia | In Development, Performance, YUI Theater | 11 Comments

Nicole Sullivan is a website performance specialist and a former member of Yahoo’s Exceptional Performance Team. She is currently writing a book for O’Reilly with Stoyan Stefanov on performance optimization and she and Stoyan are the creators of Smushit, an engine for crushing images.

Nicole visited Yahoo last week to do an encore of her "Design Fast Websites" talk in which she outlines a set of practical guidelines for building websites that are supremely fast and visually rich. Her advice is to know your craft, to engage your designers, and to make sure that your collaboration with designers works intelligently in the service of users. She was kind enough to let us record the talk and share it with you here on YUI Theater.

• Slides (on Slideshare)

Nicole Sullivan: "Design Fast Websites" @ Yahoo! Video

download (m4v)

In Case You Missed…

Some other recent videos from the YUI Theater series:

• Todd Kloots: “Developing Accessible Widgets Using ARIA” (Yahoo! Video | .m4v download)
• Nicholas C. Zakas: "Test-Driven Development with YUI Test" (Yahoo! Video | .m4v download)
• Douglas Crockford: "Web Forward" (Yahoo! Video | .m4v download)
• Eric Miraglia and Matt Sweeney: "YUI3 — A Look Ahead" (Yahoo! Video | .m4v download)

Subscribing to YUI Theater:

• YUI Theater RSS feed
• YUI Theater on iTunes

Find conversations: bloglines | technorati

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

Image Optimization, Part 5: AlphaImageLoader

December 8, 2008 at 5:48 am by Stoyan Stefanov | In Development, Performance | 13 Comments

This is part 5 in an ongoing series. You can read the other parts here:

• Image Optimization Part 1: The Importance of Images
• Image Optimization Part 2: Selecting the Right File Format
• Image Optimization Part 3: Four Steps to File Size Reduction
• Image Optimization Part 4: Progressive JPEG…Hot or Not?

This installment of the image optimization series is about the IE-proprietary AlphaImageLoader CSS filter, which developers often use as a workaround to solve transparency issues with truecolor PNGs in IE. The problem with AlphaImageLoader is that it hurts page performance and, therefore, hurts user experience. I argue that AlphaImageLoader should be avoided when at all possible.

Quick Refresher

As mentioned in a previous article, PNGs come in several different types but can roughly be divided into:

• Indexed (palette), also referred to as PNG8 which have up to 256 colors.
• Truecolor PNG, also referred to as PNG32 or PNG24.

Both formats support alpha (variable) transparency and, while PNG8 images degrade to a GIF-like non-variable transparency in IE6 (example, source), truecolor PNGs show an uglyish background in place of the transparent pixels (source W3C).

The AlphaImageLoader fix

IE6 (and older versions of IE) provides a solution to the problem through its proprietary filter CSS property. The following code will display the proper image cross-browser:

#some-element {
    background: url(image.png);
    _background: none;
    _filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(src='image.png', sizingMethod='crop');
}

As you can see, the underscore CSS hack is used to target IE < 7 and

1. “undo” the background; and
2. load the same image, using Microsoft’s AlphaImageLoader filter.

The reason to target IE prior to version 7 is that IE7 supports the alpha transparency natively without the need for filters. (IE8 does too and it actually changes the filter syntax completely.)

It’s interesting to note that the filter doesn’t change the image; rather, it changes the HTML element this style is applied to. The other interesting thing is that each element is processed synchronously in a single UI thread. The process applying the filter takes some resources for each element and the more “filtered” elements you have, the worse it gets, even if you use the same image for all the elements.

The question is: How does this affect the overall performance of the page?

Freeze! Side Effect #1

Here’s a simple experiment: Create a page that has a CSS filter and then simulate (and exaggerate) network latency by delaying the image used in the filter by ten seconds. The result? Not only is nothing rendered (blank page) for ten seconds, but the browser freezes, meaning you cannot interact with it, click its icons or menus, type in the URL…you can’t even close it.

Here’s a test example.

In the example, I didn’t use the underscore hack so you can see the (d)effect in IE7 too, even in IE8 in “compatibility mode”.

While the effect is exaggerated for demo purposes, network latencies are a fact of life and this is probably the worst user experience you can deliver: Someone comes to your page and their browser freezes.

Note that parallel downloads are not blocked. The browser still downloads the other page components in the background, but there’s no progressive rendering. You can think of it this way — since IE will not render anything until the very last bit of CSS comes down the wire (more info), and your because CSS has a dependency on a filtered image, the rendering is blocked until the dependency is satisfied.

What if you have several AlphaImageLoader filters on the page? They are processed synchronously one after the other so the problem is multiplied. If you have 5 images, each delayed 2 seconds on the server, then the browser freezes for a total of 10 seconds.

Time and Memory — Side Effects #2 and #3

Another negative effect of using the AlphaImageLoader is the increase of the amount of memory required to process and apply the filters. These days we might be tempted to think our visitors’ computers have a virtually indefinite supply of memory, but for older computers (those more likely to run IE6 and under) this may not be the case.

And at the end of it, it’s the performance we’re most interested in, performance as measured by the time it takes for the page to load in the browser. Let’s do a test to measure how much time and memory is required by the filters.

First, let’s have a baseline page — one that has a hundred <div>s with the same non-filtered background image. Then let’s have a second page with a filter applied to the divs (all 100 divs use the same). A hundred elements with filtered backgrounds is unlikely to be found in a normal page, but a little exaggeration will help with the measurements.

The time is measured from the start of the page to the onload event of the page, after the images have been cached, thus eliminating the time required to download the page and the images. The memory consumption is measured with the help of the ProcessExplorer tool and given as the before/after delta of the private bytes measurement, showing the “price” of rendering the page.

Here are the median results from 10 runs in IE6 on a PC with a dual 2GHz CPU and 500M RAM. On a less powerful computer, the load times are likely to be even worse.

AlphaImageLoader test results

test page	time, seconds	memory, MB
Test #1 – no filters	0.031	0.6
Test #2 – with filters	0.844	46.8

As you can see, the AlphaImageLoader effect is pretty bad — our test page loads 27 times slower and eats up 78 times more memory. These results are, of course, highly speculative — it’s just one image tested on just one PC (relatively powerful and underworked). With different images, applied to a different number of elements and on different machines, results may vary considerably, especially when there’s less RAM or CPU, or if you throw network latency (side effect #1) into the mix. But this example illustrates the important concepts:

• AlphaImageLoader is slow and requires more memory
• It’s applied per element, not per image

If you have a sprite image and you use it for different elements (sprites with alpha filters are trickier, but doable), you’ll pay the penalty for each element the sprite is used on.

Yahoo! Search Case Study

Using lab tests like the one above can give us some idea of the AlphaImageLoader “price,” and you might be tempted to test and calculate approximately how much you pay for each filtered element, but there’s nothing better than a real life test with millions of requests coming form different parts of the world with different browsers, computers and bandwidth.

Yahoo!’s search results page used to have a truecolor PNG sprite and employed AlphaImageLoader to achieve the transparency (an older version of the sprite is still around if you’re curious). Replacing the truecolor PNG with a gracefully degrading PNG8 (discussed previously) decreased the pageload time by 50-100ms for the users of IE 5 and 6. 100ms may not seem like much, but for a page that loads under a second, it’s at least 10%. Also, according to an Amazon study, 100ms slower means 1% fewer sales (even for their content-heavy pages). Earning 1% more by just replacing an image doesn’t look like a bad deal at all.

So Now What?

The best thing would be to avoid AlphaImageLoader completely and, like Y!Search, take the time to create PNG8 images that degrade nicely in IE6 and look good in all other browsers. How do you create a gracefully degrading PNG8? Well, create a GIF-like image first, one that has only fully transparent or fully opaque pixels. After making sure it looks acceptable (it will look like this in IE6), proceed to enhancing the image with semi-transparent pixels which will smooth any rounded corners or other parts that would benefit from transparency. Unfortunately, as far as I know, Fireworks is currently the only image processing software capable of handling alpha transparency in PNG8. You can also try command line tools such as pngnq and pngquant, although automated truecolor-to-palette PNG conversion might not always yield satisfactory results and you might need to pick the fully opaque pixels manually.

There might be cases when you won’t be able to get by with a PNG8 and absolutely need to use AlphaImageLoader — for example when most or all pixels are semi-transparent (imagine a “play” button over a video thumbnail). Dave Artz of AOL has some other cases where PNG8 will not be good enough. In such cases (but only after you try your best to persuade the designer to reconsider the use of transparency), make sure you use the underscore hack (_filter) so that you don’t penalize IE7 users.

Sometimes instead of PNG8 people use GIF for IE6 and truecolor PNG for the others, but that’s not necessary; with one PNG8 you achieve both binary and alpha transparency.

Additional benefits from using a PNG8 are:

1. PNG8 is usually smaller than truecolor PNG,
2. only one image to maintain for all browsers
3. cleaner CSS with no hacks, branches or proprietary tags
4. ability to repeat background

Transparency with VML

Using VML is yet another option in IE to make a truecolor PNG transparent, and it solves several problems: alpha transparency, performance, and background repeat. Unfortunatelly, it comes with the price of extra non-standard markup (or dependency on JavaScript to generate it if you want your initial markup clean) and more propritary CSS. Here’s an example on how to implement it.

If, for example, you have an empty div, you need to wrap it in one VML :rect (or :shape) and one :fill element, like this:

<v:rect>
  <v:fill type="tile" src="alphatest.png">
    <div>&nbsp;</div>
  </v:fill>
</v:rect>

Somewhere in the markup before that you also need to declare a VML namespace:

<xml:namespace ns="urn:schemas-microsoft-com:vml" prefix="v" />

And in your stylesheet you need:

v\:rect  {
    behavior:url(#default#VML);
    width: 100px;
    height: 100px;
    display: block;
}

v\:fill  {
    behavior:url(#default#VML);
}

A test page with 100 VML rect elements loads in 0.094 seconds (almost 10 times faster than using filters) and the memory usage is under 4Mb (10 times less than the filtered page).

As you can see this solution adds more markup and proprietary CSS, but it’s still a solution and doesn’t have the penalties of the AlphaImageLoader.

(Thanks go to this post by Drew Diller and also HTML Remix, who accidentally found this side effect while working on another problem — rounded corners with VML, via snook.ca)

P.S. …and What about Other Filters

AlphaImageLoader is not the only filter that exists. Another popular one is the opacity filter.

For example, for 50% element opacity developers use the properties:

• opacity: 0.5 (standard),
• -moz-opacity: 0.5 (early Mozilla versions, before Firefox 0.9), and
• for IE, filter: alpha(opacity=50).

A quick test in IE6 shows that the opacity filter is not nearly as slow as the AlphaImageLoader, but it’s still making the page slower and takes the same amount of memory. This test uses color background, not an image, but even with an image the results are pretty much the same.

opacity filter test results

test page	time, seconds	memory, MB
Test #3 – 100 divs, no opacity	0.016	0.2
Test #4 – 100 divs with opacity	0.093	46.7

Find conversations: bloglines | technorati

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

Image Optimization, Part 3: Four Steps to File Size Reduction

November 14, 2008 at 8:37 am by Stoyan Stefanov | In Development, Performance | 27 Comments

This is part 3 in an ongoing series. You can read the other parts here:

• Image Optimization Part 1: The Importance of Images
• Image Optimization Part 2: Selecting the Right File Format
• Image Optimization Part 4: Progressive JPEG…Hot or Not?

This post is about some common tools you can use to reduce the file size of your images. The idea is to be able to just take the images your designer has created and instead of using them “as is”, go ahead and tidy them up in short time and no effort, without even looking at them.

The good news is that this process is:

• lossless – you strip bytes, hence you lose some information, but not the pixel data and the resulting image looks exactly the same as the source with no quality loss
• uses free tools – all the tools we mention here are free and open-source, and work on both Windows and Unix
• automated – since these are command line tools, they are easy to script and automate; one example of such automation is the smush.it tool

Step 1: Crush PNG

PNGs store information in so-called “chunks” and not all of those chunks are required for the display of the image. In fact most of them are not. You can safely use a tool such as pngcrush and strip all the unneeded chunks. For example:

> pngcrush -rem alla -brute -reduce src.png dest.png

Let’s take a look at the options of this command:

• src.png is the source image, dest.png is the destination (result) image
• -rem alla means remove all chunks but keeps the one for transparency
• -reduce tries to reduce the number of colors in the palette if this is possible
• -brute tries over a hundred different methods for optimization in addition to the default 10. It’s slower and most of the times doesn’t improve much. But if you’re doing this process “offline”, one or two more seconds are not important since there’s a chance if a filesize win. Remove this option in performance-sensitive scenarios.

Running this command on the PNGs found on Alexa’s top 10 sites gives us an average file size reduction of 16.05%. This means you can easily strip weight off your PNG images, save bandwidth and disk space and improve load times, without sacrificing quality and without even touching a single line of application code.

PNGcrush is only one of the available tools for this sort of optimizations. Other tools you can take a look at include:

• pngrewrite
• OptiPNG
• PNGOut

Now that we’ve got a pretty good PNG solution, let’s see if we can do the same for the other image types.

Step 2: Strip JPG Metadata

JPEGs files contain meta data such as:

• comments
• application-specific (think Photoshop) meta data
• EXIF information such as camera information, date the photo was taken and even thumbnails of the actual image or audio!

This meta data is not required for the display of the image and can safely be stripped with no pixel quality loss. As discussed previously, JPEG is a lossy format, which means you lose quality every time you save. But luckily there are some operations that are lossless. Such operations include cropping a part of the image, rotation and the personal favorite – copying metadata. One tool that allows you to do these is called jpegtran.

Here’s a command to copy the source image, optimize it and don’t carry over any metadata in the new copy:

> jpegtran -copy none -optimize src.jpg dest.jpg

Note that depending on the version you have, you might need to use the syntax ending with src.jpg > dest.jpg

The -optimize option will cause jpegtran to optimize the Huffman tables and improve compression.

Running this command on Alexa top 10 sites resulted in average savings of 11.85%.

You may be able to further improve image size by using jpegtran’s -progressive option. It produces JPEGs that load progressively in the browser, starting from a lower quality version of the image and improving as new image information arrives.

Important note on stripping meta information: do it only for images that you own, because when jpegtan strips all the meta, it also strips any copyright information contained in the image file.

Step 3: GIF to PNG

What’s the best way to improve a GIF? Convert it to a PNG. As funny as it may sound, it’s true. Most of the time you get a smaller file size from a PNG and the same quality and browser support, as we discussed in a previous article. Note that PNG will not always be smaller, but most of the time it will be, so it’s worth checking after the conversion and keeping the smaller of the two files.

In order to automatically change your GIFs, you can use ImageMagick’s convert:

> convert image.gif image.png

If you want to force PNG8 format you can use:

> convert image.gif PNG8:image.png

This is probably not necessary, since GIFs will most likely be converted to a PNG8 anyway because ImageMagick picks the appropriate format based on the number of colors.

Once you’ve converted the GIF to a PNG, don’t forget to still crush the PNG result (as shown in step 1).

If the top 10 sites switch all their GIFs for PNGs (except those that don’t yield a smaller file size), on average, this will result in 20.42% file size reduction. The only inconvenience here is that you also need to write a search/replace script to find all the references to the GIF files and change them to the new PNG versions.

Step 4: Optimize GIF animations

Now that all GIFs are PNGs, PNGs are crushed and so are the JPEGs, what do we have left? GIF animations. One tool that can help you with those guys is called GIFsicle. Since the animations consist of frames and some parts of the image don’t change from one frame to another, GIFsicle doesn’t carry over the duplicate pixel information. The way to run it is:

> gifsicle -O2 src.gif > dest.gif

Smush.it

As we said at the beginning, the beauty of those four steps is that they don’t cause quality loss, so you don’t have to open and compare the results before and after. They are also all command-line tools that can be automated easily. So you have nothing to lose by running all your images through those tools before you FTP them to your web server, you can only win.

And you can always try the smush.it tool, just to get an idea of how much you can potentially save.

Find conversations: bloglines | technorati

Share and extend: Bookmark with del.icio.us | digg it! | reddit!

Image Optimization Part 2: Selecting the Right File Format

November 4, 2008 at 9:16 am by Stoyan Stefanov | In Design, Development, Performance | 23 Comments

This is part 2 in an ongoing series. You can read the other parts here:

• Image Optimization Part 1: The Importance of Images
• Image Optimization Part 3: Four Steps to File Size Reduction
• Image Optimization Part 4: Progressive JPEG…Hot or Not?

This second installment of the image optimization series talks about file formats and how to chose the right one for the job. We’ll briefly discuss the popular GIF and JPEG formats and then move on to highlighting the rock star, PNG, hopefully helping correct some misconceptions about it.

GIF

GIF is a palette (also called “indexed”) type of image. It contains a palette of indexed colors, up to 256, and for every pixel of the image there is a corresponding color index. The format is no longer subject to patent issues, so you can create GIFs without the risk of going to jail. (For more on the history of the GIF format, click here.)

GIF is a non-lossy format, which means that when you modify the image and save it, you don’t lose quality.

GIF also support animations, which, in the dark Web 1.0 ages, resulted in a plethora of blinking “new” images, rotating @ signs, birds dropping … an email, and other annoyances. In the much more civilized Web 2.0 era, we still have “loading…” animations while we wait for the results of the next Ajax request to update the page, but there are also things like the good old shiny sparkles that people like to put in their social network profiles. Nevertheless, animation support is here if you need it.

GIF also supports transparency, which is a sort of boolean type of transparency. A pixel in a GIF image is either fully transparent or it’s fully opaque.

JPEG

JPEG doesn’t have the 256 colors restriction associated with GIFs; it can contain millions of colors and it has great compression. This makes it suitable for photos and, in fact, most cameras store photos in JPEG format. It’s a lossy format, meaning you lose quality with every edit, so it’s best to store the intermediate results in a different format if you plan to have many edits. There are, however, some operations that can be performed losslessly, such as cropping a part of the image, rotating it or modifying meta information, such as comments stored in the image file.

JPEG doesn’t support transparency.

PNG

PNGs is a non-lossy format that comes in several kinds, but for practical purposes, we can think of PNGs as being of two kinds:

1. PNG8, and
2. truecolor PNGs.

PNG8 is a palette image format, just like GIF, and 8 stands for 8 bits, or 2⁸, or 256, the number of palette entries. The terms “PNG8″, “palette PNG” and “indexed PNG” are used interchangeably.

How does PNG8 compare to GIF?

• Pros:
  • it usually yields a smaller file size
  • it supports alpha (variable) transparency
• Cons:
  • no animation support

The second type of PNGs, truecolor PNGs, can have millions of colors, like JPEG. You can also sometimes come across the names PNG24 or PNG32.

And how does truecolor PNG compare to JPEG? On the pros side, it’s non-lossy and supports alpha transparency, but on the cons side, the file size is bigger. This makes truecolor PNG an ideal format as an intermediate between several edits of a JPEG and also in cases where every pixel matters and the file size doesn’t matter much, such as taking screeenshots for a help manual or some printed material.

Internet Explorer and PNG transparency

We said that both PNG types support alpha transparency, but there are some browser eccentricities that affect both types and about which you should be aware.

With PNG8, whenever you have semi-transparent pixels they appear as fully transparent in IE (version 6 and lower). This is not ideal but it’s still useful and is the same behavior that you get from a GIF. So by using a PNG8, in the worst case (IE < 7) you get the same user experience as with a GIF, while for other browsers (Firefox, Safari, Opera) you get a better experience. Below is an example that illustrates this, note how in IE6 the semi-transparent light around the bulb is missing (source: SitePoint):

For truecolor PNGs, the situation is a much less attractive compromise. All the semi transparent pixels appear grey in IE prior to version 7 (source: W3C).

IE7 introduces proper native support for alpha transparency in both PNG8 and truecolor PNGs. For earlier IE versions you need to fix the truecolor PNG transparency issue using CSS and an AlphaImageLoader filter, which we’ll discuss in more details in a follow-up article. (Spoiler alert: avoid AlphaImageLoader.)

“All we are saying is: Give PiNG a chance!”

Although PNG8 should be the preferred of the PNGs, because it’s smaller in filesize and degrades well in early IEs without special CSS filters, there are still some use cases for truecolor PNGs:

• When the 256 colors of the PNG8 are not enough, you may need a truecolor PNG. This is a case you should try to avoid. On one hand, if you have thousands and thousands of colors, this starts to look like a case where JPEG will be better suited and will give better compression. On the other hand, if the colors are around a thousand or so, you may try to convert this image to a PNG8 and see if it looks acceptable. Very often, the human eye is not sensitive enough to tell the difference between 200 and 1000 colors. That depends on the image, of course; often you can safely remove 1000 colors, but sometimes removing even 2 colors results in an unacceptable image. In any event, try your potential truecolor PNG candidate as PNG8 and as JPEG and see if you like the result in terms of quality and file size.
• When most of the image is semi-transparent. If only a small part of the image is semi-transparent, like around rounded corners, the GIF-like degradation of PNG8 is often OK. But if most of the image is translucent (think a PLAY button over a video thumbnail), you might not have a choice but to use the AlphaImageLoader hack.

At the end, let’s summarize what was discussed in this article highlighting that:

• JPEG is the format for photos.
• GIF is the format for animations.
• PNG8 is the format for everything else — icons, buttons, backgrounds, graphs…you name it.

Find conversations: bloglines | technorati

Share and extend: Bookmark with del.icio.us | digg it! | reddit!