Combo Handler Service Available for Yahoo-hosted JS

Blog
About

Site Search:

Combo Handler Service Available for Yahoo-hosted JS

July 16, 2008 at 11:16 am by Eric Miraglia | In Development, Performance |

We’ve been talking for a long time at Yahoo about the importance of minimizing HTTP requests to improve performance. One important technique for YUI users has long been to use the pre-built "rollup" files (like yahoo-dom-event.js, which combines the YUI Core in a single minified HTTP request) and to create custom rollups that aggregate all of your YUI JS content in a single file. You’ll notice that we do a lot of this on our core Yahoo properties. For example, if you go to check on the Tour de France on Yahoo! Sports, you’ll find that numerous YUI components are aggregated with custom Sports-specific JS resources in a single HTTP request (here’s the aggregate file).

Thanks to the hard work of the Yahoo Exceptional Performance team and the groups that support our CDN, we’re now able to offer ad-hoc file aggregation — "combo handling" — to file served from yui.yahooapis.com. So, a request for the full YUI Rich Text Editor, which previously looked like this…

<script type="text/javascript" 
   src="http://yui.yahooapis.com/2.5.2/build/yahoo-dom-event/yahoo-dom-event.js"></script>
<script type="text/javascript" 
   src="http://yui.yahooapis.com/2.5.2/build/container/container_core-min.js"></script> 
<script type="text/javascript" 
   src="http://yui.yahooapis.com/2.5.2/build/menu/menu-min.js"></script> 
<script type="text/javascript" 
   src="http://yui.yahooapis.com/2.5.2/build/element/element-beta-min.js"></script> 
<script type="text/javascript" 
   src="http://yui.yahooapis.com/2.5.2/build/button/button-min.js"></script>
<script type="text/javascript" 
   src="http://yui.yahooapis.com/2.5.2/build/editor/editor-beta-min.js"></script>

…can now be written this way:

<script type="text/javascript" 
src="http://yui.yahooapis.com/combo?2.5.2/build/yahoo-dom-event/yahoo-dom-event.js&
2.5.2/build/container/container_core-min.js&2.5.2/build/menu/menu-min.js&
2.5.2/build/element/element-beta-min.js&2.5.2/build/button/button-min.js&
2.5.2/build/editor/editor-beta-min.js"></script>

In one step, this eliminates five separate HTTP requests.

A few notes regarding combo handling on yui.yahooapis.com:

If you’re using the YUI Configurator, this option ("Combine All JS Files") is enabled by default as long as you’re using the default base path.
Combo-handling of YUI CSS files is not supported at this time.
In an upcoming release, we’ll provide built-in combo-handling support in YUI Loader and restructure filepaths in YUI’s CSS resources to make them combinable as well.
YUI Configurator will always output the current version of the library, but all YUI JS files from 2.2.0 onward are present on yui.yahooapis.com and can be combined using the same combo-handling syntax.

We hope combo handling provides a easy performance win for those of you letting Yahoo serve your YUI files. Discussion of combo handling and all YUI issues takes place in our community forum — please join us there and let us know how this works for you.

Share and extend: Bookmark with Yahoo! My Web | Bookmark with del.icio.us | digg it! | reddit!

20 Comments »

RSS feed for comments on this post.

This is a huge improvement and an area we had been worrying about as we added more scripts to our web application. Thanks to the YUI team :)

Comment by davemo — July 16, 2008 #
This is really great.
Will a combine php script be included in future releases for people who want to host their own copies of YUI?

-Ed

Comment by Ed Haber — July 16, 2008 #
@Ed — We’d like to see that happen, whether through YUI or YDN. You can count on us trying to grease the wheel for that! -Eric

Comment by Eric Miraglia — July 16, 2008 #
This is really great news! Will the aggregated files take full advantage of caching, far-future expiry, etags, gzip… etc?

Comment by Luke Chambers — July 16, 2008 #
Repeating the version number and the entire filename seems redundant.

http://yui.yahooapis.com/2.5.2/combo-min?yahoo-dom-event&menu&container&button&element&editor

this would be more readable

Comment by John — July 16, 2008 #
@Ed,

I did this a bit ago and outlined it here: http://www.jaisenmathai.com/blog/2008/02/27/speed-up-apache-how-i-went-from-f-to-a-in-yslow/

Also, I store the result in a static file or memcached so that every request doesn’t need to piece the javascript together.

Comment by Jaisen — July 16, 2008 #
Eric,

This is a really cool idea and it’s great to see an extension to the original hosted scripts.

One thing that might be a problem though is the way component files are passed to the combo script as query string parameters. As far as I’m aware the HTTP specifications state (rather un-clearly) that browsers shouldn’t cache requests containing query string components. Internet Explorer and Firefox do I think, but Safari and Opera don’t.

A rewritten form (mod rewrite or similar) form of the overall URL would solve this problem if indeed, as I suspect, it is an issue.

Ed

Comment by Ed Eliot — July 16, 2008 #
This is great guys!

Comment by Nick Sullivan — July 16, 2008 #
@Ed — Nice to hear from you! The querystring issue is an interesting one. W3c specifies that the presence of a querystring prevents caching *unless the response header has explicit expiry information*. [ http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.9 ]. The correct headers are being set on combo-handled files (far-future expiry, etc.), so the querystring parameter should be innocuous. (I am seeing normal caching in all tested browsers.)

@Luke — the above speaks to your point as well. From an edge-hosting perspective, this is just like any other GET request. It hits the origin server if the resource is not in the edge cache, but it’s cached on the edge once retrieved.

@John — No disagreement on this, other than to say this is a very simple and flexible pattern that allows the combo handler script to be generic. To the extent that this causes you to spend lots of bytes on the URI for your scripts, there may be an argument for rolling/hosting your own custom aggregate.

Comment by Eric Miraglia — July 16, 2008 #
Why not just src=”http://yui.yahooapis.com/combo?2.5.2/build/editor-beta-min”

I don’t really need to know what the dependencies are, right??

Comment by Michiel van der Blonk — July 16, 2008 #
@Michiel — A combo handler engine that intrinsically knows about YUI dependencies and could calculate a dependency tree from a component list would be a nice next step — something that works the way YUI Loader does on the client side. For now, you don’t need to know about the dependencies — you can rely on the Configurator (http://developer.yahoo.com/yui/articles/hosting/#configure ) to generate the combo URI for you. But the combo handler doesn’t do dependency calculation today, so that needs to be done in advance. -Eric

Comment by Eric Miraglia — July 16, 2008 #
@Eric - thanks for the spec reference. I had an interesting debate on this issue with some of the guys in London a few months ago and we were concerned it was a problem. It’s cool that your testing proves that all browsers do follow as expected.

@Ed - I created a PHP script for merging resources (JS and CSS) a year or so back which will work fine with locally hosted copies of the YUI. It regenerates the merged file automatically as component files changes, adding versioning information and maintains and archive of old revisions to ensure that old references in locally cached HTML can still retrieve the correct CSS & JS.

Comment by Ed Eliot — July 16, 2008 #
Cool was waiting for something like that ;-)

Comment by Asvin Balloo — July 17, 2008 #
@Ed

I’ve written one that didn’t work exactly how I wanted it to. But i’ve been meaning to revisit this.

@Jaisen

I’m taking a loot at what you did. I started noticing a long load time and its several seconds of loading individual js files or if they are cached checking on the files and seeing that they are not modified.

@John

I agree. I really like the more terse format you are suggesting. Is it ever recommended to include files from different versions of YUI?

-Ed

Comment by Ed Haber — July 17, 2008 #
Hey Eric, This is great. As we’ve been using more and more features our js hits have bloated. This is the perfect remedy. Thanks for reading my mind :)

Comment by Jim Morris — July 17, 2008 #
@Ed - Minify is a PHP library for combining and serving minified JS/CSS files with HTTP caching and compression.

Comment by Steve Clay — July 18, 2008 #
This is VERY nice guys! Especially sites that use a lot of widgets will benefit enormously from this service.

Comment by Marco — July 20, 2008 #
I noticed the YUI Configurator contains a blank combo link if you just specify the css files…

Comment by Drew Freyling — July 21, 2008 #
@Drew–Thanks for the catch. -Eric

Comment by Eric Miraglia — July 22, 2008 #
I’m working for Kelkoo Front-End team and I have already developped a script which is able to :
. read a configuration file where a list of JS files is declared. For each file, some options can be set (has to be minify or not, has to be included in the final file, has to be obfuscated, …).
. launch JSLint on each of the files and send a mail to a mailing list with only the diff between the old version and the new one.
. launch YUICompressor to compress the file with the given options (obfuscated or not)
. concat all JS files into a bigger one and commit this file to a CVS server if it is different from the older version.

On Kelkoo, we have several “global” files, each one is included on somes pages (depending on the needs : no need to include a carousel on a page without one for example :) ). So, my script is able to handle this and to generate a set of global files. Currently, it works for a few weeks, every hour on separate servers. Webdevs just need to commit their “small” file and global files are automatically generated and commited but in an asynchronous way.

On development servers, we have a PHP script which generates global JS files on the fly using the same configuration file.

In conclusion, thanks to those tools, we only have one JS file included on each pages and we have increase our performances. Nevertheless, we still need to have a dynamic-generated JS file on production servers for setting some JS configuration. :(

Comment by Remi Alvado — July 24, 2008 #

Yahoo! User Interface Blog