Writing Effective JavaScript Unit Tests with YUI Test

By YUI TeamJanuary 5th, 2009

One of the biggest under-the-radar movements in JavaScript development during 2008 was the reemergence of an interest in unit testing. YUI Test, YUI’s unit testing framework, reached GA status in February and other libraries either introduced their own unit testing frameworks or started publicizing existing ones. As a result, there’s a lot more documentation regarding the creation of unit tests for JavaScript. Simply having JavaScript unit tests isn’t enough, though; if your tests are written improperly, they can lead to a lot of lost time. Learning to write effective JavaScript unit tests will save you time and headaches in the future.

What are you testing?

The key to writing effective unit tests is to understand the word "unit." In testing terms, a unit is an isolated part of code that can be tested independent of other pieces of code. In an object-oriented language like JavaScript, each method is considered to be a unit. Proper OO design typically entails nicely encapsulated methods that serve a single purpose and are therefore easy to test.

Traditional unit testing is designed to test the implementation of an interface, so private methods don’t get tested explicitly. This is called black box testing. The idea is that you can swap out the implementation of an interface and the unit tests will all still pass because they are completely agnostic to the underlying implementation. All the tests know is a set of constraints that must be met; they don’t care how those constraints are met.

Writing tests

As I said in my talk, unit tests should test inputs and outputs. Inputs can be named method arguments or changes in globally accessible variables that the method depends upon to function correctly. Outputs can be return values, changes in the state of variables, and even thrown errors. For each input-output set, there should be a single unit test. Each test should explicitly state, "given these inputs, I expect these outputs." Any deviation from that statement is a failed test.

Each test should be as simple as possible and test only one input-output set; combining sets into a single test minimizes the effectiveness of the unit test. For example, consider the following test of a function called trim():

var testCase = new YAHOO.tool.TestCase({
    name: "trim() Tests",
    testTrim: function(){
        var result1 = trim(" Hello world");
        YAHOO.util.Assert.areEqual("Hello world", result1, "Leading white space should be stripped.");
        var result2 = trim("Hello world ");
        YAHOO.util.Assert.areEqual("Hello world", result2, "Trailing white space should be stripped.");
    }
});

Here, the testTrim() method of the test case is actually testing two different input-output sets:

  1. Input string has leading white space; return value has no leading white space.
  2. Input string has trailing white space; return value has no trailing white space.

The problem is that these two sets have literally no relation to one another, yet if the first input-output set fails to produce the correct result, the second set will never be tested. This is a situation where one failure masks another. It is more effective to separate out these input-output sets into two tests:

var testCase = new YAHOO.tool.TestCase({
    name: "trim() Tests",
    testTrimWithLeadingWhiteSpace: function(){
        var result = trim(" Hello world");
        YAHOO.util.Assert.areEqual("Hello world", result, "Leading white space should be stripped.");
    },
    testTrimWithTrailingWhiteSpace: function(){
        var result = trim("Hello world ");
        YAHOO.util.Assert.areEqual("Hello world", result, "Trailing white space should be stripped.");
    }
});

This code now properly tests the trim() function’s input-output sets, keeping them separate.

Unit tests are always written as if the code being tested works correctly. Good software design involves mapping out these input-output sets ahead of time so that you know exactly what the result should be in each case. In this way, unit tests become a type of technical requirement document in addition to actual code.

Effective assertions

One of the most important parts of writing unit tests is proper assertion definition. Each assertion specifies a condition that, if not met, indicates that the functionality isn’t behaving appropriately. It’s important to use only as many assertions as necessary to properly test the code output. Too many assertions can lead to false failures while too few can lead to false passes.

In the previous example, each test contains a single assertion because that is all that’s needed. I know exactly the value that to be returned and so I test specifically for that. The tests may both look very simple, but they get the job done. Again, there’s no rule about the number of assertions that make a good test, just make sure you’re testing every expected output of the code for the given input.

To make test failures more coherent, you should include a failure message with each assertion. In YUI Test, this is always the last argument of any assertion method. A failure message should tell you what should have happened, not what did happen. Some examples:

//Bad failure message
YAHOO.util.Assert.areEqual("Hello world", result, "The result wasn't 'Hello world'");

//Good failure message
YAHOO.util.Assert.areEqual("Hello world", result, "Leading white space should be stripped.");

Note the difference between the bad and good failure messages: the bad tells you what happened and the good tells you what was expected. When running your tests, a failure already indicates that something unanticipated happened, so there’s no need to simply repeat that something unanticipated happened. It’s more helpful to know what should have happened because it is an exact representation of your requirement. By taking this approach, failures end up being a list of unfulfilled requirements that you can go back over and evaluate.

Working with the DOM

JavaScript is unique to other languages in that it frequently has ties to the environment, the DOM. Methods that interact heavily with the DOM are difficult to unit test because the entire environment must be setup in order for the method to execute completely. Further complicating matters is the tendency of JavaScript to be triggered by a user action such as a mouse click. YUI Test provides event simulation to aid in creating tests for methods that are reliant on DOM interaction, however, this starts to cross over into the area of functional testing.

Functional testing, as opposed to unit testing, is designed to test the user’s experience with the product rather than input-output sets for code. If you find yourself wanting to test that the user interface responds in a specific way due to user interaction, then you really want to write some functional tests rather than unit tests. YUI Test can be used to write some basic functional tests, but the most popular (and quite good) tool for such testing is Selenium.

The best way to determine if something is a unit test is to ask if it can be written before the code that it’s designed to test actually exists. Unit tests, as part of test-driven development, are actually supposed to be written ahead of the actual code as a way to guide development efforts. Functional tests, on the other hand, cannot exist ahead of time because they are so tied to the user interface and how it changes in response to user interaction.

Structuring test hierarchies

YUI Test, just like other unit testing frameworks, supports a hierarchy of test cases and test suites. Each test suite can contain other test suites as well as test cases; only test cases can contain actual tests (methods beginning with the word "test"). The best way to organize your test hierarchy is to follow a very simple pattern:

  • Create one test suite for every object you’re going to test.
  • Create one test case for every method of an object you’re going to test and add it to the object’s test suite.
  • Create one test in each test case for each input-output set.

In this way, your test hierarchy mirrors the code you’re testing and it’s easier to figure out where new tests should be created.

Run your tests!

Perhaps the most important part of unit testing is to run your tests frequently. Testing is only effective when done on a regular basis. At a minimum, you should be running your unit tests before checking in changes to source control. Optimally, you’d also run the tests automatically on a regular basis to validate any changes after they’ve been committed to source control. This is how you’ll get the biggest benefit of unit testing: quick discernment, and hopefully prevention, of regressions.

Further information

6 Comments

  1. Are the YUI tests run automatically on a regular basis? If so can you share any information on how you do this? I’m particularly interested in how you deal with spawning and closing the browser processes used to run the tests in during an automated build, and also how you collect and aggregate the test results for the build.

  2. The best solution I’ve seen thusfar is to use Selenium to manage the browser lifecycle for testing. You can set it up to run periodically and then monitor the page for the results.

  3. First of all, thanks for writing this wonderful blog entry. I would like to comment on the following:

    In an object-oriented language like JavaScript, each method is considered to be a unit

    Personally prefer thinking of a class as a unit. The class provides me some behavior and I want to explain that behavior through my tests. Going at a method level seems that we are drilling into the implementation in some sense.

    unit tests should test inputs and outputs

    I would prefer if the you qualifies your style of unit testing as state based unit testing. There is a different school of unit testing which believes in interaction based testing. Where its not so much about the output in terms of return values, changes in the state of variables, or thrown errors. The focus is on the interaction (method calls) with it’s collaborators.

    In YUI Test, failure message is always the last argument of any assertion method

    They seem to have broken the xUnit convention. in xUnit failure message is always the first argument.

    You claims:

    //Bad failure message
    YAHOO.util.Assert.areEqual("Hello world", result, "The result wasn't 'Hello world'");

    //Good failure message
    YAHOO.util.Assert.areEqual("Hello world", result, "Leading white space should be stripped.");

    While I agree with you that the second example is better than the first one, but I think the failure message in the second case is redundant. If you look at the whole test method below,

    testTrimWithLeadingWhiteSpace: function(){
    var result = trim(" Hello world");
    YAHOO.util.Assert.areEqual("Hello world", result, "Leading white space should be stripped.");
    }

    The method name already tells you that leading white space should be trimmed or stripped out. In this specific case I would not give any failure message. If its your company’s best practice to give one, then I would rather prefer your failure message as “Leading white space was not stripped”

    Functional tests, on the other hand, cannot exist ahead of time because they are so tied to the user interface and how it changes in response to user interaction.

    This is not always true. In a lot of cases you can actually write your functional or acceptance tests before you create any code. Refer to Acceptance Test Driven Development

    Create one test suite for every object you’re going to test

    Why is this important? I don’t see how this will help?

    Create one test case for every method of an object you’re going to test and add it to the object’s test suite.

    I would rather create a test class per important entity in my production code and write one test method per behavior the class fulfills. Sometimes I might have more than 1 test for one method and sometimes none. I’m not sure if I would create a guideline saying every method should have one test.

  4. Hi Naresh,

    Thanks for your insightful comments. Indeed, there are many schools of thoughts surrounding test methodology, and you’ve touched on a few of them. I’d just like to clarify a couple of the points.

    Making the assertion failure message a statement of what should be happening isn’t redundant, it’s explanatory of what just happened. The test name and the failure message may be similar, and in this case are because the test is so simple; however, they may be very different if there are a number of assertions contained within the test.

    My recommendation for setting up test suites and test cases is to get people started. I’m not saying that every method should have one test, I’m saying that every method should have a test case containing tests that explore each input-output set that you have. The point is to have a logical structure that maps directly to the objects that you are programming.

    I hope this clarifies some of the points; thanks for your comments.

  5. Hi Nicholas,
    I recently started exploring yuitest for unit testing of JS code on an existing portal that we have inherited for support and is already using some yui library 2. thanks for all the videos and blogs that you have put out.
    1. are there examples/samples of yuitest being used in more practical/realistic scenarios like different contexts, extensive dom usgae etc. examples on yuitest site are very simplistic.
    2. reason i ask is because content on pages is generally dynamic and is based on some pre-conditions..the container element is same but content changes. is it advisable to stub out different htmls for each scenario?
    3. there could be some scripts that use lot of DOM. should entire html like original html be stubbed out in such cases.

  6. Hi Kaanta,

    All of the examples I can share are currently in the documentation or included as part of the YUI download (check the tests directory in the ZIP file).

    If you’re testing a lot of interaction on a page, then I usually suggest loading YUI Test onto the actual page and running the tests there. That way, you don’t have to worry about stubbing out certain pieces of the page to things working.