(Cet article n’est disponible qu’en Anglais)

Update April 2th 2015 : I created a Grunt task to simplify the bridging between Blanket.js and Coveralls.io, you can find it here: grunt-qunit-blanket-lcov.

---

With QueryBuilder I did/still do a lot of experiments in build and test tools. It all began with a small Grunt to automate the minification process. Then I added QUnit tests, automatically run on Travis CI. And today I added test coverage evaluation with Blanket.js pushed to Coveralls.

As I spent many hours to configure my project, I wanted to share my experience, in a step-by-step guide about how to test my web frontend library on Travis CI and Coveralls ?. I will only talk about fontend libraries which involve HTML & CSS in addition to JavaScript, because it raises new issues.

Introduction

I will make the following assumptions: your project is hosted on Github, you know how to use Grunt and you already have QUnit tests. If not I suggest you read the Getting started Grunt guide and the QUnit introduction to unit testing.

Here are the points addressed in this guide :

• Run Qunit tests with Grunt
• Publish your project to Travis CI
• Hook Coveralls to Travis CI

An headless web browser

QUnit is basically a web application, it requires a web browser to run, thus it can’t run on NodeJS which does not have a display view. But our goal is to automatically run our tests suite on a very light virtual machine without Firefox or Chrome and still get significant results.

The solution is PhantomJS, an headless web browser ; that means it is capable of « compiling » a complete webpage, run JS scripts, trigger events, etc, without any rendering view. But running our QUnit page on Phantom and parsing the results would be a pain in the a** without this Grunt QUnit tasks. It basically spawns a new PhantomJS process for each tests suite and returns a formatted report in the console.

Installing

So in order to run the QUnit tests with Grunt, start to add « grunt-contrib-qunit » to your project :

$ npm install grunt-contrib-qunit --save-dev

Good. Now it’s time to edit your « Gruntfile.js » file. Add the following in the initConfig object :

qunit: {
    all: {
        options: {
            urls: ['tests/index.html'],
            noGlobals: true
        }
    }
}

And declare the new tasks :

grunt.loadNpmTasks('grunt-contrib-qunit');

grunt.registerTask('test', [
    'qunit'
]);

And it’s done ! You should be able to run grunt test from the console and see the tests results.

$ grunt test
Running "qunit:all" (qunit) task
Testing tests/index.html ............OK
>> 31 assertions passed (487ms)

Done, without errors.

Time to move on.

Continuous integration with Travis

Continuous integration consists in running your tests automatically everytime you push new commits. This way you are always sure that you didn’t break anything (and if it’s not the case you receive a nice email with insults !). The other big advantage is that if you use Github and Travis CI together, tests will be run for pending pull-request too and you will be able to quickly see if it’s a breaking change or not.

Installing

Starting with Travis CI is very easy, there are only two things to do.

Update your package.json file to declare the test script :

"scripts": {
    "test": "grunt test"
}

Create a « .travis.yml » file with the following content :

language: node_js
node_js:
  - "0.12"
before_install:
  - npm install -g grunt-cli
install: npm install

This tell Travis CI that your are running your tests on NodeJS 0.12 and ask it to install Grunt and your Node dependencies before running the tests.

If your tests rely on Bower dependencies you need to add :

before_install:
  - npm install -g bower
before_script:
  - bower install

Registering

Final step : register your project on Travis CI. Once logged to the Travis website with your Github account go to your profile and enable continuous integration on your project. I suggest you go to the project settings and enable « Build only if .travis.yml is present » to prevent useless builds of forks who removed the file.

Then you just have to push some commits to trigger a build and see the result online. You can even get a nice badge to add to your README : .

Time to evaluate your unit tests !

Code coverage for winners

Ok your unit tests are all passing, but are they relevant ? Do you test everything ?

That’s what a code coverage tool can tell you, it analyses which parts of your code were executed during the unit tests and return a coverage percentage, if you are not at 100% then you didn’t test everything. Caution a coverage of 100% does not mean you tested everything either, it just means that every bit of code where executed, you can still have bugs for not intended cases.

Installing

In order to calculate code coverage of our QUnit suite we will use Blanket.js, which is great because it require zero configuration.

First, install Blanket :

$ bower install blanket --save-dev

Next, edit your tests suite by adding after « qunit.js » :

<script src="../bower_components/blanket/dist/qunit/blanket.min.js"></script>

… and adding data-cover to the scripts your are testing :

<script src="../dist/query-builder.js" data-cover></script>

Then open your tests page in your browser, check « Enable coverage » and see the coverage results at the bottom of the page. You can even click on the file name and see which part of the code are not tested.

So you know your code is covered at 85%. How to automatically publish the result on Coveralls ?

Bridging

That’s where the tricky part starts. We need to get the Blanket results and send them to Coveralls API, but these results are only available inside PhantomJS process (remember ? we run the tests in an headless browser).

Hopefully Blanket has a module to export its report in a format comprehensive to Coveralls, and we also have a grunt task to upload data to Coveralls. Let’s plug everything together.

First step : modify you tests suite to send formatted coverage results back to PhantomJS, add this after other scripts :

// LCOV reporter, only if called with "lcovReport" parameter
if (location.href.match(/(\?|&)lcovReport($|&|=)/)) {
   blanket.options("reporter", "../bower_components/blanket/src/reporters/lcov_reporter.js");
   blanket.options("reporter_options", { toHTML:false });

   // send results to PhantomJS
   QUnit.done(function() {
      alert(JSON.stringify(['qunit.report', window._$blanket_LCOV]));
   });
}

Modify the qunit task in « Gruntfile.js » to use this url : tests/index.html?coverage=true&lcovReport.

Also add an event handler to get coverage data and save them in a file :

grunt.event.on('qunit.report', function(data) {
    grunt.file.write('.coverage-results/core.lcov', data);
});

Install Grunt Coveralls module :

$ npm install grunt-coveralls --save-dev

Add the coveralls task to « Gruntfile.js » :

coveralls: {
    options: {
        force: true
    },
    all: {
        src: '.coverage-results/core.lcov',
    }
}

And finally configure « .travis.yml » to send coverage report to Coveralls.io on success :

after_success: grunt coveralls

Registering

You just have to register on Coveralls with your Github account, go to your profile and add your repository. As you use public Travis CI for the build you don’t need to add any special configuration.

If for some reason you need to push coverage results from your local computer you will have to add a « .coveralls.yml » with the token you can find on Coveralls website do not commit this file.

repo_token: xxxxxxxxxxxxxxx

Now everytime you push to your repository, tests will run on Travis CI and coverage results will be sent to Coveralls, which also offer you a nice badge for your README :

Bonus : code quality

In QueryBuilder I also use JSHint to prevent coding mistakes. In this bonus part we will add a Grunt task to make the build fail the code quality check does not pass.

Start by adding Grunt contrib JSHint :

$ npm install grunt-contrib-jshint --save-dev

Edit your « Gruntfile.js » to configure the task :

jshint: {
    lib: {
        files: {
            src: ['dist/query-builder.js']
        }
    }
}

… and declare the tasks :

grunt.loadNpmTasks('grunt-contrib-jshint');

grunt.registerTask('test', [
    'jshint',
    'qunit'
]);

Now grunt test will run JSHint in addition to QUnit, and the Travis CI build will fail if you forgot a comma or made a mess with callbacks !

$ grunt test
Running "jshint:lib" (jshint) task
>> 1 file lint free.

Bibliography

• Building a Node.js project – Travis CI
• Introduction to Unit Testing – QUnit
• Automated Code Coverage Enforcement for QUnit Using Grunt and Blanket – Dave Cadwallader