Generating reports for protractor projects

When it comes to web application automation, reporting is needed at two levels. First, there needs to be a centralized location where users can get a holistic view on the status of the automation effort. This requirement is most of the time satisfied using dashboards managed by the build server (think Jenkins). Secondly, there needs to be a way for users to pull-up further information on specific tests whenever the need arise. To meet this requirement there is a second layer of reporting generated at the build level.

In this post, we will look at how reports can be generated at the two level discussed in the previous paragraph when using protractor as the automation tool  and VSTS as the build server.

1. Generating an XML results document

The first step in publishing results to the build server is to get the test framework to generate a results document in a universally accepted format. Even though there’s no industry standard when it comes to test result documents, going with Junit, the Java standard cant hurt.  This requirement can be easily met using the jasmine-reporters package[1]. Set it up in the onPrepare jasmine function as shown below.

[1] - https://www.npmjs.com/package/jasmine-reporters

2. Generating an HTML report

To improve the readability of the results, it makes sense to generate an HTML report. This can be done using quite a few report packages available in the market, but in my experience the best at the moment is the protractor-html-reporter-2[2]. It is the best aesthetically and provides all the functionally expected. Configure the package in the onComplete jasmine block as show below,

The package expects screenshots to be captured through an external component, one possible way in which this can be done is using jasmine2-protractor-utils[3]. Configure it as shown below,

Note that the screenshot path set in the plugin and the HTML report package should be the same. When we build the automation project it will now generate a Junit results document and a beautiful HTML results document.

[2] - https://www.npmjs.com/package/protractor-html-reporter-2

[3] - https://www.npmjs.com/package/jasmine2-protractor-utils

3. Publishing the results to VSTS

In order to populate the rest results in VSTS, the results generated at build time should be explicitly published once the build has completed. Now that we have an Junit results document, it can be used along with a Publish Test Results build step in VSTS build pipeline. Configure it by giving it the location of the Junit xml results file as shown below,

Now when the build completes on VSTS, the results will be available in the results tabs.