Generating reports for protractor projects

When it comes to web application automation, reporting is needed at two levels. First, there needs to be a centralized location where users can get a holistic view on the status of the automation effort. This requirement is most of the time satisfied using dashboards managed by the build server (think Jenkins). Secondly, there needs to be a way for users to pull-up further information on specific tests whenever the need arise. To meet this requirement there is a second layer of reporting generated at the build level.

In this post, we will look at how reports can be generated at the two level discussed in the previous paragraph when using protractor as the automation tool  and VSTS as the build server.

1. Generating an XML results document

The first step in publishing results to the build server is to get the test framework to generate a results document in a universally accepted format. Even though there’s no industry standard when it comes to test result documents, going with Junit, the Java standard cant hurt.  This requirement can be easily met using the jasmine-reporters package[1]. Set it up in the onPrepare jasmine function as shown below.

[1] - https://www.npmjs.com/package/jasmine-reporters

2. Generating an HTML report

To improve the readability of the results, it makes sense to generate an HTML report. This can be done using quite a few report packages available in the market, but in my experience the best at the moment is the protractor-html-reporter-2[2]. It is the best aesthetically and provides all the functionally expected. Configure the package in the onComplete jasmine block as show below,

The package expects screenshots to be captured through an external component, one possible way in which this can be done is using jasmine2-protractor-utils[3]. Configure it as shown below,

Note that the screenshot path set in the plugin and the HTML report package should be the same. When we build the automation project it will now generate a Junit results document and a beautiful HTML results document.

[2] - https://www.npmjs.com/package/protractor-html-reporter-2

[3] - https://www.npmjs.com/package/jasmine2-protractor-utils

3. Publishing the results to VSTS

In order to populate the rest results in VSTS, the results generated at build time should be explicitly published once the build has completed. Now that we have an Junit results document, it can be used along with a Publish Test Results build step in VSTS build pipeline. Configure it by giving it the location of the Junit xml results file as shown below,

Now when the build completes on VSTS, the results will be available in the results tabs.

Moving from Selenium to Protractor for Web Automation

99x Technology invests heavily in Quality Assurance, and this one of the reasons why the company has been able to carve out a name as a trusted product engineering company. The brunt of the QA work at 99x is achieved using automation, specifically protractor. 

As someone who is familiar with Selenium,TestNG and Java, I found myself a little intimidated at the prospect of learning a new stack including a new language(javascript) but to my delight I found the experience to be relatively pain-free and intuitive. In hindsight, I can clearly see the reasons as to why making the shift from Selenium to protractor makes sense.The main reasons as I see it are,

The fact that Angular and JS is taking over the web

As a technology invented and backed by Google, it was just a matter of time before AngularJS piggy backed into becoming the dominant web application technology around. Naturally, as Protractor is custom built to test angular apps, it makes sense to use it to test angular apps.  

Not having to meddle around with the web drivers 

Downloading the latest greatest web driver for each browser being tested and then setting up the driver in code and all the other steps that follow are pains associated with Selenium. As protractor is built as an API encapsulating selenium, it takes care of most of these pains. All you have to do is run the command to update and start the drivers and the API takes care of the rest. 

Not having to meddle around browser weights(sort of!)

Unlike Selenium Protractor provides is hard-wired for implicit weights for elements, it also supports explicit waits in the way of actions and browser sleep operations.

The backing of a Healthy community

Personally, i found the protractor community to be more active compared to that of Selenium’s. Maybe this is because its relatively. As protractor is built as an Node.JS package, it benefits from the NPM community. You can easily find extensions like reporters and data providers.    

....

Moving on, when making the switch(given you’re moving from java to javascript,  I did), give special attention the fact that JS is an asynchronous language, meaning, the language may process two program statements inside a code block parallelly as opposed to processing them sequentially as we expect in Java. The protractor API hides the complexities of working with the asynchronous nature of JS whenever you’re performing an action on the browser but whenever you’re receiving data from the browser to the test code, you need to explicitly handle the logic required to handle the parallelism, this is done through a mechanism named promises.

I’ve put together a project[1] that automates a set of test-cases for a banking app put up by way2automation tutorials site. The project uses most features available in protractor as well as some of the main functionality of jasmine and some of its extensions such as the html/screenshot reporter. The project embeds protractor as node module, therefore it can be executed as follows(not the project is expected to be run on windows).

npm run webdriver-update

npm run webdriver-start

npm run start