Let's learn about defect reports

 

The difference between errors and defects

A couple of months ago I came to the conclusion that it is a lot faster to push out vblog entries than to hunch over my laptop and type away until I'm satisfied with the outcome.

In this entry, I speak about the difference between errors and defects.

Why SQE is hard and why it’s still worthwhile to pursue it as a career Pt. 2

In my previous post, we looked at why SQE is hard. So then why should anyone consider a career in it? The short answer is, you should consider it if it works for you, but that’s no answer is it?

Here are 5 good reasons I can think of off the top of my head, 

1) It will give you flexibility in your career

SQE has a wide breadth. Contrary to popular opinion, SQE doesn’t start after the developer compiles the code/project and sends out an email about it and gloats about it to Jane from pre-sales for 2 hours. No, SQE spans the entire SDLC (whichever model it may be).

As you progress in your career in SQE you will get more opportunities to shift left and get involved earlier on in the process. Product Owners, Project Managers, and members from other business functions will value the technical exposure you have had at the ground level. In my experience, I have seen engineers move left but I don’t think I have heard of anyone moving from left to right.

The other side of the story is SQE's develop more carry-forward skills(A.K.A. Transferable Skills) than developers. For example, the testing body of knowledge is not bound by any technology or tool. Yes, if you join a new company you will still need to learn to use their automation framework or other technologies but you won’t need to learn how to design test cases or how to manage defect reports again.  

2) It's a stable software engineering discipline  

This is because, one, the carry forward skills SQEs develop. It’s a lot easier for an SQE to jump companies after putting in 5 years in a niche domain than a developer.

Two, the relatively slow speed of change in SQE tools, methods, and technologies. Because there isn’t as much money to be made making tools for SQEs as there is for developers, the rate of change and new entries to the market is low.   

3) It's a relatively safe way to keep scratching the software engineering itch

The world has a way of giving you what you ask of it and the truth of the matter is if you’re in software engineering because of an itch to understand the nuts and bolts, then because of the nature of the beast sooner or later there’s going to be way too many types of nuts and bolts for you to even have an idea of where they all fit. Yes, you can always put in the time and figure out where a particular type of nut or bolt should go but then you quickly come to the realization that you don’t have that kind of time and that sometimes people just keep handing you more just because you’re not just in it for the money.

   

Now if you are a developer, to climb the ranks (in your organization or community) you need to show your expertise(maybe the level of expertise you need to show is negatively correlated to how good you are with your people skills and social standing but let’s not get into it <wing tongue out emoji>). This is hard because sometimes the technologies and tools the management pushes down on you are unproven or have no easy way of learning let alone mastering. So you can find yourself between a rock and a hard place.

The situation is better if you’re in SQE. Depending on how technical of SQE role you’re playing you may be expected to know absolutely nothing about the innards of a System (think black box or COTS UAT) or just enough to verify and validate the developers' work. This translates to safety. You can learn new things or the same things as the developer but at your own speed without risking a mess.  

      

4) You get to shape the quality of a product

As an SQE you will have a say in the quality of the products that run through you. Unless your company is banking on a cost advantage it makes sense for them to push out the highest quality products. Even if the price is your company's competitive advantage, if the quality is lacking new arrivals can take market share. If you are someone who really appreciates quality over quantity, chances are you will find SQE work inherently rewarding and you will grow.

5) Pay is good

Depending on how technical the SQE role is and a few other factors like how good you are at your work and how well your company is doing, SQEs can expect to be paid as much as developers.

Why SQE is hard and why it’s still worthwhile to pursue it as a career Pt. 1

Learning SQE is like learning to play the drums, anyone can learn to keep a beat, but very few can master the instrument. SQE is hard because of its breadth and depth. Let me stretch the drumming analogy further and explore this idea.

For the world to start calling you a great drummer, you need to be able to keep the beat and add bells and whistles regardless of the type of music the rest of your bandmates might start to play. Similarly, to be a good SQE you need to be good at a bunch of things. You need to,

1. Be good with the SQE body of knowledge: The process, the levels, the types, test design techniques etc.
2. Be good with functional testing (both manual and automated).
3. Be good with non-functional testing (including load, reliability, stress, and security)
4. Be able to do static testing.
5. Be able to manage stakeholders.

I could have made the above list even more granular, but I think you get the idea. The breadth of SQE is wide.

So, you managed to keep the beat going and added the bells and whistles to everything the rest of the band threw at you. Now the band is grooving, and the next thing you know, ‘the man’ opens the thick soundproofed door to the studio, hold a wide grin and say,

‘Kid... Pack your bags because you’re getting the golden ticket.’

You’re a little suspicious of the grin but you abide. Now you’re in the big leagues. Playing clubs with the big boys. The bells and whistles you added in on the run-down drum-kit back at home is not going to cut it here. The other drummers have 12-piece sets, they know how to mic the drum kit just right, they know how to sweet talk the owners of the clubs and man do they know how to drum.

Getting back to SQE, its not just enough to wet your feet in the 5 verticals I listed at the start. You need to climb down deeper into each of them. Here are a few tips if you're just getting started,

1. To be good at the SQE body of knowledge:  Study senior testers. After a few years, I recommend you read the ISTQB syllabus (or better yet get certified). This way you will easily be able to map real-world experiences to what you will find in the syllabus. It will stick in your mind better.
2. To be good at functional testing: Start off with a manual functional testing job and learn on the job. Understand how the testing activities are carried out across the different test levels. Get exposure and then move to automation. Spend a few years in test automation learning on the job. The key is to be familiar with the general process so that if you need to switch the technology stack you can still adjust in time.
3. To be good at non-functional: Get a good grasp of reliability, stress and load testing. Like with automation, Focus on the general process and concepts rather than on specific tools but do get real-world experience using a popular tool.
4. To be good at static testing: Keep you’re programming skills sharp. Take on side projects even if they’re basic, just to keep your coding and DevOps skills sharp. Learn on the side, this way you won't feel lost if you need to do shift-left. Look for opportunities to get exposure.
5. To manage stakeholders better: Learn on the job, study test managers and leads. Put yourself out there and venture out into management studies consider an MBA.

Nothing can substitute for on-the-job exposure. So, if you’re starting out try to join a company that can give that to you or work with your company so that you can chart a way forward. Like with other software engineering disciplines, learning stops when you stop engineering, so keep learning!

Sounds like a lot of work? Yes, it is. So in the next post, I’ll touch on why SQE is still worth all the trouble. 

Even lovers need a holiday

It’s been a while! I last wrote just before I left 99x (a great place to work, give them a try), but to explain my absence I need to rewind 4-5 years further.

I shifted from Technical Writing to SQE in 2013 right after I got my bachelor's (The reasons for this shift are better covered in a future post). I walked through the doors of the company as a novice software quality engineer not knowing a defect from an error. It also didn’t help that the company was probably the techiest employer in the country.

WSO2 does open source B2B, so naturally, development was fast phased and at the cutting edge. The products were built and shipped fast using various reputable open-source components. The product line aimed to cater to all the needs an enterprise might possibly have as they moved along their digital transformation journeys. So, for a fresh off-the-mill engineer like I was at the time, it was a fat kid at the buffet kind of situation (Honestly, I’m not body shaming here. The phrase just felt right in the sentence).

The testing work my team did was mostly manual and on the fly. In hindsight, I can understand why. For such fast phased open-source development and for the business model employed by the company it just didn’t make sense to automate each and everything thing above the component layer before shipping or be uber methodical (bare in mind this was some time ago, things may have changed since I left). So, I walked out with great technical exposure (for a tester) but little exposure to general testing processes and test automation.

The next couple of years I spent polishing up my automation skills and landed a great job at Sysco LABS. The company is a favored employer in Sri Lanka and in my opinion, a great place to learn carry forward in demand SQE skills. This is what I did for the past 3 years. I humbly sharpened my core testing skills by learning and using them all on the job. I am very fortunate to have got the opportunity to cover the full QE stack at the company.

I am definitely a better SQE to have had the fast paced startup testing experience at WSO2 and the methodical process-oriented testing experience at Sysco LABS. 

Well, all of what I’ve written so far doesn’t really answer why I haven’t blogged for 5-6 years. Well, I was busy. I had a lot on my plate. Things to learn on the job, a kid to care for, to get in shape, and I somehow found the time to get my masters. Now that the master's is out of the way, and I’ve put some miles behind the job I should be able to push out a post every now and then. 

Next time with some real SQE stuff...

Generating reports for protractor projects

When it comes to web application automation, reporting is needed at two levels. First, there needs to be a centralized location where users can get a holistic view on the status of the automation effort. This requirement is most of the time satisfied using dashboards managed by the build server (think Jenkins). Secondly, there needs to be a way for users to pull-up further information on specific tests whenever the need arise. To meet this requirement there is a second layer of reporting generated at the build level.

In this post, we will look at how reports can be generated at the two level discussed in the previous paragraph when using protractor as the automation tool  and VSTS as the build server.

1. Generating an XML results document

The first step in publishing results to the build server is to get the test framework to generate a results document in a universally accepted format. Even though there’s no industry standard when it comes to test result documents, going with Junit, the Java standard cant hurt.  This requirement can be easily met using the jasmine-reporters package[1]. Set it up in the onPrepare jasmine function as shown below.

[1] - https://www.npmjs.com/package/jasmine-reporters

2. Generating an HTML report

To improve the readability of the results, it makes sense to generate an HTML report. This can be done using quite a few report packages available in the market, but in my experience the best at the moment is the protractor-html-reporter-2[2]. It is the best aesthetically and provides all the functionally expected. Configure the package in the onComplete jasmine block as show below,

The package expects screenshots to be captured through an external component, one possible way in which this can be done is using jasmine2-protractor-utils[3]. Configure it as shown below,

Note that the screenshot path set in the plugin and the HTML report package should be the same. When we build the automation project it will now generate a Junit results document and a beautiful HTML results document.

[2] - https://www.npmjs.com/package/protractor-html-reporter-2

[3] - https://www.npmjs.com/package/jasmine2-protractor-utils

3. Publishing the results to VSTS

In order to populate the rest results in VSTS, the results generated at build time should be explicitly published once the build has completed. Now that we have an Junit results document, it can be used along with a Publish Test Results build step in VSTS build pipeline. Configure it by giving it the location of the Junit xml results file as shown below,

Now when the build completes on VSTS, the results will be available in the results tabs.

Moving from Selenium to Protractor for Web Automation

99x Technology invests heavily in Quality Assurance, and this one of the reasons why the company has been able to carve out a name as a trusted product engineering company. The brunt of the QA work at 99x is achieved using automation, specifically protractor. 

As someone who is familiar with Selenium,TestNG and Java, I found myself a little intimidated at the prospect of learning a new stack including a new language(javascript) but to my delight I found the experience to be relatively pain-free and intuitive. In hindsight, I can clearly see the reasons as to why making the shift from Selenium to protractor makes sense.The main reasons as I see it are,

The fact that Angular and JS is taking over the web

As a technology invented and backed by Google, it was just a matter of time before AngularJS piggy backed into becoming the dominant web application technology around. Naturally, as Protractor is custom built to test angular apps, it makes sense to use it to test angular apps.  

Not having to meddle around with the web drivers 

Downloading the latest greatest web driver for each browser being tested and then setting up the driver in code and all the other steps that follow are pains associated with Selenium. As protractor is built as an API encapsulating selenium, it takes care of most of these pains. All you have to do is run the command to update and start the drivers and the API takes care of the rest. 

Not having to meddle around browser weights(sort of!)

Unlike Selenium Protractor provides is hard-wired for implicit weights for elements, it also supports explicit waits in the way of actions and browser sleep operations.

The backing of a Healthy community

Personally, i found the protractor community to be more active compared to that of Selenium’s. Maybe this is because its relatively. As protractor is built as an Node.JS package, it benefits from the NPM community. You can easily find extensions like reporters and data providers.    

....

Moving on, when making the switch(given you’re moving from java to javascript,  I did), give special attention the fact that JS is an asynchronous language, meaning, the language may process two program statements inside a code block parallelly as opposed to processing them sequentially as we expect in Java. The protractor API hides the complexities of working with the asynchronous nature of JS whenever you’re performing an action on the browser but whenever you’re receiving data from the browser to the test code, you need to explicitly handle the logic required to handle the parallelism, this is done through a mechanism named promises.

I’ve put together a project[1] that automates a set of test-cases for a banking app put up by way2automation tutorials site. The project uses most features available in protractor as well as some of the main functionality of jasmine and some of its extensions such as the html/screenshot reporter. The project embeds protractor as node module, therefore it can be executed as follows(not the project is expected to be run on windows).

npm run webdriver-update

npm run webdriver-start

npm run start