October 19, 2016


Automation Testing with Jenkins: The Game Changer in Test Automation

If we aren’t using Continuous Integration (CI) with our automated testing setup, we’re really missing out on something big.  In my honest opinion, CI with automated tests is today a game-changer in Test Automation.

One of the basic principles of Continuous Integration is that a build should be verifiable. One should be able to objectively determine whether a particular build is ready to proceed to the next stage of the process, and the most efficient way to do this is by using automated tests. Without proper automated testing, you find yourself having to retain many build artifacts and test them manually.

The benefits of Continuous Integration are closely related to solid testing practices. Nowadays, practices like Test-Driven Development are more widely used, resulting in a growing confidence in the results of the automated builds. The application is no longer simply compiled and tested, but if the tests pass, it is automatically deployed to an application server for more comprehensive end-to-end tests and performance tests.

Why Jenkins with Automation Testing?

Following are some of the key features of Jenkins automated testing which makes the life of an automation tester using automation tools like Selenium, much easier.

1. Jenkins can schedule tests to run at a specific time.

Jenkins automated testing comes with an excellent functionality that allows the testers to schedule test cases and run them as per their convenience. This definitely saves the testers’ time and effort significantly.

2. Jenkins displays test result trends

Jenkins automation displays test result trends on the project home page. The user can clearly see how the tests have been faring over time—test failures from previous builds will also appear as red in the Test Result Trend graph. This provides the user with insights to take appropriate actions.

3. Jenkins displays a summary of the test results

One of the most critical tasks of a tester is to report on the test results. Not just the test failures, but also the number of tests that were executed, how long did it take to execute, and so on. Jenkins provides a summary of the test results, organized by high-level packages rather than modules to simplify the process.

4. Jenkins displays the details of a test failure

When a test fails, the tester would definitely want to know why. With Jenkins automated testing you can access the details of a particular test failure by just single click on the corresponding link. It will display all the complex details, including the error message and the stack trace, as well as a reminder of how long the test has been failing. This is an indicator of either a tricky technical problem in the automated test cases that might need attention, or a complacent attitude to failed builds which is more serious and definitely should be investigated closely.

5. Build time trends can be a good indicator of how fast the tests are running

Jenkins automation can easily tell us how long the tests have been taking to run over time. Jenkins provides a graph along the lines of the one in the Figure,

Now, tests are not the only things that happen in a build job, but if we have too many tests to worry about, they will take a large proportion of the time. Hence, this graph is a great way to see how well the tests are performing without consuming too much time.

6. Jenkins can automatically trigger the Test Results mails after the execution of test suits.

Out of the box, Jenkins provides support for email notifications. This can be activated by ticking the “E-mail Notification” checkbox in the ‘Post-build Actions’ and just entering the email addresses of the team members who will need to know when the test suits have been completed along with the test results and the screenshots of the failed test cases.

Continuous Integration is a critical part of Automation Testing in any environment, and should be taken very seriously. As in the other areas of CI, and perhaps even more so here, feedback is critical to ensure that all the tests are executed as fast as possible, even the integration and functional ones.

Reference Links:

  1. Jenkins The Definitive Guide by John Ferguson Smart
  2. http://www.guru99.com/maven-jenkins-with-selenium-complete-tutorial.html