Jenkins - A Centralized Tool to run your Automated Tests

Jenkins is an award-winning application that monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Jenkins focuses on the following two jobs:

    •    Building/testing software projects continuously, just like CruiseControl or DamageControl. In a nutshell, Jenkins provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases the productivity.
    •    Monitoring executions of externally-run jobs, such as cron jobs and procmail jobs, even those that are run on a remote machine. For example, with cron, all you receive is regular e-mails that capture the output, and it is up to you to look at them diligently and notice when it broke. Jenkins keeps those outputs and makes it easy for you to notice when something is wrong.


Jenkins is an open-source continuous integration software tool written in the Javaprogramming language for testing and reporting on isolated changes in a larger code base in real time. The software enables developers to find and solve defects in a code base rapidly and to automate testing of their builds.

Continuous integration has evolved since its conception. Originally, a daily build was the standard. Now, the usual rule is for each team member to submit work on a daily (or more frequent) basis and for a build to be conducted with each significant change. When used properly, continuous integration provides various benefits, such as constant feedback on the status of the software. Because CI detects deficiencies early on in development, defects are typically smaller, less complex and easier to resolve.

Jenkins is a fork of a project called Hudson, which is trademarked by Oracle and is currently being developed parallel to Jenkins. The development community and its governing body hosts open meetings about the software.

Most of the times Test team will end up writing different test harnesses to test different components and there should be  a platform from which a user can start a run of specific automation suite on a specified environment with given configurations remotely using a web page.

Features
    •    Single Sign-on for all your Automation Needs
    •    Common Console for starting any Automation Run
    •    Configure all your Automation Suites from a single place
    •    Horizontal coverage by adding all automation suites
    •    Vertical dig down possible for an Automation Engineer to understand root cause for Test failures
    •    Email Notification once run in completed
    •    Confluence Page is updated with results/custom message once run is completed
    •    Code Coverage can be published

Performance Testing vs Load Testing vs Stress Testing - Examples

Performance testing - It is performed to evaluate the performance of components of a particular system in a specific situation. It very wide term. It includes: Load Testing, Stress Testing, capacity testing, volume testing, endurance testing, spike testing, scalability testing and reliability testing etc. This type of testing generally does not give pass or fail. It is basically done to set the benchmark & standard of the application against Concurrency / Throughput, Server response time, Latency, Render response time etc. In other words, you can say it is technical & formal evaluation for responsiveness, speed, scalability and stability characteristics.

Load Testing is subset of performance testing. It is done by constantly increasing the load on the application under test till the time it reaches the threshold limit. The main goal of load testing is to identify the upper limit of the system in terms of database, hardware and network etc. The common goal of doing the load testing is to set the SLAs for the application.

Example of load testing can be:
Running multiple applications on a computer simultaneously - starting with one application, then start second application, then third and so on....Now see the performance of your computer.
Endurance test is also a part of load testing which used to calculate metrics like Mean Time Between Failure and Mean Time to Failure.

Load Testing helps to determine:
    •    Throughput
    •    Peak Production Load
    •    Adequacy of H/W environment
    •    Load balancing requirements
    •    How many users application can handle with optimal performance results
    •    How many users hardware can handle with optimal performance results

Stress testing - It is done to evaluate the application's behaviour beyond normal or peak load conditions. It is basically testing the functionality of the application under high loads. Normally these are related to synchronization issues, memory leaks or race conditions etc. Some testing experts also call it as fatigue testing. Sometimes, it becomes difficult to set up a controlled environment before running the test. Example of Stress testing is:

A banking application can take a maximum user load of 20000 concurrent users. Increase the load to 21000 and do some transaction like deposit or withdraw. As soon as you did the transaction, banking application server database will sync with ATM database server. Now check with the user load of 21000 does this sync happened successfully. Now repeat the same test with 22000 thousand concurrent users and so on.

Spike test is also a part of stress testing which is performed when application is loaded with heavy loads repeatedly and increase beyond production operations for short duration.
Stress Testing helps to determine:
    •    Errors in slowness & at peak user loads
    •    Any security loop holes with over loads
    •    How the hardware reacts with over loads
    •    Data corruption issues at over loads