How ReportPortal Helps Continuous Integration and Testing Processes

Introduction

Software development is the process of specifying, designing, programming, documenting, regression testing, and bug fixing involved in the creation and maintenance of software components.

During the process of software development, it is imperative to collaborate the code changes made by each contributor into a single software project. After this step, a build is triggered and different types of testing are run on the code, and build output is returned. The practice of automating the above steps is called Continuous Integration (CI). There are many open-source software platforms for Continuous Integration such as Buddy, Jenkins, Buildbot, etc.

A part of the Continuous Integration process is software testing. Software testing is done to identify errors, gaps, or missing requirements with respect to actual requirements. Properly tested software products ensure reliability, security, and high performance.

Optimize digital experiences with the performance testing capabilities of the HeadSpin. Learn more.

There are different types of software systems: Functional, Non-functional, and Maintenance testing. Continuous regression testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.

ReportPortal is a great addition to the Continuous Integration and Testing processes. It uses built-in analytic features to provide increased capabilities to speed up results analysis and reporting. ReportPortal is a great tool with many features full of benefits for your team. Also, a software called Headspin can deliver the perfect digital experiences and enable development to product teams by getting data science insights into UX. It is possible to run thousands of tests parallelly at scale, without worrying about device availability with the reliable global device infrastructure. For more details, please visit https://www.headspin.io/.

The following technologies are used in ReportPortal:

1. NoSQL MongoDB
2. REST Web Service
3. Mobile responsive UI

Uses of ReportPortal

1. Easily Integrates with regression testing frameworks.

ReportPortal software can seamlessly integrate with platforms like Jenkins for continuous integration, and with the majority of unit and functional unit testing frameworks for continuous testing. It can also integrate with Jira and BDD process software.

2. ReportPortal receives analytics in real-time.

With each execution, ReportPortal automatically figures out the root cause of a failure. Engineers will be alerted about this issue to provide further analysis: if it has been resolved already or which test results require actual human analysis. Also, the results of all autotests on your project are accumulated in one place.

The test cases are shown together with all related data like logs, screenshots, binary data in one place. You can see all info about a test case in one place and find a failure cause in previous runs easily.

3. ReportPortal also gives you the ability to collaboratively analyze the mobile test automation results.

Particular test cases can be classified as a product bug, an automation issue, or a system issue. If not, then the test case can be submitted as an issue ticket directly from the execution result. It can also leverage historical data of test execution to provide enhanced capabilities with auto-results analysis.

With ReportPortal, you can define the failure reasons of a test case and set a Defect type for it: Product bug, Auto Bug, System Issue, or custom type, so you can visualize the structure of the failures. A part of analytics work can be delegated to Artificial Intelligence. Auto-Analyzer based on Machine Learning helps reduce time, cost, and helps analyze the failure reasons.

4. Integration with Bug tracking systems is possible.

Integrate ReportPortal with Jira or Rally through which you can create issues with failures information in BTS, link failures to the existing issues in BTS, and control the number of open issues via the widget.

ReportPortal consists of the following services:

1. Authorization Service: In charge of access tokens distribution.
2. Gateway Service: The main entry point to the application. Port used by the gateway should be opened and accessible from the outside network.
3. API Service: Main application API.
4. UI Service.
5. JIRA Service.
6. Rally Service.
7. TFS Service.

Installation of ReportPortal

Some of the prerequisites for optimal performance of ReportPortal software are Disk I/O, CPU utilization, Configuration testing results and saturation points database separate from other services, PostgreSQL Performance Tuning, and Application connections pool tuning. For a detailed understanding of the prerequisites, please refer here.

There are many ways to install ReportPortal software. It can be deployed through Docker and Kubernetes. For detailed steps, please refer here.

Deploy with Docker ReportPortal

1) Make sure the Docker Engine and Compose are installed.

2) Download the latest ReportPortal Docker compose file from here. You can make it run by the following command:

curl

https://raw.githubusercontent.com/reportportal/reportportal/master/docker-compose.yml -o docker-compose.yml

3) Make the ElasticSearch configuration prerequisites for the analyzer service.

1. Set {vm.max_map_count} kernel setting before ReportPortal deploying with the following commands.
2. Give right permissions to ElasticSearch data folder using the following commands:

  
mkdir -p data/elasticsearch

chmod 777 data/elasticsearch

chgrp 1000 data/elasticsearch

4) PostgreSQL Performance Tuning:

You can additionally optimize your PostgreSQL performance by adding some parameters to the “command” option in the Docker compose file. For more details, please refer here.

5) Start the application using the following command:

     docker-compose -p reportportal up -d –force-recreate

6) Open your web browser with an IP address of the deployed environment at port 8080.

Deploy ReportPortal with Docker on Windows

For using Reportportal with Docker on Windows, change the ‘volumes’ value for Postgres container from “For unix host” to the “For windows host”:

     volumes:

  
# For windows host

– postgres:/var/lib/postgresql/data

# For unix host

# – ./data/postgres:/var/lib/postgresql/data

After this, uncomment:

# Docker volume for Windows host

    volumes:

    postgres:

Deploy with Kubernetes ReportPortal

Helm package manager is used to bootstrap a ReportPortal deployment on a Kubernetes cluster.

For more details, please visit https://github.com/reportportal/kubernetes.

Integration Procedure

ReportPortal can integrate with different test frameworks in many languages like Java, Javascript, Python, .NET, PHP, and others. To see all the supported test frameworks, please refer here.

Integration with Pytest

We will show the integration procedure for ReportPortal with Pytest. Pytest is a python-based test framework. Below are the steps needed to achieve this:

1. Set up ReportPortalDownload and install ReportPortal software or use the demo server. For this experiment, a demo server has been used. Make a note of UUID, endpoint, and project details from the profile link.
2. Create test case functions and add markers.  
3. Set up the Pytest plugin.
   The agent-python-pytest Python plugin allows you to integrate Pytest with ReportPortal.For installing the plugin, you can run:pip install pytest-reportportalAdd the ReportPortal details like UUID, endpoint and project detail and use it in the pytest.ini file. Example:[pytest]
4. rp_uuid = fb586627-32be-47dd-93c1-678873458a5f
5. rp_endpoint = http://192.168.1.10:8080
6. rp_project = temp_project
7. rp_launch = ABC
8. Note: The UUID and other details mentioned above need to be updated as per your ReportPortal details.
9. Creating a conftest.However, instead of modifying the pytest.ini file, we can create a conftest file. If not, create a conftest.py file where we can add the ini details conditionally required by ReportPortal.
10. Running the test
    You can run the test in ReportalPortal using any of the markers as specified belowpy.test -m gui_test –reportportal
11. Now you can run tests and analyze the report for specific tests in ReportPortal.

‍

‍

For more details, please visit here.

Integration with JIRA

ReportPortal can be integrated with JIRA for projects that collect defects in a separate tracking tool. It provides an exchange of information between ReportPortal and the JIRA, such as posting and linking issues, getting status updates, and many more.

For more detailed steps on integration with JIRA, please refer here.

Integration with Jenkins

Jenkins plugin is not required for ReportPortal for having results from tests executed by Jenkins job. ReportPortal adapter connects to the test engine (like JUnit, TestNG, NUnit, etc.) and fires events, and the data is sent back to the ReportPortal server. However, there is a plugin for representing statistics and a test results visualization of already launched builds on the Jenkins job page, called plugin-jenkins

‍

‍

For more details, please refer here.

Conclusion

ReportPortal is a service that uses AI and built-in analytic features to provide increased capabilities to speed up results analysis. It can easily integrate with 15 mainstream software platforms like Jenkins, Jira, BDD process, and a majority of functional testing softwares. ReportPortal’s collaborative test automation environment enables your team to make smarter decisions.