API Test Reporting

     We all know that Postman is a great tool when trying to dissect RESTful APIs made by others or test ones you have made yourself. It offers a sleek user interface with which to make HTML requests, without the hassle of writing a bunch of code just to test an API’s functionality. This tool has many HTTP requests methods like GET, POST, PUT, PATCH, DELETE etc., and also it support to write the test cases to validate the response code, response body data, headers etc.,

     In this article, I would like to share the concept of different requests and different assertions that I have created using Postman. Also discussing the important part of test execution and reporting. I have created API requests with GET, POST, PUT, PATCH and DELETE methods inside a collection in Postman. In each request, added the validations to check the the response code and response body. Following are some sample test validations that I have created:

pm.test("Verify status code is 200", function () {
pm.response.to.have.status(200);
});

pm.test("Verify user updated", function () {
pm.expect(pm.response.text()).to.include("morpheus");
pm.expect(pm.response.text()).to.include("zion resident");
});

Execute Collections from Postman

     Once completed the HTTP requests and scripting under particular collection, next you can execute your collection. Here, I have created TestPOC collection and added all the requests under this collection,

     You can click on the |->icon right to your collection name and select Run to get the Collection Runner window or you have to click Runner button at top below to the menu in Postman window to get the Collection Runner window.

     In the Collection Runner window, you can see the selected collection and also the selected requests in the run order. In this window, you can set the iterations, delay to start the execution of each requests criteria. Once all set you can click on Run <your_collection_name> here Run TestPOC button to start the execution and after few seconds you will get the results in Run Results section as below,

     In the Run Results, you can see the total pass, fail, details of requests, assertions. You can also export this results in JSON format. In this case, I intentionally failed one case to see it in the results. This method of executing collections from Postman is pretty straightforward. But I have a thought of effortlessly run and test a Postman collection directly from the command-line, also generate some good HTML report with test execution details. We will see how can achieve the execution of Postman collections via command-line and also generate the HTML reports.

Execute Collections from Command-line

     Command-line execution is possible with help of newman package. Newman is a command-line collection runner for Postman. It allows effortless execution and test a Postman collection directly from the command-line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems. Before using this package, you need to install it using npm, use the below command in your command prompt to install newman,

npm install -g newman

     Once you are ready with newman installed, next you can export your collection from Postman and it will be in the JSON format (I got the file in the name as TestPOC.postman_collection.json). Save the JSON file in a location you can access with your terminal and navigate to the path. Once you are in the directory execute below command replacing the collection_name with the name you used to save the collection,

newman run collection_name.json

You will get the results like below,

     In the command-line output, you can see the API requests details, assertion status, and table view with executed and failed. You can also be iterated for the respective no# of times using the below command,

newman run collection_name.json -n <count>

     Next, we will see how the newman can generate HTML reports. Newman provides a different execution report of the collections in formats (like HTML, JSON). But, the accurate way to present execution status to the customer is HTML instead of JSON. To do this need to add an HTML reporter for Newman using the following command,

npm install -g newman-reporter-html

     Once the newman-reporter-html is installed, you can run and generate an HTML report using the following command,

newman run postman_collection.json -r html

     Once the execution of the above command is complete, the HTML report will generate in the newman directory where your collection JSON exists. The file name like newman-run-report-<date_time>. In this report, you can see the details of iterations, requests, total scripts, failed, execution duration, and failures with details. Below is an excerpt of the report,

     To have the report more user-friendly, we have another powerful tool called HTML Extra reporter that generates beautiful reports. Also makes it possible to have an overview of all test runs, and also we have a description feature available for collections, folders, and requests. To do this we need to add HTML Extra reporter for Newman using the following command,

npm install -g newman-reporter-htmlextra

     Once the newman-reporter-htmlextra installed, you can run and generate user-friendly HTML report using following command,

newman run postman_collection.json -r htmlextra

     Once the execution of the above command is complete, the HTML report will generate in the newman directory where your collection JSON exists. The file name like <your_collection_name-date_time>. Below is an excerpt of the report,

     There is also global information about the request, response, test pass, test fail information,

     I hope you really enjoyed to read this article and definitely got some idea on Postman collection execution from Postman tool and from the command-line with help of newman. Also, you got some understanding of the HTML report generation for the API collections. Try to implement this concepts in your Postman API testing activities for better reporting.

make it perfect!