Validate Large Number of Response Data in Postman

     We know that validation of response data from an API is very important at the prior stage of functional and also during the QA execution stage. You may need to validate the response data from API against the AUT or you may need to validate the response data from API against the database result set.

     I had a situation or scenario to validate millions of records in API response against the database table result set. It’s very difficult to achieve millions of record validation, even if we do via manual testing it will take time or getting bore after 50 or 60 record validation. We can achieve this validation by following below procedures:

Prerequisites: Make sure that the Dataset or Resultset available as CSV or JSON file. 

  1. Create Postman request.
  2. Add test case in Postman to iterate and validate the data from CSV or JSON against the response of the API. 
  3. Run the collection using Test Runner, there map the CSV or JSON file.

     I did achieve this via postman scripting and later at the time of execution using collection runner validated the response against the dumps exported from the database. I used below sample script in Postman tests,

pm.test(“Verify ID Present”, function () {
pm.expect(pm.response.text()).to.include(pm.iterationData.get(“column_name_in_the_CSV”));
console.log(“ID “+data[“column_name_in_the_csv”]+” is present”)
});

In the above test, validating the ID values from response data against the ID values that exported from database result set. Like this, we can create multiple tests to validate the response data. pm.iterationData.get(“column_name_in_the_CSV”) this method will iterate through all the data from the respective column of the CSV file and check against the API response.

     Once we create the test scripts in Postman, we can open Run Collection against your collection to get the Runner view (we discussed the execution process in the previous article). In the Runner view, we can upload the CSV file that exported from the database and starts execution. Finally, we will get the results including Passed and Failed cases. If we export the collection as JSON from Postman, we can run the collection with CSV file and generate an HTML report using newman library via command-line. The code as follows:

newman run “collection_JSON_file_path” -d “CSV_file_path” -r htmlextra

     The above command helps to run the same collection, validate the CSV data, and finally, it will generate an HTML report. You can try this way of validation instead of stretching to achieve this kind of testing situation.

make it perfect!

API Test Reporting

     We all know that Postman is a great tool when trying to dissect RESTful APIs made by others or test ones you have made yourself. It offers a sleek user interface with which to make HTML requests, without the hassle of writing a bunch of code just to test an API’s functionality. This tool has many HTTP requests methods like GET, POST, PUT, PATCH, DELETE etc., and also it support to write the test cases to validate the response code, response body data, headers etc.,

     In this article, I would like to share the concept of different requests and different assertions that I have created using Postman. Also discussing the important part of test execution and reporting. I have created API requests with GET, POST, PUT, PATCH and DELETE methods inside a collection in Postman. In each request, added the validations to check the the response code and response body. Following are some sample test validations that I have created:

pm.test(“Verify status code is 200”, function () {
pm.response.to.have.status(200);
});

pm.test(“Verify user updated”, function () {
pm.expect(pm.response.text()).to.include(“morpheus”);
pm.expect(pm.response.text()).to.include(“zion resident”);
});

Execute Collections from Postman

     Once completed the HTTP requests and scripting under particular collection, next you can execute your collection. Here, I have created TestPOC collection and added all the requests under this collection,

     You can click on the |->icon right to your collection name and select Run to get the Collection Runner window or you have to click Runner button at top below to the menu in Postman window to get the Collection Runner window.

     In the Collection Runner window, you can see the selected collection and also the selected requests in the run order. In this window, you can set the iterations, delay to start the execution of each requests criteria. Once all set you can click on Run <your_collection_name> here Run TestPOC button to start the execution and after few seconds you will get the results in Run Results section as below,

     In the Run Results, you can see the total pass, fail, details of requests, assertions. You can also export this results in JSON format. In this case, I intentionally failed one case to see it in the results. This method of executing collections from Postman is pretty straightforward. But I have a thought of effortlessly run and test a Postman collection directly from the command-line, also generate some good HTML report with test execution details. We will see how can achieve the execution of Postman collections via command-line and also generate the HTML reports.

Execute Collections from Command-line

     Command-line execution is possible with help of newman package. Newman is a command-line collection runner for Postman. It allows effortless execution and test a Postman collection directly from the command-line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems. Before using this package, you need to install it using npm, use the below command in your command prompt to install newman,

npm install -g newman

     Once you are ready with newman installed, next you can export your collection from Postman and it will be in the JSON format (I got the file in the name as TestPOC.postman_collection.json). Save the JSON file in a location you can access with your terminal and navigate to the path. Once you are in the directory execute below command replacing the collection_name with the name you used to save the collection,

newman run collection_name.json

You will get the results like below,

     In the command-line output, you can see the API requests details, assertion status, table view with executed and failed. You can also be iterated for the respective no# of times using below command,

newman run collection_name.json -n <count>

     Next we will see how the newman can generate HTML reports. Newman provides a different execution report of the collections in formats (like HTML, JSON). But, the accurate way to present execution status to the customer is HTML instead of JSON. To do this need to add HTML reporter for Newman using the following command,

npm install -g newman-reporter-html

     Once the newman-reporter-html installed, you can run and generate HTML report using following command,

newman run postman_collection.json -r html

     Once the execution of above command complete, the HTML report will generate in the newman directory where your collection JSON exists. The file name like newman-run-report-<date_time>. In this report, you can see the details of iterations, requests, total scripts, failed, execution duration, failures with details. Below is an excerpt of the report,

     To have the report more user-friendly, we have another a powerful tool called HTML Extra reporter that generates beautiful reports. Also makes it possible to have an overview of all test runs, and also we have description feature available for collections, folders and requests. To do this we need to add HTML Extra reporter for Newman using the following command,

npm install -g newman-reporter-htmlextra

     Once the newman-reporter-htmlextra installed, you can run and generate user-friendly HTML report using following command,

newman run postman_collection.json -r htmlextra

     Once the execution of above command complete, the HTML report will generate in the newman directory where your collection JSON exists. The file name like <your_collection_name-date_time>. Below is an excerpt of the report,

     There is also global information about the request, response, test pass, test fail information,

     I hope you really enjoyed to read this article and definitely got some idea on Postman collection execution from Postman tool and from the command-line with help of newman. Also, you got some understanding of the HTML report generation for the API collections. Try to implement this concepts in your Postman API testing activities for a better reporting.

make it perfect!

Testing REST endpoints using REST Assured

RESTfil-API

     REST Assured is a Java library that provides a domain-specific language for writing powerful, maintainable tests for RESTful APIs. REST Assured supports the familiar Given/When/Then syntax from behavior-driven development (BDD), resulting in a test that is easy to read and takes care of everything with just a single line of code. Here I would like to explain commonly using GET, POST, DELETE actions with examples. Along with this I would like to explain how verification happening with each test.

Prerequisites

  1. Create Maven project.
  2. Add dependencies like rest-assured, hamcrest-all and testng.
  3. Import following packages:
    1. import staticrestassured.RestAssured.*;
    2. import static org.hamcrest.Matchers.*;

Important Actions or Operations:

  1. makeSureThatApiIsUp() – To make sure that API is up and running

@Test(description = “To make sure that API is up”)
public void makeSureThatApiIsUp() {
given().when().get(“http://restapi.demoqa.com/utilities/weather/city/Hyderabad&#8221;).then().assertThat().statusCode(200);
}

  1. verifyDataInResponseBody() – To verify the given data belongs to the response body of the API

@Test(description = “To verify that given data belongs to the response body”)
public void verifyDataInResponseBody() {
given().when().get(“http://restapi.demoqa.com/utilities/weather/city/Cochin&#8221;).then().body(containsString(“Hyderbad”));
}

  1. verifyMultipleDataInResponseBody() – To verify that multiple values belongs to the response body of the API

@Test(description = “To verify that multiple values belongs to the response body”)
public void verifyMultipleDataInResponseBody() {
given().when().get(“http://restapi.demoqa.com/utilities/weather/city/Cochin&#8221;).then().body(“City”, equalTo(“Cochin”)).body(“WindSpeed”, equalTo(“7.2 Km per hour”));
}

  1. verifySingleSpecificDataInResponseBody() – To verify the specific value belongs to the response body of the API

@Test(description = “To verify that specific value belongs to the response body”)
public void verifySingleSpecificDataInResponseBody() {
given().when().get(“http://restapi.demoqa.com/utilities/weather/city/Cochin&#8221;).then().body(“City”,equalTo(“Cochin”));
}

  1. verifyContentType() – To verify the content type

@Test(description = “To verify the content type”)
public void verifyContentType() {        given().when().get(“http://restapi.demoqa.com/utilities/weather/city/Cochin&#8221;).then().assertThat().statusCode(200).and().contentType(“application/json”);
}

  1. verifyContentTypeAndHeaderValue() – To verify the content type and the header values

@Test(description = “To verify the content type and header values”)
public void verifyContentTypeAndHeaderValue() {
given().when().get(“http://restapi.demoqa.com/utilities/weather/city/Cochin&#8221;).then().assertThat().statusCode(200).and().contentType(“application/json”).and().header(“Content-Length”, equalTo(“158”));
}

  1. verifyHeaderValue() – To verify the header values

@Test(description = “To verify the header value”)
public void verifyHeaderValue() {
given().when().get(“http://restapi.demoqa.com/utilities/weather/city/Cochin&#8221;).then().assertThat().statusCode(200).and().header(“Content-Length”, equalTo(“159”));
}

  1. verifyAPIWithQueryParameter() – To verify the API request with query parameter

@Test(description = “To verify API with query parameter”)
public void verifyAPIWithQueryParameter() {
String cityname = “Cochin”;
given().param(“text”, cityname).when().get(“http://md5.jsontest.com/?text=&#8221; + cityname).then().body(containsString(“Cochin”));
}

  1. verifyAPIWithBasicAuthentication_ShouldBeGivenAccess() – To verify the basic authentication

@Test(description = “To verify basic authentication”)
public void verifyAPIWithBasicAuthentication_ShouldBeGivenAccess() {
given().auth().preemptive().basic(“username”, “password”).when().get(“https://xxxxxxxxxxxxxxxxx.com/api/auth/login/&#8221;).then().assertThat().statusCode(200);
}

  1. verifyAPIWithOAuth2Authentication_ShouldBeGivenAccess() – To verify the OAuth2 secured authentication

@Test(description = “To verify OAuth2-secured authentication”)
public void verifyAPIWithOAuth2Authentication_ShouldBeGivenAccess(){
given().auth().oauth2(“460bfecd0bbba1a0d9dfdbbbecdf1008”).when().get(“https://openweathermap.org/&#8221;).then().assertThat().statusCode(200);
}

  1. extractDataFromResponseBody() – To extract value from the response body.

@Test(description = “To extract value from Response body”)
public void extractDataFromResponseBody() {
String circuitId = given().when().get(“http://restapi.demoqa.com/utilities/weather/city/Cochin&#8221;).then().extract().path(“Temperature”);
System.out.println(circuitId);
}

  1. toDeleteData() – To delete data

@Test(description = “To delete data”)
public void toDeleteData() {
given().pathParam(“empNumber”,69).when().delete(“/emp/details/{empNumber}”).then().statusCode(200);
}

  1. toPostDate() – To post data

@Test(description = “To post data”)
public void toPostDate() {
Map<String, String> empdata = new HashMap<>();
empdata.put(“empNumber”, “69”);
empdata.put(“empName”, “sanoj”);
empdata.put(“empDesignation”, “Test Lead”);       given().contentType(“application/json”).body(empdata).when().post(“/emp/details”).then().statusCode(200);
}

  1. verifyMultipleAPIsUpAndRunning() – To verify multiple APIs up and running in a single shot execution using excel test data file

@Test(description = “To verify multiple APIs running”)
public void verifyMultipleAPIsUpAndRunning() {
ReadExcelData data = new ReadExcelData();
SoftAssert assertnow = new SoftAssert();
for (int i = 0; i < data.getAPIFromExcel(“.//APITestData.xlsx”, “APIS”, 2).size(); i++) {
System.out.println(data.getAPIFromExcel(“.//APITestData.xlsx”, “APIS”, 2).get(i).toString());
System.out.println(given().when().get(data.getAPIFromExcel(“.//APITestData.xlsx”, “APIS”, 2).get(i)).then().extract().statusCode());
assertnow.assertEquals(given().when().get(data.getAPIFromExcel(“.//APITestData.xlsx”, “APIS”, 2).get(i)).then().extract().statusCode(), 200);
}
assertnow.assertAll();
}

NOTE: getAPIFromExcel() is a user defined function to read the APIs from the excel file by passing the excel file path, sheet name and the column index.

You can do all the relevant actions from your end for your APIs. Create test suites and execute it.

make it perfect !