Trace Network Logs during Automation

   We know that network logs provide detailed information about network requests, responses, and any errors or anomalies encountered during communication between software components or systems. We can use these logs to identify and troubleshoot network-related issues such as timeouts, connectivity problems, or data transmission errors. Incorporating network log analysis into automated testing frameworks enables testers to validate network-related functionality and performance metrics automatically. Automated tests can simulate various network conditions and monitor how the software behaves, helping to uncover issues early in the development cycle. In this article, I would like to share the implementation logic of network logs while using Selenium test automation with various programming languages.

Network Logs Implementation in Selenium Java

   Following is the logic we can use to trace the networks in Selenium Java. If we are using the TestNG and the listener class, we can call the below method from onTestStart(ITestResult result) method:

private void trackNetworkLogs(ITestResult result, String testName) {
		try {
			WebDriver driver = null;
			Object currentClass = result.getInstance();
			try {
				driver = ((AutomationBase) currentClass).getDriver();
			} catch (Exception e) {
			}

			DevTools devTools = null;
			if (driver instanceof EdgeDriver) {
				devTools = ((EdgeDriver) driver).getDevTools();
			} else if (driver instanceof ChromeDriver) {
				devTools = ((ChromeDriver) driver).getDevTools();
			} else {
				System.out.println("Network logs supported for Edge and Chrome browsers");
			}
			if (devTools != null) {
				devTools.createSession();
				devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));

				if (!new File(System.getProperty("user.dir") + "//Logs//Network_logs").exists()) {
					(new File(System.getProperty("user.dir") + "//Logs")).mkdir();
					(new File(System.getProperty("user.dir") + "//Logs//Network_logs")).mkdir();
				}
				fileWriter = new FileWriter(new File(System.getProperty("user.dir") + "\\Logs\\Network_logs\\"
						+ testName + ".txt"));

				devTools.addListener(Network.responseReceived(), responseReceived -> {
					Response response = responseReceived.getResponse();
					String log = "Network Log: " + response.getUrl() + "\n";

					try {
						fileWriter.write(log);
					} catch (IOException e) {
						e.printStackTrace();
					}
					try {
						fileWriter.flush();
					} catch (IOException e) {
						e.printStackTrace();
					}
				});
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

   In the above logic, we can have a AutomationBase class with the driver instantiation logic and getDriver() method will get the current driver session in the TestNG listener class. The above logic will create a text file with test case name inside the Network_logs folder inside Logs folder in the project structure and able to see the network transaction in the log file.

Network Logs Implementation in Selenium C#

   Following is the logic we can use to trace the networks in Selenium C#. If we are using the NUnit framework, we can call the method StartNetworkLogging() in [SetUp] and call the method StopNetworkLogging() in [TearDown] in the AutomationBase class where the driver sessions instantiate. Following are the logic of StartNetworkLogging() and StopNetworkLogging():

private void StartNetworkLogging()
{
    try
    {
        IDevTools devTools = driver as IDevTools;
        session = devTools.GetDevToolsSession();
        devToolsSession = session.GetVersionSpecificDomains<DevToolsSessionDomains>();
        devToolsSession.Network.Enable(new Network.EnableCommandSettings());
        devToolsSession.Network.ResponseReceived += OnResponseReceived;
    }
    catch (Exception)
    {
        throw;
    }
}

private void StopNetworkLogging()
{
    try
    {
        devToolsSession.Network.ResponseReceived -= OnResponseReceived;
    }
    catch (Exception)
    {
        throw;
    }
}

private void OnResponseReceived(object sender, Network.ResponseReceivedEventArgs e)
{
    try
    {
        string url = e.Response.Url;
        string headers = string.Join(Environment.NewLine, e.Response.Headers.Select(h => $"{h.Key}: {h.Value}"));
        var workingDirectory = Environment.CurrentDirectory;
        var projectDirectory = Directory.GetParent(workingDirectory).Parent.Parent.FullName;
        DirectoryInfo di = Directory.CreateDirectory(projectDirectory + "/Logs/Network_logs/");
        string logFilePath = projectDirectory + "/Logs/Network_logs/" + _testName + ".txt";
        using (var writer = new StreamWriter(logFilePath, append: true))
        {
            writer.WriteLine($"URL: {url}");
            writer.WriteLine($"Status Code: {e.Response.Status}");
            writer.WriteLine($"Headers:{Environment.NewLine}{headers}");
            writer.WriteLine("=======================================");
            writer.WriteLine();
        }
    }
    catch (Exception)
    {
        throw;
    }
}

   The above logic will create a text file with test case name inside the Network_logs folder inside Logs folder in the project structure and able to see the network transaction in the log file.

Network Logs Implementation in Selenium Python

   Following is the logic we can use to trace the networks in Selenium Python. If we are using the Pytest framework, we can call the method _track_network_logs(driver_session) in @pytest.fixture(scope=”class”, autouse=True) where the driver sessions instantiate(in conftest.py file). Following are the logic of _track_network_logs(driver_session):

def _track_network_logs(driver_session):
    """
    Method to track and collect the network logs for the automation execution
    @Author : Sanoj Swaminathan
    @Date: 16-04-2024
    :param driver_session:
    :return:
    """
    try:
        # Get the captured network logs
        logs = driver_session.requests

        # Create Logs directory if not exists
        current_directory = os.getcwd()
        new_folder_path = os.path.join(current_directory, '..', 'Logs')
        if not os.path.exists(new_folder_path):
            os.mkdir(new_folder_path)

        # Write the logs to the file
        log_file_path = new_folder_path + '/' + 'network_logs_' + datetime.now().strftime("%d-%m-%Y %H-%M-%S") + '.txt'
        with open(log_file_path, 'w') as file:
            for data in logs:
                file.write(f"URL: {data.url}\nStatus Code: {data.response.status_code}\n"
                           f"Headers: {data.response.headers}\n")
    except Exception as e:
        print('The exception is ', e)

   The above logic will create a text file with timestamp inside the network_logs folder inside Logs folder in the project structure and able to see the network transaction in the log file.

   Tracing the network logs and validating them during automation is one of the important aspect. In this article, we discussed about network log tracing while using Selenium with different programming languages (Java, C# and Python). I hope you really enjoyed reading this article and try to utilize the implementation logic in your live automation projects if you have such a requirement to trace and validate the network logs.

make it perfect!