Writing Resilient Test Code

     Ideally, you should be able to write your tests once and run them across all supported browsers. While this is a rosy proposition, there is some work to make this a reliable success. And sometimes there may be a hack or two involved. But the length you must go really depends on the browsers you care about and the functionality you’re dealing with in your application.

     By using high quality locators we’re already in good shape, but there are still some issues to deal with.  This is especially true when working with dynamic, JavaScript heavy pages (which is more the rule than the exception in a majority of web applications you’ll deal with).

     But there is a simple approach that makes up the bedrock of reliable and resilient Selenium tests — and that’s how you wait and interact with elements. Gone are the days of waiting for the page to finish loading, or hard-coding sleeps, or doing a blanket wait time (a.k.a. an implicit wait).  Now are the wonder years of waiting for an expected outcome to occur for a specified amount of time. If the outcome occurs before the amount of time specified, then the test will proceed. Otherwise, it will wait the full amount of time specified.

We accomplish this through the use of explicit waits.

An Example

Let’s step through an example that demonstrates this against a dynamic page on the-internet. The functionality is pretty simple — there is a button. When you click it a loading bar appears for 5 seconds, after which it disappears and is replaced with the text Hello World!

Part 1: Create A New Page Object And Update The Base Page Object

Here’s the markup from the page.

Capture

        At a glance it’s simple enough to tell that there are unique id attributes that we can use to find and click on the start button and verify the finish text.

        When writing automation for new functionality like this, you may find it easier to write the test first (to get it working how you’d like) and then create a page object for it (pulling out the behavior and locators from your test). There’s no right or wrong answer here. Do what feels intuitive to you. But for this example, we’ll create the page object first, and then write the test.

Let’s create a new page object file called DynamicLoading.java in thepageobjects package.

Capture

     In this file we’ll establish inheritance to the base page object and specify the locators and behavior we’ll want to use.

Capture1

     Since there are two examples to choose from we created the methodloadExample which accepts a String of the example number we want to visit as an argument. And similar to our login page object, we have a display check for the finish text (e.g., finishTextPresent()). This check is slightly different though. Aside from the different name, it has a second argument (an integer value of 10). This second argument is how we’ll specify how long we’d like Selenium to wait for an element to be displayed before giving up.

     Let’s update our base page object to enable explicit waits, adding this newwaitForIsDisplayed method to use them.

Capture

     Selenium comes with a wait function which we wrap in a private method (e.g.,private void waitFor) for reuse in this class.

     This method accepts two arguments — the condition we want to wait for (e.g.,ExpectedCondition<WebElement>) and the amount of time we want Selenium to keep checking for (e.g., Integer timeout). If a null value is passed as an argument for the timeout, then the wait time is set to 5seconds. This is handled by a ternary operator (e.g., timeout = timeout != null ? timeout : 5;).

     The waitForIsDisplayed method has two parameters — one for a locator, another for the timeout. Inside the method we call waitFor and send it anExpectedCondition to check for the visibility of an element (e.g.,.visibilityOfElementLocated(locator)). This is similar to our previous display check, but it uses a different Selenium API function that will work with the explicit waits function. You can see a full list of      Selenium’sExpectedConditions here. Unfortunately, this function doesn’t return a Boolean, so we provide one . If the condition is not met by Selenium in the amount of time provided, it will throw a timeout exception. When that happens, we catch it and return false. Otherwise, we return true.

     It’s worth noting that the second parameter is optional when callingwaitForIsDisplayed (e.g., Integer… timeout). If a timeout value is specified, it will get passed to the waitFor method. If nothing is specified,nullwill get passed instead. This gives us the freedom to call this method in our page objects without specifying a timeout (e.g.,waitForIsDisplayed(locator)) or with a timeout (e.g.,waitForIsDisplayed(locator, 20)`).

More on Explicit Waits

     In our page object when we’re using waitForIsDisplayed(finishText, 10) we are telling Selenium to check if the finish text is visible on the page repeatedly. It will keep checking until either the element is displayed or reaches ten seconds — whichever comes first.

     It’s important to set a reasonably sized default timeout for the explicit wait method. But you want to be careful not to make it too high. Otherwise you can run into similar timing issues you get from an implicit wait. But set it too low and your tests will be brittle, forcing you to run down trivial and transient issues.

     The major benefit of explicit waits is that if the behavior on the page takes longer than we expect (e.g., due to slow load times, or a feature change), we can simply adjust this one wait time to fix the test — rather than increase a blanket wait time (which impacts every test). And since the wait is dynamic (e.g., constantly polling), it won’t take the full amount of time to complete (like a static sleep would).

     If you’re thinking about mixing explicit waits with an implicit wait — don’t. If you use both together, you’re going to run into issues later on due to inconsistent implementations of implicit wait across local and remote browser drivers. Long story short, you’ll see inconsistent and odd test behavior.

Part 2: Write A Test To Use The New Page Object

     Now that we have our new page object and an updated base page, it’s time to write our test to use it.

     Let’s create a new file called TestDynamicLoading.java in the testspackage.

Capture

     The contents of this test file are similar to TestLogin with regards to the imported classes and the setUp/tearDown methods.

Capture1

     In our test (e.g., public void hiddenElementLoads()) we are visiting the first dynamic loading example and clicking the start button (which is accomplished in dynamicLoading.loadExample(“1”)). We’re then asserting that the finish text gets rendered.

     When we save this and run it (mvn clean test -Dtest=TestDynamicLoading from the command-line) it will run, wait for the loading bar to complete, and pass.

Part 3: Update Page Object And Add A New Test

     Let’s step through one example to see if our explicit wait approach holds up.

     The second dynamic loading example is laid out similarly to the last one. The only difference is that it renders the final text after the progress bar completes (whereas the previous example had the text on the page but it was hidden).

Here’s the markup for it.

Capture

     In order to find the selector for the finish text element we need to inspect the page after the loading bar sequence finishes. Here’s what it looks like.

Capture

     Let’s add a second test to TestDynamicLoading.java calledelementAppears() that will load this second example and perform the same check as we did for the previous test.

Capture

     When we run both tests (mvn clean test -Dtests=TestDynamicLoading from the command-line) we will see that the same approach will work for both cases.

Browser Timing

     Using explicit waits gets you pretty far. But there are a few things you’ll want to think about when it comes to writing your tests to work against various browsers.

     It’s simple enough to write your tests locally against Firefox and assume you’re all set. Once you start to run things against other browsers, you may be in for a rude awakening. The first thing you’re likely to run into is the speed of execution. A lot of your tests will start to fail when you point them at either Chrome or Internet Explorer, and likely for different reasons.

     In my experience, Chrome execution is very fast, so you will see some odd timeout failures. This is an indicator that you need to add explicit waits to parts of your page objects that don’t already have them. And the inverse is true when running things against older version of Internet Explorer (e.g., IE 8). This is an indicator that your explicit wait times are not long enough since the browser is taking longer to respond — so your tests timeout.

      The best approach to solve this is an iterative one. Run your tests and find the failures. Take each failed test, adjust your code as needed, and run it against the browsers you care about. Repeat until you make a pass all the way through each of the failed tests. Then run a batch of all your tests to see where they fall down. Repeat until everything’s green.

     Once you’re on the other side of these issues, the amount of effort you need to put into it should diminish dramatically.

Conclusion

      By explicitly waiting to complete an action, our tests are in a much more resilient position because Selenium will keep trying for a reasonable amount of time rather than trying just once. And each action can be tuned to meet the needs of each circumstance. Couple that with the dynamic nature of explicit waits, and you have something that will work in a multitude of circumstances — helping you endure even the toughest of browsers to automate.

     This is one of the most important concepts in testing with Selenium. Use explicits waits often.

Reference

http://elementalselenium.com/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s