Testing Insights and Analytics

Performance Comparison between Selenium-3 and Selenium-4

Selenium-4 has some significant improvement by shifting from Selenium-3
Emin Bilgic
5 mins read

Summary

Selenium-4 was officially released On October 13,  2021, and brought lots of upgrades with this release. The most prominent change in this release is the deprecation of JSON wire Protocol. It changed Selenium architecture in a significant way. Besides that, there are also lots of standardizations and stabilizations about the Selenium domain. In this article, we are going to compare coding practices between Selenium-3 and Selenium-4, and we will make a performance comparison between them.

Architectural upgrades on Selenium-4

The most prominent change in this release is the deprecation of JSON Wire Protocol. Actually, this can be considered as the main reason for the Selenium-4 upgrade and Selenium-4 does not include JSON Wire Protocol anymore.

Selenium used to use this protocol for communicating with the web browser. But Selenium-4 has direct communication between the client and server using the W3C (World Wide Web Consortium) protocol, offering lots of advantages compared to the Web Driver used in the versions before Selenium-4.

Architecture of Selenium-3
Architecture of Selenium-3

Native support has been removed for the Opera Driver, and the Opera Driver and browser are no longer active. Users can use Chrome as the Opera browser which is based on Chromium.

Architecture of Selenium-4

After W3C Protocol is replaced with JSON Wire Protocol, you do not need to encode and decode the requests. This direct communication brings more stability and consistency.

The Deprecates from Selenium-3

Find Element methods:

The utility methods to find element(s) in the ‘FindBy’ interface have been removed as they have been used internally only. Instead of this, the ‘By’ class is annotated to increase stability.  

Old:

driver.findElementByClassName("className");

New:

driver.findElement(By.className("className"));

Desired Capabilities:

Desired capabilities were used to define test environments primarily in Selenium Grid. These are replaced with Options in Selenium-4, so you are able to define specific options for drivers.

Old:

DesiredCapabilities caps = DesiredCapabilities.firefox();caps.setCapability("platform", "Windows 10");caps.setCapability("version", "92");caps.setCapability("build", myTestBuild);caps.setCapability("name", myTestName);WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), caps);

New:

FirefoxOptions browserOptions = new FirefoxOptions();browserOptions.setPlatformName("Windows 10");browserOptions.setBrowserVersion("92");Map cloudOptions = new HashMap<>();cloudOptions.put("build", myTestBuild);cloudOptions.put("name", myTestName);browserOptions.setCapability("cloud:options", cloudOptions);WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), browserOptions);

Actions:

Actions class is for simulating user gestures like a double click, click and hold, and etc. There are newly added methods in this class and they are replaced with the old ones.

Old:

moveToElement(toElement).click()

New:

click(webElement)

These are the significant changes from Selenium 3 and there are lots of deprecated methods that came with Selenium-4. Some of the classes and methods that include them are:

  • Timeout Parameters
  • Merging Capabilities
  • Wait parameters
  • Browser Interface
  • Firefox Legacy
  • Browser type

What is new in Selenium-4

There comes lots of remarkable innovations that will make testing easier for you with Selenium-4. For instance, getting screenshots of elements, opening a new tab or window used to be painful. With Selenium-4, these operations are standardized with selenium libraries.

Screenshot of an Element

This was a very painful event in older Selenium versions. You had to take a screenshot of the screen and then you had to crop it using desired element’s coordinates. With Selenium-4, it can be accomplished with a few lines of code:

Old

WebDriver driver = new ChromeDriver();driver.get("https://www.google.com/");WebElement logoElement = driver.findElement(By.className("lnXdpd"));File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);BufferedImage fullImg = ImageIO.read(screenshot);Point point = logoElement.getLocation();int eleWidth = logoElement.getSize().getWidth();int eleHeight = logoElement.getSize().getHeight();BufferedImage eleScreenshot = fullImg.getSubimage(point.getX(), point.getY(),eleWidth, eleHeight);ImageIO.write(eleScreenshot, "png", screenshot);File screenshotLocation = new File("outputs/elementShotFromScreenShot.png");FileUtils.copyFile(screenshot, screenshotLocation);driver.quit();

New

WebDriver driver = new ChromeDriver();driver.get("https://www.google.com/");WebElement logoElement = driver.findElement(By.className("lnXdpd"));File myObj = logoElement.getScreenshotAs(OutputType.FILE);File DestFile = new File("outputs/elementShotTest.png");FileUtils.copyFile(myObj, DestFile);driver.quit();

This brings us a significant performance improvement on taking screenshot because there was lots of operations like crop and save in the old version.

In my example; I ran a test method 3 times which uses the old version of taking screenshots of an element. Then I updated this function with using the new technique and ran it 3 times again.

You can notice a significant reduction in the execution times of the tests in the latter 3 runs than the first 3 runs.

Reducing the execution times of taking a photo of an element in migrating from Selenium-3 to Selenium-4

Multiple Tabs

When you wanted to open a new tab or a new window for your tests, you had to send which shortcut key sequences should be pressed on your screen or you had to run some javascript commands with javascript executed.

With Selenium-4, you can simply tell the driver to switch new tab now.

Old:

String keys = Keys.chord(Keys.CONTROL,"t");driver.findElement(By.cssSelector("body")).sendKeys(keys);//orJavascriptExecutor jse = (JavascriptExecutor)driver;jse.executeScript("window.open()");

New:

driver.switchTo().newWindow(WindowType.TAB);

Relative Locators

Locating elements might be painful even if you have a wide acquaintance with UI testing. With relative locators, you don't need to reach each element with the ‘findElement’ methods. You can locate an element from any element you already located.

WebDriver driver = new ChromeDriver();driver.get("https://www.google.com/");WebElement searchBar = driver.findElement(By.name("q"));searchBar.sendKeys("Relative Locators");WebElement searchButton = driver.findElement(with(By.tagName("input")).below(searchBar));searchButton.click();driver.quit();

This code block finds the search bar in google and sends ‘Relative Locator’ keys to it. Then it checks if there are any elements with an ‘input’ tag below in the located search bar.

Connection Conditions

You can set some connection features for your tests with Selenium-4 like latency, going offline, and setting download or upload throughput.

‘new ChromiumNetworkConditions().setOffline(true)’

Full page screenshot on firefox

Taking a full-page screenshot in Selenium-4 doesn't mean the same thing as taking a screenshot in an older version of Selenium. In older versions, you can only take a screenshot of the current view, not the whole page. With Selenium-4 you can take a screenshot of the infinite scrolled page.

WebDriver augmentedDriver = new Augmenter().augment(driver);File file = ((HasFullPageScreenshot) augmentedDriver).getFullPageScreenshotAs(OutputType.FILE);

Performance comparison

Since W3C was replaced with JSON Wire Protocol, our tests can directly communicate with the web browser. It may sound like Selenium is more efficient and fast now, but it actually does not make much difference in the speed of the tests. As the Selenium web driver and web browsers use the same protocol, tests will execute only more consistently. So flakiness of Selenium tests will be reduced substantially.

We already talked about stability and consistency improvements in Selenium-4. Now we are going to compare if there is any performance improvement in this release.

To do this, I have created a repository that contains 2 projects which are use Selenium-4 and Selenium-3 jars from maven and they contain the same test method with annotation @RepeatedTest(15).

@RepeatedTest(15)public void testAQuery() throws InterruptedException {WebElement searchBar = driver.findElement(By.name("q"));searchBar.sendKeys("Selenium");WebElement searchButton = driver.findElement(By.name("btnK"));Thread.sleep(500);searchButton.click();List results = driver.findElements(By.tagName("h3")) ;results.get(0).click();}

I created 2 different GitHub workflows for running them separately and scheduled them to run 6 times a day.

Now it is time to monitor these tests.

To be able to monitor my workflows, I connected my repository to Foresight while my workflows generate data.

After nearly a week I got this graph from Foresight which means there is no significant performance difference between Selenium-3 and Selenium-4.

Workflow run times of Selenium-3 and Selenium-4

In my performance comparison test, I've only used 15 tests that process a simple google search on a single driver. This may not be a very healthy comparison but we can tell that our tests are more consistent and stable with Selenium-4.

If you want to monitor your GitHub workflows altogether in a single dashboard, go ahead and signup for  Foresight.