Selenium Automation?

Selenium Automation? Everything you need to know to evaluate if Selenium is the test automation solution for you

Table of contents 3 Introduction 5 What is Selenium? 5 6 A short history of Selenium The Selenium tool suite 8 The benefits of Selenium 9 The limitations of Selenium 12 Searching for an alternative: what to consider 14 Leapwork: a code-free Selenium 15 Selenium vs. Leapwork: a comparison Selenium Automation? 2

Introduction In the early 2000s, a test engineer named Jason Huggins came up with an idea. It was an idea that would make his job, as well as many other testers’, a whole lot easier. He created a software tool for web browsers and applications that enabled him to automate his testing, making it possible to find out if the code he wrote worked across all web browsers. Fast forward to almost 20 years later, and a lot has happened. A few more bright-minded test engineers pitched in with their additions to the tool, resulting in countless testers around the world taking it up and implementing it as a core part of their quality assurance processes. Today, Selenium is a popular open-source software testing tool that supports most browsers and operating systems. It’s perhaps most popularly known for allowing testers to automate in any preferred programming language. The problem is this: Not all testers know how to program. Selenium Automation? 3

The Selenium paradox (why most people get stuck with Selenium) In many ways, Selenium test automation is a bit of a paradox: It’s supposed to relieve testers from manually executing test cases, and with that, save them time and keep errors from occuring. But in reality, automating has become almost synonymous with programming. This means testers often end up spending unnecessary time on setting up and maintaining automation flows, when they could be spending it on increasing test coverage and improving the quality of their tests. For this reason, testers can end up spending a significant amount of time on setting up and maintaining automation flows – more than they save once the automation is set up. So, the question is: Is Selenium worth the time investment? To help you answer this question, we have put together this whitepaper. We’ll give you an overview of Selenium that will give you a basic understanding of the Selenium tool suite and the pros and cons of the tool as a whole. This will help you evaluate if it meets your requirements and resource availabilities as a tester or testing team, or if you should be looking at alternatives. Selenium Automation? 4

What is Selenium? A short history of Selenium The first version of Selenium was created by Jason Huggins in 2004. He was tired of spending time and energy on testing web applications and came up with a Javascript library that allowed him to automatically run tests against multiple browsers. Selenium thus became the first tool that enabled its users to control a browser using any programming language. But although Selenium allowed its users to automate many things, it wasn’t without drawbacks. Firstly, because it was based on Javascript, certain things were impossible to do. On top of that, web apps became more complex with time, resulting in additional limitations to the tool. Selenium history timeline Selenium becomes open-sourced First version of Selenium (Selenium Core) 2004 Selenium WebDriver Selenium IDE 2005 2006 WebDriver merges with Selenium (Selenium 2) Selenium Grid 2007 2008 2009 2010 2011 Selenium RC Selenium Automation? 5

A couple of years later, an engineer at Google named Simon Stewart got fed up with Selenium’s limitations. He wanted a tool that would speak directly to the browser using its native language and operating system. Hence WebDriver was born. It took a few years for Selenium to merge with WebDriver, but when they did join forces, it meant taking the best of both worlds and bringing a massive community of the brightest minds in test automation under one roof. A lot more has happened since then, and in addition to Selenium WebDriver, the Selenium project has evolved into a tool suite that also includes Selenium IDE and Selenium Grid. Selenium Remote Control (RC) was also a part of the toolbox, but has since been depreciated, mainly because it was slow. Next, let’s have a closer look at each of the tools in the Selenium tool suite. The Selenium tool suite Selenium consists of three tools that each enable skilled programmers to automate different things. Below is an overview of the tools and their key differences. Selenium IDE Selenium IDE is a ‘record and playback tool’ that can help you create and edit test cases and test suites. IDE (Integrated Development Environment) is a Chrome and Firefox plugin that is used to create and execute test cases. It lets you create test cases by recording your interactions with the browser. Your interactions are transformed into steps in Selenium’s own language. These steps can then be exported in different programming languages, including Java, Python and Ruby. The end result is the entire test script in your language of choice. Selenium Automation? 6

The downside of Selenium IDE is that it is mainly meant for prototyping and syntaxing, not full-fledged testing. Restrictions include not being able to script with regular programming language, not being able to test dynamic web applications, and not being able to import data to support data-driven testing. Selenium WebDriver Selenium WebDriver is a programming interface that can be used to create and execute test cases. WebDriver allows you to test across all the major programming languages, browsers and operating systems. The test cases are created using element locators. You locate your elements with one of the eight Selenium element locator techniques and the WebDriver methods will then let you perform actions on those elements. The script you create interacts directly with the browser (which is the reason it’s much faster than the depreciated Selenium RC). There are different drivers for different browsers, which ‘interpret’ the script you write for it. The downsides of Selenium WebDriver include that it doesn’t generate test reports on its own and that there is no centralized maintenance of objects or elements. Selenium Grid Last is Selenium Grid. Selenium Grid allows you to run multiple tests at the same time on multiple machines, also known as parallel testing. It’s an environment with multiple systems, but one ‘master’ system (the hub) which controls the child systems (the nodes). The child systems can for example be Child 1: Windows 10 running Chrome, Child 2: MAC running Safari, Child 3: Linux running Chrome, and so on. From your master system you can check if your application runs as you want it to on all the different child systems. The instructions are sent through the hub in your chosen programming language. Selenium Automation? 7

The benefits of Selenium Combined, the Selenium tool suite offers a number of possibilities for test automation. Selenium has become one of the most popular testing tools, mainly because it’s an open-source framework that’s free and accessible to all, and because it enables skilled testers to automate many testing processes, such as functional tests and regression tests. Selenium can be used across most browsers, operating systems and programming languages, meaning it integrates well with many business systems. Furthermore, due to it being open-source, the tool can be modified to the needs of the user, and there is a large community of users to learn from and share best practices with. In sum, the benefits of Selenium are: It’s free, as opposed to most other automation tools that include licensing costs It’s open-source, making it possible to extend and modify source code It supports multiple programming languages It supports most operating systems It supports all major browsers It has community support, due to the large network of users It has integration options, allowing e.g. parallel testing and reporting Selenium Automation? 8

The limitations of Selenium Although Selenium has many advantages, it also lacks a number of functionalities to make it the ultimate test automation tool. First of all, Selenium only allows you to test web applications – not mobile or desktop. Useful functionalities such as reporting and data-driven testing are also only possible with the integration of additional tools. In other words, Selenium is in no way an ‘all-in-one’ platform. The biggest disadvantage to testers, however, is probably that it requires programming skills to write tests. Although it may be considered an advantage to programmers that Selenium allows you to use any programming language you like, to most testers, it won’t be an advantage, because testers don’t necessarily know how to write code. This leaves the tester with two options: Rely on a developer to set up and maintain the tests or learn how to program. In addition to writing scripts, you’ll need to maintain them. Again, if you’re a highly skilled programmer with lots of time on your hands, you might be able to find and fix potential flaws in your script. But if programming is new to you, you might find that maintaining such a script can be a tiresome affair. Selenium Automation? 9

An example of the error message you may find in your Selenium script after it has run for a couple of weeks If you aren’t seeing the desired ROI on your test automation efforts, you’re not alone According to Capgemini’s World Quality Report 2021-2022, only 37% are getting the desired ROI on their test automation efforts. Why? Perhaps because more than half of QA teams are struggling to find the right skilled and experienced test automation resources. Only 68% have the required test automation tools, despite a vast number of tools being available for more than 15 years now. What’s more, teams are under increasing pressure to optimize their QA efforts but more than a third don’t have enough time to build and maintain tests. These statistics are symptomatic of a mismatch between the tools offered and the resources available to QA teams. 37% Get ROI from their automation efforts 52% Don’t have the right skilled and experienced test automation resources 32% Don’t have the required automation tools 37% Don’t have enough time to build/maintain tests Source: Capgemini’s World Quality Report 2021-2022 Selenium Automation? 10

Aside from setting up and maintaining the script, you’ll also need to make sure that your code ‘speaks’ with the rest of the release pipeline. This means that you as a tester will need to know the code versioning used in your Git repository. Although the Selenium community may be able to help with various coding tasks, testing teams cannot always rely on this support for larger enterprise projects due to the necessity for privacy and quick resolution time. Additionally, maintaining a Selenium Grid is a significant and time-consuming task. It involves continuous monitoring as well as upgrading and patching. Not only for specific browser versions, but also the OS they are running on. Overall, creating a fast and efficient test automation ecosystem based on Selenium is time-consuming to set up and maintain and requires specialized tester resources. In sum, the limitations of Selenium are: It’s difficult to set up and use, potentially leading to high startup costs It’s difficult to maintain Slow test development, as all test cases need to be scripted It only supports web browsers, not mobile and desktop Integration with the rest of the pipeline is difficult No dedicated support, only user communities Requires 3rd party solutions to cover all testing needs With the pros and cons of Selenium described above, you should now have a clearer picture of how Selenium can and cannot contribute to your QA testing needs. Selenium Automation? 11