Playwright Api Testing

Software systems are becoming increasingly complex due to the constantly growing number of applications on different platforms. A decisive factor for the success of a software product is its quality. For this reason, more and more companies are carrying out systematic checks and tests, if possible on the various platforms, in order to be able to ensure a specified quality standard. In order to be able to maintain short release cycles despite the higher testing effort, it becomes necessary to automate the tests. This in turn leads to the need to define a test automation strategy. One of the first steps in implementing a test automation strategy is to evaluate suitable test automation tools. Since each project is unique, both the requirements and the choice of tools vary. This blog series is intended to provide guidance in selecting the appropriate solution. Figure 1: Part of implementing a test automation strategy is evaluating appropriate test automation tools.

Introduction

As part of my final thesis, I took on the task of providing the Scrum teams at ZEISS Digital Innovation (ZDI) with a tool to help them find the right test automation tool quickly and flexibly. The challenge here was that the projects have specific scenarios and the requirements may have to be weighted separately. I would like to present the status of the work and the results to you in this and the next blog articles. Software development has long been an area of rapid change. But while in the past these developments took place mainly at the technological level, we are currently observing major changes in the area of processes, organization and tools in software development. However, these new trends come with challenges such as changes in requirements management, shortened timelines, and especially the increased demands on quality. Today, both efficiency and effectiveness within software development are increased through the development of demand-driven solutions and the optimization of quality. In addition, software systems are becoming increasingly complex due to the constantly growing number of applications on different platforms. Since quality is a decisive factor for the success of a software product, more and more companies are carrying out systematic checks and tests, if possible on the various platforms, to ensure a specified quality standard. A SmartBear survey conducted in early 2021 found that many companies, regardless of industry, already perform various types of testing, with web application testing leading the way at 69% and API/web services testing second at 61%. Desktop testing is performed by 42% of respondents. A total of 62% of respondents state that they perform mobile testing, 29% of which is for native applications (apps) (SmartBear, 2021, p. 12). In order to maintain short release cycles despite the higher testing effort, it becomes necessary to automate the tests. As ZDI, we support our customers inside and outside the ZEISS Group in their digitization projects. In addition, we offer individual test services. That is why we have a large number of projects with different technologies and different solutions. As a small selection, keywords such as Java, .Net, JavaFX, WPF, Qt, Cloud, Angular, Embedded, etc. can be mentioned here. In such complex projects, quality assurance is always in the foreground and the projects are dependent on the use of modern test tools, which support the project participants in the manual, but especially in the automated tests. It would be desirable to have a tool that supports this automation effectively and efficiently. However, testers face a number of challenges when selecting a test automation tool.

Challenges

During the research and interviews conducted as part of my thesis, the greatest challenges in test automation were cited as the variety of test tools available and the ongoing fragmentation of the market. Because of this fragmentation, there are a multitude of tools available for the same purpose. The choice becomes even more difficult because the tools differ not only in the technology they can test, but also in their approach to work. When automation is mentioned, it is always associated with scripting. However, in recent years a new approach to GUI testing has been developed, called NoCode/LowCode. This approach basically requires no programming skills and is therefore popular with automation novices. Nevertheless, scripting remains the dominant method, although sometimes both approaches are used to involve as many of the quality assurance team as possible (SmartBear, 2021, p. 33). The type of test automation approach, and thus the use of a test automation tool, depends on the requirements in the project. This means that the requirements must always be reassessed for each new project.

Inventory

During the interviews, the analysis of the current approach and the evaluation of an internal survey, I was able to identify the following approaches for the selection of tools in the projects, which have become established as a “quasi-standard”:

  • The tool is open source and costs nothing.
  • The tool has been worked with before and was considered trustworthy.

One goal of the survey was to find out to what extent the project situation has an influence on the tool selection. Therefore, the next step was to examine the project situations and the process models used in the projects and their influence. It turned out that not the procedure models have a direct influence on the tool selection, but the persons or groups, which use the tool as future users or which release the employment. When examining the influence of these operationally-involved stakeholders, it became apparent that there are other stakeholders that have a direct or indirect influence on the selection of a tool. These are for example:

  • Software architects, who define the technology or tool chains within the project,
  • the management, which defines specifications for the purchase or use of the tools (OpenSource, GreenIT, strategic partnerships, etc.) or
  • the company’s IT, which prevents the use of certain tools through the infrastructure used (FireWall, Cloud, etc.).

In addition, the first approaches of checklists for the selection of test tools were already found during the research. They were mostly based on a few functional criteria and were determined in workshops by the project members. The evaluation showed that there was no uniform method and that the tools selected in this way were very often later replaced by other tools. This became necessary because necessary requirements for the tool were overlooked during the selection process. In the majority of cases, the forgotten requirements were of a non-functional nature, such as usability criteria or performance requirements. Therefore, a next step in checking relevant criteria was to refer to ISO/IEC 25000 Software Engineering (Quality Criteria and Evaluation of Software Products). Not always, but very often, the overlooked requirements were non-functional in nature. Therefore, a next step in the review of relevant criteria was to refer to ISO/IEC 25000 Software Engineering (Quality criteria and evaluation of software products). Figure 2: Criteria for software according to ISO/IEC 25010 The next blog article in this series will show how the criteria catalog is structured and how the final list of criteria is composed.

Literature

SmartBear (2021): State of Software Quality | Testing. This article was technically supported by: Kay Grebenstein Kay Grebenstein works as a tester and agile QA coach for ZEISS Digital Innovation, Dresden. He has been assuring quality and testing software in projects of different business domains (telecommunication, industry, mail order, energy, …) over the last years. He shares his experiences at conferences, meetups and in publications of different kinds. Show all articles of the author Playwright Api Testing.




Leave a comment

Your email address will not be published. Required fields are marked *