Top Testing Trends in 2021
In 2019, SmartBear conducted its third annual survey of the software testing community. The company surveyed 2,526 people from different countries and industries, among which QA and automation engineers dominate — 37%. Also among the respondents — developers (15%), QA managers (14%), architects (8%), manual testers (7%), product managers / business analysts (5%), consultants (4%), DevOps -engineers (4%).
There are many types of software testing, and most often several approaches are taken. The average number of testing types in the respondent teams is 2.2. This shows that testing departments are not hyperspecialized, that is, they do not use multiple testing approaches, but they do not stop at one type either.
In relation to testers who work independently, there are more QA managers who set and control tasks. The researchers explain this by the constant improvement of the qualifications of the respondents and the transition to the next steps of the career ladder.
Since 2017, the size of the companies in which the respondents work has also changed. Previously, everyone wanted to get into a large company (10,000+ employees), but now they prefer smaller companies (less than 500 employees).
What was tested
Since 2017, the number of testing APIs / web services and mobile web applications has increased by about 10%. At the same time, there is a downward trend in testing for desktop applications, hybrid and progressive web applications, and boxed solutions.
Time to test
Time spent on testing has barely changed since 2018 — an average of 61% of work time.
23% of respondents consider the most time-consuming activity to do tests. The creation of automated tests is in second place (18%), followed by the writing of manual test scripts (17%). Reporting / analysis of test results remained the same — fourth place with 12% of respondents, but this is significantly higher than the 3% who said that this was the most time-consuming task in 2017.
Employees of large companies are more satisfied with the well-built software testing process. The reason may be that large organizations, as opposed to small ones, have the resources, history, and knowledge to better conduct testing.
The following correlation emerged: respondents who spent less than 20% of the time testing were less satisfied with the testing process, in contrast to those who spent 20 to 80% testing. This may reflect either greater awareness of the processes and their convenience or lower levels of interest and satisfaction among those who test less frequently.
Testing cycles
Companies continue to move to faster software testing cycles.
The most popular combination of development methodologies is Agile and DevOps. This suggests that companies that are already using Agile continue to improve in these processes and begin to implement DevOps processes.
Only one DevOps approach is used by both small and large companies. And the application of both approaches — Agile and DevOps — is observed more in large companies.
Testing tools and technologies
Jenkins is in the lead among tools for the third year in a row (52%). At the same time, the number of those who do not use CI / CD tools in their work is decreasing from year to year — in 2019 there were only 14% of the total.
The most common languages for writing applications are Java (49%) and JavaScript (44%). The frequency of using C # (24%) and VB.net (11%) is decreasing, but the frequency of using Python is increasing (18%). Swift, R, and Go continue to grow in popularity, but they all remain roughly the same response rate.
The average number of programming languages for writing applications has increased: from 2.36 to 2.54.
A similar distribution among languages has been preserved in the writing of automated tests. Unless Python has swapped places with C #.
Test management
A clear pattern emerged: the smaller the company, the more it doesn’t like to use software to manage the testing process.
Among those who still use such software, there is no single leader program:
Stress Testing
The number of respondents that stress testing is carried out before each publication of the project increased from 30% to 35%.
Among load testing tools, the open-source Apache JMeter comes out on top — it is chosen by 37% of testers. The open-source trend is growing in the IT community, so such a leader is not surprising.
The second place went to LoadRunner (18%). Unidentified, “other” instruments are used by 17%.
Test automation
The degree of software coverage by automated tests has practically not changed since 2017:
And again there is a pattern with the size of companies.
The larger the company, the greater the coverage of the application with autotests.
Keeping tests up to date as the application changes and grows remains the biggest challenge ⟶ 21% of responses. Lack of time for testing is recorded by 15%. Insufficient qualifications of employees are noted by 12%.
UI testing
Difficulties and main challenges
In the list of major difficulties in UI testing, stability testing is in the lead (18%).
New answer option “Start doing this” [haven’t tested UI before, now ready to start. — approx. ed.] immediately took second place with 17%.
Distribution of other answers:
· Definition of the test object and process control (15%).
· Testing in different environments (devices, operating systems) (13%).
· Test coverage check (11%).
· Test maintenance (11%).
· Creation of synthetic data (9%).
Tools
Among the tools for automated UI testing, there are many applications with low popularity.
For the second year in a row, Selenium Webdriver remains the leader (27%). In second place is TestNG (10%). TestComplete closes the top three (8%).
Running tests at the same time
Running several types of tests in parallel saves time on testing, which is critical in a short development cycle.
At the same time, quite a few testers use this feature: only 30% of respondents run more than 10 tests at a time.
Another regularity: almost everyone who does not run tests in parallel does not automate tests.
Concurrent testing is used by almost everyone who uses scripts or other UI testing methods in their work, unlike those who chose Record and Replay as their main UI testing method.
Running tests in the cloud
The number of people who run tests in the cloud is slowly but steadily growing. In 2017, there were 45% of such responses, and in 2019 already 50%.
Popular browsers
The top three have not changed since 2017. The most popular browser for testing web applications is Chrome (53%). Next comes Firefox (41%) and Internet Explorer (35%).
Chrome, Firefox, and Internet Explorer fell heavily in 2018, but the first two rose by about 5% in 2019, but Internet Explorer continues to fall.
38% of respondents test in three or more browsers. The overwhelming majority (51%) test only in the latest version of browsers, 20% in the last two, 11% in the last three.
Headless browsers
Headless browser technology dramatically increases the number of concurrently run tests, thereby nearly tripling the entire testing process. So far, only Chrome and Firefox have headless versions.
According to the survey, 25% of respondents use this technology. The remaining 75% do not know about the use of headless browsers in the company or are sure that this technology is not implemented.
Testing on mobile devices
Every year the business need to test the product on mobile devices grows: only 15% of respondents answered that they do not test mobile devices at all.
IOS devices remain the leader, as in the previous year. Android is on its heels.
The new consumer norm is the expectation that any product will perform as well on a mobile device as it does on a desktop.
Testing Industry Trends
Automation
In 2018, 45% of tests were automated, in 2019–46%. Although not significant, stable growth.
According to the plans of the respondents, by 2021 they want to automate 69% of tests. QA engineers have the highest expectations.
According to the survey, those with a current automation rate of 1–25% plan to increase it to 26–50%. And those who have it at 26–50% want 51–75%. That is, no one claims to be one hundred percent automation, realistically assessing their capabilities.
The most effective option is a weekly product release, which was chosen by about 37% of respondents. In second place is the monthly development cycle (28%). The third place is taken by the quarterly product release (12%).
The graph above allows us to compare the expectations of the respondents from last year’s survey about the frequency of publications and reality. Weekly and monthly publishing targets were not only met but even exceeded expectations. But many did not manage to publish it every day or publish it several times a day — the expectations were too high.
Trends on the horizon for two years
The respondents tried to predict the directions that will gain popularity in the field of testing in the next two years. From the answers that were mentioned more than 10 times, a cloud of simplified tags was collected.
According to forecasts, the most popular areas will be automation, artificial intelligence, and machine learning.
The main thing
The frequency of releases continues to rise as more companies move from Waterfall to a mix of Agile and DevOps.
The types of testing are dominated by web applications and APIs.
There are many different testing tools in use, among which manual testing is in the lead.
The most difficult task for automated testing is synchronizing test scripts with the current version of the application.
Scripting is the most common approach to test automation, and Selenium is the most popular scripting tool.
According to the plans of the respondents, by 2021 they want to automate 69% of tests.
Respondents still see the need to test across different browsers and browser versions, but the number of browser versions has been modest but declined.
A quarter of testers use headless browsers, while just over a quarter are unsure if their organizations are using the technology.
In the next two years, the most popular areas in the testing industry will be automation, artificial intelligence, and machine learning.
