LLMs: killers or boosters
of software testing research?  

12 June 2025 | Online | 10:00 | Professor Antonia Bertolino (Gran Sasso Science Institute - GSSI) 

Abstract
After the big waves in the last two decades of leveraging AI and ML-based technologies for pushing test automation, in the latest couple of years the new big thing is undoubtedly LLM-assisted testing. LLMs are increasingly used for test case generation at differing levels of the testing process, for oracle derivation, for improving test code, as well as for debugging and code repairing. Several studies already show that LLMs can be an effective collaborator in different testing activities, but also warn about possible latent issues. Beyond the specific technique or tool, in this keynote I will talk on how LLMs are already largely transforming testing research. Questions that I would like to put on the discussion table are: can LLMs really substitute human testers? And to what extent can they solve the longstanding challenge of test automation? on what problems should software testing research focus? The talk will be based on a critical reflection of white and grey literature.

Bio
Antonia Bertolino is currently an Adjunct Professor at the Gran Sasso Science Institute (GSSI), L'Aquila, Italy. Previously, she has been a Research Director of the Italian National Research Council (CNR), at the Institute for Information Science and Technologies ‘‘Alessandro Faedo’’ (ISTI), in Pisa, Italy. Her research interests cover software and services validation, testing, and monitoring, and on these topics she has published ~250 papers in international journals, conferences and workshops. She has led or participated in several collaborative projects, including more recently the H2020 European Project ElasTest (Elastic platform for testing complex distributed large software systems), the National project GAUSS (Governing Adaptive and Unplanned Systems of Systems), the FP7 CP Learn PAd (Model-Based Social Learning for Public Administrations), and the FP7 IP CHOReOS (Large Scale Choreographies for the Future Internet). Currently she is the Software Testing Area Chair for Elsevier Journal of Systems and Software, and an Associate Editor of Wiley Journal of Software: Evolution and Process. She serves regularly in the Program Committee of the most renowned conferences in the field of Software Engineering, such as ESEC-FSE, ASE and ICSE, and in software testing and analysis, such as ISSTA and ICST. She was the General Chair of the 2015 edition of the flagship ACM/IEEE ICSE in Florence (Italy), and General/Program Co-Chair, among others, of AST2020, QUATIC2018, ICST2012, CBSE2011, ESEC/FSE2007