Towards LLM-based Agents for Crowdsourced GUI Testing Simulation

Bachelor & Master Thesis

The rise of large language models (LLMs) has opened new possibilities for mobile app GUI testing, enabling natural language-driven interactions to explore apps and automatically generate test cases [1]. However, most LLM-powered explorative GUI testing methods focus solely on maximizing code coverage through predefined strategies, often overlooking the perspectives of functionality or scenario [2]. This results in blind spots, particularly in apps with complex business logic or domain-specific requirements. In contrast, crowdsourced testing thrives on diversity [3] — multiple independent testers bring unique insights, giving developers well-rounded feedback which can uncover cases that automation alone might miss. However, scaling and managing human testers can be costly and time-consuming.

This project aims to bridge the gap between automated and crowdsourced testing by developing AI-driven agents that mimic the behavior of human testers. By combining the adaptability of human-driven testing with the efficiency of automation, we want to explore a more advanced and scalable solution for automated explorative mobile app GUI testing.

Required knowledge:

Python programming; understanding of LLMs and relevant technologies; understanding of automated GUI testing techniques for mobile apps

What you will do may include:

- Develop an automated GUI testing agent based on the LLM using open-source frameworks.

- Enhance the agent's workflow through combined techniques, such as prompt engineering, knowledge graph, RAG, fine-tuning, etc.

- Construct mobile app datasets, design experiments, and perform comprehensive evaluation and analysis of the developed agent.

For more details about this topic, please contact Dr. Shengcheng Yu (shengcheng.yu[at]tum.de).

Reference:

[1] Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisions. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE '24) (pp. 1-13).

[2] Shengcheng Yu, Chunrong Fang, Mingzhe Du, Zimin Ding, Zhenyu Chen, Zhendong Su. Practical, Automated Scenario-based Mobile App Testing. IEEE Transactions on Software Engineering (2024).

[3] Zhang, Tao, Jerry Gao, and Jing Cheng. Crowdsourced testing services for mobile apps. In 2017 IEEE Symposium on Service-Oriented System Engineering (SOSE) (pp. 75-80). IEEE.