Intelligent Crowdsourced Testing

In an article for Wired Magazine in 2006, Jeff Howe defined crowdsourcing as an idea for outsourcing a task that is traditionally performed by a single employee to a large group of people in the form of an open call. Since then, by modifying crowdsourcing into different forms, some of the most successful new companies on the market have used this idea to make people’s lives easier and better. On the other hand, software testing has long been recognized as a time-consuming and expensive activity. Mobile application testing is especially difficult, largely due to compatibility issues: a mobile application must work on devices with different operating systems (e.g. iOS, Android), manufacturers (e.g. Huawei, Samsung) and keypad types (e.g. virtual keypad, hard keypad). One cannot be 100% sure that, just because a tested application works well on one device, it will run smoothly on all others.

Crowdsourced testing is an emerging paradigm that can improve the cost-effectiveness of software testing and accelerate the process, especially for mobile applications. It entrusts testing tasks to online crowdworkers whose diverse testing devices/contexts, experience, and skill sets can significantly contribute to more reliable, cost-effective and efficient testing results. It has already been adopted by many software organizations, including Google, Facebook, Amazon and Microsoft.

This book provides an intelligent overview of crowdsourced testing research and practice. It employs machine learning, data mining, and deep learning techniques to process the data generated during the crowdsourced testing process, to facilitate the management of crowdsourced testing, and to improve the quality of crowdsourced testing.