iSENSE: Completion-Aware Crowdtesting Management

iSENSE: Completion-Aware Crowdtesting Management
Speaker: Prof Ye Yang (Stevens Institute of Technology, New Jersey, USA)

Crowdtesting has grown to be an effective alternative to traditional testing, especially in mobile applications. However, crowdtesting is hard to manage in nature. Given the complexity of mobile applications and unpredictability of distributed crowdtesting process, it is difficult to estimate (a) the remaining number of bugs as yet undetected or (b) the required cost to find those bugs. Experience-based decisions may result in ineffective crowdtesting process, i.e., there are an average of 32% wasteful spending in current crowdtesting practice. This paper aims at exploring automated decision support to effectively manage crowdtesting process. The proposed iSENSE applies incremental sampling technique to process crowdtesting reports arriving in chronological order, organizes them into fixed-size groups as dynamic inputs, and predicts two test completion indicators in an incrementally manner. The two indicators are: 1) total number of bugs predicted with Capture-ReCapture (CRC) model, and 2) required test cost for achieving certain test objectives predicted with Auto Regressive Integrated Moving Average(ARIMA) model. We assess iSENSE using 46,434 reports of 218 crowdtesting tasks from one of the largest crowdtesting platforms in China. Its effectiveness is demonstrated through two applications for automating crowdtesting management, i.e., automation of task closing decision, and semi-automation of task closing trade-off analysis. The results show that decision automation using iSENSE will provide managers with greater opportunities to achieve cost-effectiveness gains of crowdtesting, i.e. a median of 100% bugs detected with 30% saved cost based on the automated close prediction.


Biography
Dr. Ye Yang is an Associate Professor in the School of Systems and Enterprises, at Stevens Institute of Technology, New Jersey, USA. She received her Ph.D. in Computer Science from the University of Southern California in 2006. Before joining Stevens, she was a Research Professor at the Institute of Software, Chinese Academy of Sciences (ISCAS). Her research interests span the spectrum of software process, software measurement and analysis, software predictive models, socio-technical aspects, software crowdsourcing, and technical debt. She has extensive work software process modeling and simulation, predictive analytics for software crowdsourcing, economic models for software cost and schedule estimation, recommendation systems for managing software projects. She has co-edited eight conference proceedings, published over 100 referred journal and conference papers, and some of them won Best Research Paper Awards, including the ACM Sigsoft Distinguished Paper Award at ICSE 2019. She has served as Program co-Chairs/co-Organizers for CSI-SE2018, SER&IP 2019, and ICSP 2010, Steering Committee member for PROMISE 2012-2013, and Program Committee member for a number of international conferences including ASE, ICSE, ESEM, APSEC, etc.