报告题目 | A Discrepancy-Based Design for A/B Testing Experiments | ||
报告人（单位） | Prof. Lulu Kang Illinois Institute of Technology | ||
点评人（单位） | 薛巍立（东南大学） | 点评人（单位） | 赖明辉（东南大学） |
时间地点 | 时间：2019年 12月 19 日（周四） 上午10 点 地点：九龙湖经管楼B201 | ||
The aim of this paper is to introduce a new design of experiment method for A/B tests in order to balance the covariate information in all treatment groups. A/B tests (or ``A/B/n tests'') refer to the experiments and the corresponding inference on the treatment effect(s) of a two-level or multi-level controllable experimental factor. The common practice is to use a randomized design and perform hypothesis tests on the estimates. However, such estimation and inference are not always accurate when covariate imbalance exists among the treatment groups. To overcome this issue, we propose a discrepancy-based criterion and show that the design minimizing this criterion significantly improves the accuracy of the treatment effect(s) estimates. The discrepancy-based criterion is model-free and thus makes the estimation of the treatment effect(s) robust to the model assumptions. More importantly, the proposed design is applicable to both continuous and categorical response measurements. We develop two efficient algorithms to construct the designs by optimizing the criterion for both offline and online A/B tests. Through simulation study and a real example, we show that the proposed design approach achieves good covariate balance and accurate estimation.
Dr. Lulu Kang is an Associate Professor of the Department of Applied Math at Illinois Institute of Technology (IIT). She obtained her M.S. in Operations Research and Ph.D. in Industrial Engineering from the Stewart School of Industrial and Systems Engineering at Georgia Tech in 2010.
Dr. Kang’s has worked on various areas in Statistics, including uncertainty quantification, statistical design and analysis of experiments, Bayesian computational statistics, etc. To be more specific, Dr. Kang develops theories and implementable algorithms to achieve effective and efficient data collection, data analysis, and optimizations for complex systems in manufacturing, energy, and other engineering fields. She has publications and submitted papers in top statistical journals including Techometrics, SIAM/ASA Journal on Uncertainty Quantification, Statistica Sinica, etc. Dr. Kang has developed and taught many statistical courses including Statistical Learning, Bayesian Computational Statistics, and Regression and Forecasting, etc. She is also the co-founder and Acting Program Director of the Data Science Program at IIT. | |||
