Efficient Learning to Learn a Robust CTR Model for Web-scale Online Sponsored Search Advertising

TY - GEN

T1 - Efficient Learning to Learn a Robust CTR Model for Web-scale Online Sponsored Search Advertising

AU - Wang, Xin

AU - Yang, Peng

AU - Chen, Shaopeng

AU - Liu, Lin

AU - Zhao, Lian

AU - Guo, Jiacheng

AU - Sun, Mingming

AU - Li, Ping

N1 - Publisher Copyright: © 2021 ACM.

PY - 2021/10/26

Y1 - 2021/10/26

N2 - Click-through rate (CTR) prediction is crucial for online sponsored search advertising. Several successful CTR models have been adopted in the industry, including the regularized logistic regression (LR). Nonetheless, the learning process suffers from two limitations: 1) Feature crosses for high-order information may generate trillions of features, which are sparse for online learning examples; 2) Rapid changing of data distribution brings challenges to the accurate learning since the model has to perform a fast adaptation on the new data. Moreover, existing adaptive optimizers are ineffective in handling the sparsity issue for high-dimensional features. In this paper, we propose to learn an optimizer in a meta-learning scenario, where the optimizer is learned on prior data and can be easily adapted to the new data. We firstly build a low-dimensional feature embedding on prior data to encode the association among features. Then, the gradients on new data can be decomposed into the low-dimensional space, enabling the parameter update smoothed and relieving the sparsity. Note that this technology could be deployed into a distributed system to ensure efficient online learning on the trillions-level parameters. We conduct extensive experiments to evaluate the algorithm in terms of prediction accuracy and actual revenue. Experimental results demonstrate that the proposed framework achieves a promising prediction on the new data. The final online revenue is noticeably improved compared to the baseline. This framework was initially deployed in Baidu Search Ads (a.k.a. Phoenix Nest) in 2014 and is currently still being used in certain modules of Baidu's ads systems.

AB - Click-through rate (CTR) prediction is crucial for online sponsored search advertising. Several successful CTR models have been adopted in the industry, including the regularized logistic regression (LR). Nonetheless, the learning process suffers from two limitations: 1) Feature crosses for high-order information may generate trillions of features, which are sparse for online learning examples; 2) Rapid changing of data distribution brings challenges to the accurate learning since the model has to perform a fast adaptation on the new data. Moreover, existing adaptive optimizers are ineffective in handling the sparsity issue for high-dimensional features. In this paper, we propose to learn an optimizer in a meta-learning scenario, where the optimizer is learned on prior data and can be easily adapted to the new data. We firstly build a low-dimensional feature embedding on prior data to encode the association among features. Then, the gradients on new data can be decomposed into the low-dimensional space, enabling the parameter update smoothed and relieving the sparsity. Note that this technology could be deployed into a distributed system to ensure efficient online learning on the trillions-level parameters. We conduct extensive experiments to evaluate the algorithm in terms of prediction accuracy and actual revenue. Experimental results demonstrate that the proposed framework achieves a promising prediction on the new data. The final online revenue is noticeably improved compared to the baseline. This framework was initially deployed in Baidu Search Ads (a.k.a. Phoenix Nest) in 2014 and is currently still being used in certain modules of Baidu's ads systems.

KW - computational advertising

KW - ctr prediction

KW - meta-learning

UR - http://www.scopus.com/inward/record.url?scp=85119194384&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85119194384&partnerID=8YFLogxK

U2 - 10.1145/3459637.3481912

DO - 10.1145/3459637.3481912

M3 - Conference contribution

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 4203

EP - 4213

BT - CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management

PB - Association for Computing Machinery

T2 - 30th ACM International Conference on Information and Knowledge Management, CIKM 2021

Y2 - 1 November 2021 through 5 November 2021

ER -