[1]GUO Shaocheng,CHEN Songcan.Sparsified factorization machine[J].CAAI Transactions on Intelligent Systems,2017,12(6):816-822.[doi:10.11992/tis.201706030]
Copy
Sparsified factorization machine
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
12
Number of periods:
2017 6
Page number:
816-822
Column:
学术论文—人工智能基础
Public date:
2017-12-25
- Title:
- Sparsified factorization machine
- Keywords:
- factorization machine; sparsity; sparse group lasso; feature selection; recommender systems
- CLC:
- TP391
- DOI:
- 10.11992/tis.201706030
- Abstract:
- Factorization machine (FM) is a recently proposed second-order linear model. One of its main advantages is that the interactions within it are factorized, making it suitable for data with high dimensionality and high sparsity. Though FM has been applied in recommender systems, it fails to consider the sparsity of variables explicitly, especially when the variable contains information on structural sparsity. Therefore, the process of feature selection should meet the following requirements: the linear terms and second-order terms that share the same feature should be included or excluded at the same time; when the feature is noise, both should be excluded, otherwise, both should be included. Based on the sparse structure described above, this paper proposes a sparse group lasso-based factorization machine (SGL-FM). By adding sparse group lasso to the loss function, SGL-FM not only achieves sparsity between groups but also within groups. From another point of view, sparsity within groups can be seen as a method of controlling the dimensionality of the factorization; therefore, SGL-FM chooses the best k automatically when faced with datasets with different properties. The experimental results show that by applying the proposed method, under conditions of excellent precision, a model with more sparsity than FM was obtained.
- References:
-
[1] RAO C R, TOUTENBURG H. Linear models[M]. New York: Springer, 1995: 3-18.
[2] ADOMAVICIUS G, TUZHILIN A. Context-aware recommender systems[M]. US: Springer, 2015: 191-226.
[3] RENDLE S. Factorization machines[C]//IEEE 10th International Conference on Data Mining. Sydney, Australia, 2010: 995-1000.
[4] RENDLE S. Learning recommender systems with adaptive regularization[C]//Proceedings of the fifth ACM international conference on Web search and data mining. Seattle, USA, 2012: 133-142.
[5] TIBSHIRANI R. Regression shrinkage and selection via the lasso[J]. Journal of the royal statistical society, Series B (Methodological), 1996,73(3): 267-288.
[6] YUAN M, LIN Y. Model selection and estimation in regression with grouped variables[J]. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2006, 68(1): 49-67.
[7] SIMON N, FRIEDMAN J, HASTIE T, et al. A sparse-group lasso[J]. Journal of computational and graphical statistics, 2013, 22(2): 231-245.
[8] BLONDEL M, FUJINO A, UEDA N, et al. Higher-order factorization machines[C]//Advances in Neural Information Processing Systems. Barcelona, Spain 2016: 3351-3359.
[9] LI M, LIU Z, SMOLA A J, et al. DiFacto: distributed factorization machines[C]//Proceedings of the Ninth ACM International Conference on Web Search and Data Mining. San Francisco, USA, 2016: 377-386.
[10] CHIN W S, YUAN B, YANG M Y, et al. An efficient alternating newton method for learning factorization machines [R].NTU:NTU,2016.
[11] DUCHI J, SINGER Y. Efficient online and batch learning using forward backward splitting[J]. Journal of Machine Learning Research, 2009, 10(12): 2899-2934.
[12] RENDLE S. Factorization machines with libfm[J]. ACM transactions on intelligent systems and technolog, 2012, 3(3): 57.
[13] LIU J, YE J. Moreau-Yosida regularization for grouped tree structure learning[C]//Advances in Neural Information Processing Systems. Vancouver, Canada, 2010: 1459-1467.
- Similar References:
Memo
-
Last Update:
2018-01-03