[1]LI Qingyong,HE Jun,ZHANG Chunxiao.Unsupervised domain adaptation algorithm based on classification discrepancy and information entropy[J].CAAI Transactions on Intelligent Systems,2021,16(6):999-1006.[doi:10.11992/tis.202010020]
Copy
Unsupervised domain adaptation algorithm based on classification discrepancy and information entropy
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
16
Number of periods:
2021 6
Page number:
999-1006
Column:
学术论文—机器学习
Public date:
2021-11-05
- Title:
- Unsupervised domain adaptation algorithm based on classification discrepancy and information entropy
- Keywords:
- domain adaptation; confrontation training; neural network; unsupervised learning; transfer learning; classification discrepancy; information entropy; decision boundary
- CLC:
- TP391
- DOI:
- 10.11992/tis.202010020
- Abstract:
- The adversarial training method has become the mainstream of the domain adaptation algorithm. The feature distributions of the source and target domains are aligned by a domain classifier to reduce the feature distribution discrepancy among different domains. However, existing domain adaptation methods only reduce the distance between different domain data without considering the relationship between the data distribution of the target domain and decision boundaries, thus decreasing the intradomain distinguishability of different categories in the target domain. Considering the shortcomings of the existing methods, an unsupervised domain adaptation algorithm based on classification discrepancy and information entropy confrontation (ACDIE) is proposed in this study. The algorithm uses the discrepancy and the domain aligning discrepancy between two classifiers and minimizes the information entropy to reduce uncertainty. Consequently, the proposed method makes the target domain feature far away from the decision boundaries and improves the distinguishability of different categories. The experimental results of the digital identification and Office-31 datasets show that the ACDIE algorithm can learn better feature representation. Moreover, the accuracy of the domain adaptation classification is considerably improved.
- References:
-
[1] WANG Xiaolong, GUPTA A. Unsupervised learning of visual representations using videos[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 2794-2802.
[2] MAHJOURIAN R, WICKE M, ANGELOVA A. Unsupervised learning of depth and ego-motion from monocular video using 3D geometric constraints[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 5667-5675.
[3] 刘建伟, 孙正康, 罗雄麟. 域自适应学习研究进展[J]. 自动化学报, 2014, 40(8): 1576-1600
LIU Jianwei, SUN Zhengkang, LUO Xionglin. Review and research development on domain adaptation learning[J]. Acta automatica sinica, 2014, 40(8): 1576-1600
[4] PAN S J, YANG Qiang. A survey on transfer learning[J]. IEEE transactions on knowledge and data engineering, 2010, 22(10): 1345-1359.
[5] ROZANTSEV A, SALZMANN M, FUA P. Beyond sharing weights for deep domain adaptation[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(4): 801-814.
[6] GHIFARY M, KLEIJN W B, ZHANG Mengjie, et al. Deep reconstruction-classification networks for unsupervised domain adaptation[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016: 597-613.
[7] SENER O, SONG H O, SAXENA A, et al. Learning transferrable representations for unsupervised domain adaptation[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: Curran Associates Inc., 2016: 2110-2118.
[8] SUN Baochen, FENG Jiashi, SAENKO K. Return of frustratingly easy domain adaptation[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Phoenix, Arizona: AAAI Press, 2016: 2058-2065.
[9] GRETTON A, BORGWARDT K M, RASCH M J, et al. A kernel two-sample test[J]. The journal of machine learning research, 2012, 13: 723-773.
[10] TZENG E, HOFFMAN J, ZHANG Ning, et al. Deep domain confusion: maximizing for domain invariance[J]. Computer science, 2014.
[11] LONG Mingsheng, CAO Yue, WANG Jianmin, et al. Learning transferable features with deep adaptation networks[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille, France: JMLR, 2015: 97-105.
[12] LONG Mingsheng, ZHU Han, WANG Jianmin, et al. Unsupervised domain adaptation with residual transfer networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: Curran Associates Inc., 2016: 136-144.
[13] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press, 2014: 2672-2680.
[14] GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks[J]. The journal of machine learning research, 2016, 17(1): 2096-2030.
[15] 王格格, 郭涛, 余游, 等. 基于生成对抗网络的无监督域适应分类模型[J]. 电子学报, 2020, 48(6): 1190-1197
WANG Gege, GUO Tao, YU You, et al. Unsupervised domain adaptation classification model based on generative adversarial network[J]. Acta electronica sinica, 2020, 48(6): 1190-1197
[16] SANKARANARAYANAN S, BALAJI Y, CASTILLO C D, et al. Generate to adapt: aligning domains using generative adversarial networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 8503-8512.
[17] SAITO K, WATANABE K, USHIKU Y, et al. Maximum classifier discrepancy for unsupervised domain adaptation[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 3723-3732.
[18] BEN-DAVID S, BLITZER J, CRAMMER K, et al. A theory of learning from different domains[J]. Machine learning, 2010, 79(1/2): 151-175.
[19] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[20] HULL J J. A database for handwritten text recognition research[J]. IEEE transactions on pattern analysis and machine intelligence, 1994, 16(5): 550-554.
[21] NETZER Y, WANG T, COATES A, et al. Reading digits in natural images with unsupervised feature learning[C]//Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning. Granada, Spain, 2011: 5-16.
[22] MOISEEV B, KONEV A, CHIGORIN A, et al. Evaluation of traffic sign recognition methods trained on synthetically generated data[C]//Proceedings of the 15th International Conference on Advanced Concepts for Intelligent Vision Systems. Poznań, Poland: Springer, 2013: 576-583.
[23] STALLKAMP J, SCHLIPSING M, SALMEN J, et al. The German traffic sign recognition benchmark: a multi-class classification competition[C]//Proceedings of 2011 International Joint Conference on Neural Networks. San Jose, USA: IEEE, 2011: 1453-1460.
[24] BOUSMALIS K, TRIGEORGIS G, SILBERMAN N, et al. Domain separation networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: Curran Associates Inc., 2016: 343-351.
[25] TZENG E, HOFFMAN J, SAENKO K, et al. Adversarial discriminative domain adaptation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 7167-7176.
[26] LIU Mingyu, TUZEL O. Coupled generative adversarial networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: Curran Associates Inc., 2016: 469-477.
[27] LONG Mingsheng, CAO Zhangjie, WANG Jianmin, et al. Conditional adversarial domain adaptation[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal, Canada: Curran Associates Inc., 2018: 1647-1657.
[28] VAN DER MAATEN L, HINTON G. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(2605): 2579-2605.
- Similar References:
Memo
-
Last Update:
2021-12-25