[1]陈小平.大模型关联度预测的形式化和语义解释研究[J].智能系统学报,2023,18(4):894-900.[doi:10.11992/tis.202306045]
CHEN Xiaoping.Research on formalization and semantic interpretations of correlation degree prediction in large language models[J].CAAI Transactions on Intelligent Systems,2023,18(4):894-900.[doi:10.11992/tis.202306045]
点击复制
CHEN Xiaoping.Research on formalization and semantic interpretations of correlation degree prediction in large language models[J].CAAI Transactions on Intelligent Systems,2023,18(4):894-900.[doi:10.11992/tis.202306045]
大模型关联度预测的形式化和语义解释研究
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第4期
页码:
894-900
栏目:
热点与评论
出版日期:
2023-07-15
- Title:
- Research on formalization and semantic interpretations of correlation degree prediction in large language models
- 分类号:
- TP391
- 摘要:
- 本文探讨“大型语言模型是什么”的问题。为此对大模型的评判标准展开实验观察，对大模型的基础设施关联度预测进行直观分析，构建关联度预测的一种形式化LC，进而研究关联度预测的语义解释。在此基础上讨论大模型的真实性挑战、共识挑战、内容属性挑战和非封闭性挑战。主要发现包括：语元关联度是体现人类语言习惯的可自动提取的语言痕迹；关联度预测具有语境相关的统计性质；LC具有弱共识性实质语义；LC是一个非概念化公理系统。这些特点颠覆了科学理论、形式化方法和软件的传统理念在人工智能领域的主导地位，是大模型输出既出人预料、又符合语言习惯的深层原因。
- Abstract:
- To explore the problem of what a large language model is, we conduct experimental observation on the evaluation criteria for large language models, intuitively analyze the infrastructure of large language models—correlation degree prediction, of which a formalization LC is constructed and semantic interpretations are explored. On top of these, four challenges of truthfulness, consensus, content attribute, and non-closeness for large language models are discussed. The main findings include: the correlation degrees between tokens are automatically extractable language traces that reflect human language habits; correlation degree prediction has the context-sensitive statistical property; LC has a substantive semantics of weak consensus; LC is a non-conceptualized axiomatic system. These radically differ from the traditional notions of scientific theory, formal methods, artificial intelligence(AI) and software, and are the deep reasons why large language models can behave unexpectedly yet consistent with human language habits.
- 参考文献/References:
-
[1] BENGIO Y, DUCHARME R, VINCENT P. A neural probabilistic language model[J]. NIPS, 2000: 932?938.
[2] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[EB/OL]. (2013?10?16)[2023?06?20]. https:// arxiv.org/abs/1310.4546.
[3] VASWANI A, SHAZEER N M, PARMAR N, et al. Attention is all you need[EB/OL]. (2017?06?12)[2023?06?20].https:// arxiv.org/abs/ 1706.03762.
[4] DEVLIN J, CHANG Mingwei, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. NAACL-HLT (1), 2019: 4171–4176.
[5] BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[EB/OL]. (2020?05?28)[2023?06?20]. https:// arxiv.org/abs/ 2005.14165.
[6] OUYANG Long, WU J, JIANG Xu, et al. Training language models to follow instructions with human feedback [EB/OL]. (2020?03?04)[2023?06?20]. https:// arxiv.org/abs/ 2203.02155.
[7] PETRONI F, ROCKTASCHEL T, LEWIS P, et al. Language models as knowledge bases? [EB/OL]. (2019?09?03)[2023?06?20]. https:// arxiv.org/abs/ 1909.01066.
[8] TURING A. Intelligent machinery[M]. The Turing digital archive, 1948.
[9] 陈小平. 人工智能伦理建设的目标、任务与路径: 六个议题及其依据[J]. 哲学研究, 2020(9): 79–87,107,129
CHEN Xiaoping. The target, tasks, and implementation of artificial intelligence ethics: six issues and the rationale behind them[J]. Philosophical research, 2020(9): 79–87,107,129
[10] NILSSON N J. Artificial intelligence: A new synthesi[M]. Beijing: China Machine Press, 1999.
[11] 陈小平. 人工智能中的封闭性和强封闭性: 现有成果的能力边界、应用条件和伦理风险[J]. 智能系统学报, 2020, 15(1): 114–120
CHEN Xiaoping. Criteria of closeness and strong closeness in artificial intelligence—limits, application conditions and ethical risks of existing technologies[J]. CAAI transactions on intelligent systems, 2020, 15(1): 114–120
备注/Memo
收稿日期:2023-06-20。
基金项目:国家自然科学基金项目（92048301, U1613216）；国家重点研发计划项目（2020YFB1313602）.
作者简介:陈小平，教授、机器人实验室主任，广东省科学院人工智能首席科学家，中国人工智能学会人工智能伦理与治理专委会主任，中国管理科学学会大数据管理专委会副主任。曾任 2015世界人工智能联合大会（IJCAI2015）机器人领域主席、2008 和 2015 机器人世界杯及学术大会（RoboCup2008, 2015）主席、Journal of Artificial Intelligence Research 和 Knowledge Engineering Review 编委。
通讯作者:陈小平.E-mail:xpchen@ustc.edu.cn
更新日期/Last Update:
1900-01-01