[1]CHEN Xiaoping.Research on formalization and semantic interpretations of correlation degree prediction in large language models[J].CAAI Transactions on Intelligent Systems,2023,18(4):894-900.[doi:10.11992/tis.202306045]
Copy

Research on formalization and semantic interpretations of correlation degree prediction in large language models

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 4 Page number: 894-900 Column: 热点与评论 Public date: 2023-07-15
Title:
Research on formalization and semantic interpretations of correlation degree prediction in large language models
Author(s):
CHEN Xiaoping
School of Computer Science and Technology, University of Science and Technology of China, Hefei 230026, China
Keywords:
large language models; formalization; semantics; conceptualization; weak consensus
CLC:
TP391
DOI:
10.11992/tis.202306045
Abstract:
To explore the problem of what a large language model is, we conduct experimental observation on the evaluation criteria for large language models, intuitively analyze the infrastructure of large language models—correlation degree prediction, of which a formalization LC is constructed and semantic interpretations are explored. On top of these, four challenges of truthfulness, consensus, content attribute, and non-closeness for large language models are discussed. The main findings include: the correlation degrees between tokens are automatically extractable language traces that reflect human language habits; correlation degree prediction has the context-sensitive statistical property; LC has a substantive semantics of weak consensus; LC is a non-conceptualized axiomatic system. These radically differ from the traditional notions of scientific theory, formal methods, artificial intelligence(AI) and software, and are the deep reasons why large language models can behave unexpectedly yet consistent with human language habits.
References:
[1] BENGIO Y, DUCHARME R, VINCENT P. A neural probabilistic language model[J]. NIPS, 2000: 932?938.
[2] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[EB/OL]. (2013?10?16)[2023?06?20]. https:// arxiv.org/abs/1310.4546.
[3] VASWANI A, SHAZEER N M, PARMAR N, et al. Attention is all you need[EB/OL]. (2017?06?12)[2023?06?20].https:// arxiv.org/abs/ 1706.03762.
[4] DEVLIN J, CHANG Mingwei, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. NAACL-HLT (1), 2019: 4171–4176.
[5] BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[EB/OL]. (2020?05?28)[2023?06?20]. https:// arxiv.org/abs/ 2005.14165.
[6] OUYANG Long, WU J, JIANG Xu, et al. Training language models to follow instructions with human feedback [EB/OL]. (2020?03?04)[2023?06?20]. https:// arxiv.org/abs/ 2203.02155.
[7] PETRONI F, ROCKTASCHEL T, LEWIS P, et al. Language models as knowledge bases? [EB/OL]. (2019?09?03)[2023?06?20]. https:// arxiv.org/abs/ 1909.01066.
[8] TURING A. Intelligent machinery[M]. The Turing digital archive, 1948.
[9] 陈小平. 人工智能伦理建设的目标、任务与路径: 六个议题及其依据[J]. 哲学研究, 2020(9): 79–87,107,129
CHEN Xiaoping. The target, tasks, and implementation of artificial intelligence ethics: six issues and the rationale behind them[J]. Philosophical research, 2020(9): 79–87,107,129
[10] NILSSON N J. Artificial intelligence: A new synthesi[M]. Beijing: China Machine Press, 1999.
[11] 陈小平. 人工智能中的封闭性和强封闭性: 现有成果的能力边界、应用条件和伦理风险[J]. 智能系统学报, 2020, 15(1): 114–120
CHEN Xiaoping. Criteria of closeness and strong closeness in artificial intelligence—limits, application conditions and ethical risks of existing technologies[J]. CAAI transactions on intelligent systems, 2020, 15(1): 114–120
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems