[1]丁俐夫,颜钢锋.多智能体系统安全性问题及防御机制综述[J].智能系统学报,2020,15(3):425-434.[doi:10.11992/tis.201812015]
 DING Lifu,YAN Gangfeng.A survey of the security issues and defense mechanisms of multi-agent systems[J].CAAI Transactions on Intelligent Systems,2020,15(3):425-434.[doi:10.11992/tis.201812015]
点击复制

多智能体系统安全性问题及防御机制综述(/HTML)
分享到:

《智能系统学报》[ISSN:1673-4785/CN:23-1538/TP]

卷:
第15卷
期数:
2020年3期
页码:
425-434
栏目:
综述
出版日期:
2020-09-05

文章信息/Info

Title:
A survey of the security issues and defense mechanisms of multi-agent systems
作者:
丁俐夫1 颜钢锋2
1. 浙江大学 电气工程学院,浙江 杭州 310027;
2. 浙江大学 华南工业技术研究院,广东 广州 510760
Author(s):
DING Lifu1 YAN Gangfeng2
1. College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China;
2. Huanan Industrial Technology Research Institute, Zhejiang University, Guangzhou 510760, China
关键词:
多智能体系统分布式人工智能安全威胁防御机制网络安全通信协议访问控制协调机制
Keywords:
multi-agent systemdistributed artificial intelligencesecurity threatdefense mechanismnetwork securitycommunication protocolaccess controlcoordination mechanism
分类号:
TP18;TP309
DOI:
10.11992/tis.201812015
摘要:
多智能体系统作为分布式人工智能的重要分支,已成为解决大型、复杂、分布式及难预测问题的重要手段。在开放网络中,多智能体系统仍面临许多安全问题,潜在的安全威胁很可能影响其实际应用的稳定性、快速性和准确性。基于目前已知的多智能体系统通用模型,介绍了多智能体系统通信协议、访问控制和协调机制中潜在的安全问题,规范了多智能体系统安全性问题的研究体系,总结了系统设计过程中可行的防御技术和隐私保护技术,最后展望了多智能体系统安全研究的发展方向。
Abstract:
Currently, multi-agent systems are an important branch of distributed artificial intelligence and an important means of solving large, complex, distributed, and difficult-to-predict problems. However, in an open network, multi-agent systems still face many security issues. Potential security threats are likely to affect the stability, speed, and accuracy of the practical applications of multi-agent systems. Based on the general models of currently known multi-agent systems, the potential security problems in multi-agent system communication protocol, access control, and coordination mechanism are introduced. Research on multi-agent system security problems is standardized, and feasible defense and privacy protection technologies in the system design process are summarized. Finally, the development trend of multi-agent system security research is analyzed.

参考文献/References:

[1] GREENBERG M S, BYINGTON J C, HARPER D G. Mobile agents and security[J]. IEEE communications magazine, 1998, 36(7): 76-85.
[2] CENTENO R, FAGUNDES M, BILLHARDT H, et al. Supporting medical emergencies by MAS[C]//Proceedings of the 3rd Kes International Symposium on Agent and Multi-Agent Systems: Technologies and Applications. Uppsala, Sweden, 2009: 823-833.
[3] DIOUBATE M, TAN Guanzheng, MOHAMED L T. An artificial immune system based multi-agent model and its application to robot cooperation problem[C]//Proceedings of the 2008 7th World Congress on Intelligent Control and Automation. Chongqing, China, 2008: 3033-3039.
[4] SZYMAK P, PRACZYK T. Control systems of underwater vehicles in multi-agent system of underwater inspection[C]//Proceedings of the 11th WSEAS International Conference on Automatic Control, Modelling and Simulation. Istanbul, Turkey, 2009: 153-156.
[5] HUANG Xiao, ZHANG Qi, WANG Yu. Research on multi-agent traffic signal control system based on VANET information[C]//Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems. Yokohama, Japan, 2018: 1-6.
[6] 姚尧, 卢淑娟, 徐德民. IMMAS算法在复杂环境下路径规划中的应用[J]. 计算机仿真, 2007, 24(12): 148-151
YAO Yao, LU Shujuan, XU Demin. IMMAS for robot path planning in complex environment[J]. Computer simulation, 2007, 24(12): 148-151
[7] IQBAL S, ALTAF W, ASLAM M, et al. Application of intelligent agents in health-care: review[J]. Artificial intelligence review, 2016, 46(1): 83-112.
[8] BIJANI S, ROBERTSON D. A review of attacks and security approaches in open multi-agent systems[J]. Artificial intelligence review, 2014, 42(4): 607-636.
[9] HUYNH T D, JENNINGS N R, SHADBOLT N R. An integrated trust and reputation model for open multi-agent systems[J]. Autonomous agents and multi-agent systems, 2006, 13(2): 119-154.
[10] WANG S, HU J, LIU A, et al. Security frame and evaluation in mobile agent system[C]. International Conference on Mobile Technology Applications and Systems. Guangzhou, China, 2005: 1-6.
[11] JUNG Y, KIM M, MASOUMZADEH A, et al. A survey of security issue in multi-agent systems[J]. Artificial intelligence review, 2012, 37(3): 239-260.
[12] Foundation for Intelligent Physical Agents (FIPA). FIPA Abstract Architecture Specification 2002 [EB/OL]. (2002-02-25)[2018-12-01]http://www.fipa.org/specs/fipaoo001/SC00001L.pdf.
[13] Foundation for Intelligent Physical Agents (FIPA). FIPA ACL Message Structure Specification [EB/OL]. (2000-01-08)[2018-12-01]http//www.fipa.org/specs/fipa00061/SC00061G.html.
[14] Foundation for Intelligent Physical Agents (FIPA). FIPA agent management specification [EB/OL]. (2004-03-18) [2018-12-01]http://www.fipa.org/specs/fipaO0023/SCO0023K.pdf.
[15] MOURATIDIS H, GIORGINI P, MANSON G. Modelling secure multiagent systems[C]//Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multiagent Systems. Melbourne, Australia, 2003: 859-866.
[16] BRESCIANI P, GIORGINI P, MOURATIDIS H, et al. Multi-agent systems and security requirements analysis[M]//LUCENA C, GARCIA A, ROMANOVSKY A, et al. Software Engineering for Multi-Agent Systems II. Berlin, Germany: Springer, 2004: 35-48.
[17] POSLAD S, CHARLTON P, CALISTI M. Specifying standard security mechanisms in multi-agent systems[M]//FALCONE R, BARBER S, KORBA L, et al. Trust, Reputation, and Security: Theories and Practice. Berlin, Germany: Springer, 2003: 163-176.
[18] BRESCIANI P, PERINI A, GIORGINI P, et al. Tropos: an agent-oriented software development methodology[J]. Autonomous agents and multi-agent systems, 2004, 8(3): 203-236.
[19] MOURATIDIS H, GIORGINI P. Secure tropos: a security-oriented extension of the tropos methodology[J]. International journal of software engineering and knowledge engineering, 2007, 17(2): 285-309.
[20] MOURATIDIS H, GIORGINI P. Enhancing secure tropos to effectively deal with security requirements in the development of multiagent systems[M]//BARLEY M, MOURATIDIS H, UNRUH A, et al. Safety and Security in Multiagent Systems. Berlin: Springer, 2009: 8-26.
[21] BEYDOUN G, LOW G, MOURATIDIS H, et al. A security-aware metamodel for multi-agent systems (MAS)[J]. Information and software technology, 2009, 51(5): 832-845.
[22] MORADIAN E. Security of E-commerce software systems[M]//HAKANSSON A, HARTUNG R. Agent and Multi-Agent Systems in Distributed Systems-Digital Economy and E-Commerce. Berlin: Springer, 2013: 95-103.
[23] BASHEER G S, AHMAD M S, TANG A Y C, et al. Certainty, trust and evidence: towards an integrative model of confidence in multi-agent systems[J]. Computers in human behavior, 2015, 45: 307-315.
[24] ZIKRATOV I, MASLENNIKOV O, LEBEDEV I, et al. Dynamic trust management framework for robotic multi-agent systems[C]//Proceedings of the 16th International Conference on Internet of Things, Smart Spaces, and Next Generation Networks and Systems. Petersburg, Russia, 2016.
[25] CHERIBI H, KHOLLADI M K. A security model for complex applications based on normative multi-agents system[C]//Proceedings of the 2015 2nd International Conference on Information Security and Cyber Forensics. Cape Town, South Africa, 2015:41-46.
[26] MAJD E, BALAKRISHNAN V. A trust model for recommender agent systems[J]. Soft computing — a fusion of foundations, methodologies and applications, 2017, 21(2): 417-433.
[27] BüRKLE A, HERTEL A, MüLLER W, et al. Evaluating the security of mobile agent platforms[J]. Autonomous agents and multi-agent systems, 2009, 18(2): 295-311.
[28] VILA X, SCHUSTER A, RIERA A. Security for a multi-agent system based on JADE[J]. Computers & security, 2007, 26(5): 391-400.
[29] VITABILE S, CONTI V, MILITELLO C, et al. An extended JADE-S based framework for developing secure Multi-Agent Systems[J]. Computer standards & interfaces, 2009, 31(5): 913-930.
[30] JANSEN W A. Countermeasures for mobile agent security[J]. Computer communications, 2000, 23(17): 1667-1676.
[31] ROTH V. Secure recording of itineraries through co-operating agents[C]//Proceedings of 1998 European Conference on Object-Oriented Programming. Belgium, Germany, 1998: 297-298.
[32] FARMER W M, GUTTMAN J D, SWARUP V. Security for mobile agents: authentication and state appraisal[C]//Proceedings of the 4th European Symposium on Research in Computer Security. Rome, Italy, 1996: 118-130.
[33] NECULA G C, LEE P. Safe, untrusted agents using proof-carrying code[M]//VIGNA G. Mobile Agents and Security. Berlin: Springer, 1998: 61-91.
[34] PARUCHURI P, TAMBE M, ORDó?EZ F, et al. Security in multiagent systems by policy randomization[C]//Proceedings of the 5th International Joint Conference on Autonomous Agents and Multiagent Systems. Hakodate, Japan, 2006.
[35] OLFATI-SABER R, FAX J A, MURRAY R M. Consensus and cooperation in networked multi-agent systems[J]. Proceedings of the IEEE, 2007, 95(1): 215-233.
[36] HE Wangli, GAO Xiaoyang, ZHONG Weimin, et al. Secure impulsive synchronization control of multi-agent systems under deception attacks[J]. Information sciences, 2018, 459: 354-368.
[37] AMULLEN E M, SHETTY S, KEEL L H. Secured formation control for multi-agent systems under DoS attacks[C]//Proceedings of 2016 IEEE Symposium on Technologies for Homeland Security. Waltham, USA, 2016.
[38] DE LA TORRE G, YUCELEN T, PETERSON J D. Resilient networked multiagent systems: a distributed adaptive control approachy[C]//Proceedings of the 53rd IEEE Conference on Decision and Control. Los Angeles, USA, 2014.
[39] 伍益明. 恶意攻击下的多智能体系统安全一致性问题研究[D]. 杭州: 浙江工业大学, 2016.
WU Yiming. Research on secure consensus for multi-agent systems under malicious attacks[D]. Hangzhou: Zhejiang University of Technology, 2016.
[40] LIU S, XIE L, ZHANG H, et al. Distributed consensus for multi-agent systems with delays and noises in transmission channels[J]. Automatica, 2011, 47(5): 920-934.
[41] WU Yiming, HE Xiongxiong. Secure consensus control for multiagent systems with attacks and communication delays[J]. IEEE/CAA journal of automatica sinica, 2017, 4(1): 136-142.
[42] MENG Deyuan, JIA Yingmin, DU Junping. Finite-time consensus for multiagent systems with cooperative and antagonistic interactions[J]. IEEE transactions on neural networks and learning systems, 2016, 27(4): 762-770.
[43] MENG Deyuan, ZUO Zongyu. Signed-average consensus for networks of agents: a nonlinear fixed-time convergence protocol[J]. Nonlinear dynamics, 2016, 85(1): 155-165.
[44] LIU Xiaoyang, LAM J, YU Wenwu, et al. Finite-time consensus of multiagent systems with a switching protocol[J]. IEEE transactions on neural networks and learning systems, 2016, 27(4): 853-862.
[45] LI Jinsha, LI Junmin. Adaptive iterative learning control for coordination of second-order multi-agent systems[J]. International journal of robust and nonlinear control, 2014, 24(18): 3282-3299.
[46] WU Yiming, XU Ming, ZHENG Ning, et al. Attack tolerant finite-time consensus for multi-agent networks[C]//Proceedings of the 2017 13th IEEE International Conference on Control & Automation. Ohrid, Macedonia, 2017.
[47] WU Yiming, HE Xiongxiong, LIU Shuai. Resilient consensus for multi-agent systems with quantized communication[C]//Proceedings of 2016 American Control Conference. Boston, USA, 2016.
[48] FENG Zhi, HU Guoqiang, WEN Guanghui. Distributed consensus tracking for multi-agent systems under two types of attacks[J]. International journal of robust and nonlinear control, 2016, 26(5): 896-918.
[49] FENG Zhi, WEN Guanghui, HU Guoqiang. Distributed secure coordinated control for multiagent systems under strategic attacks[J]. IEEE transactions on cybernetics, 2017, 47(5): 1273-1284.
[50] FENG Zhi, HU Guoqiang. Distributed secure leader-following consensus of multi-agent systems under DoS attacks and directed topology[C]//Proceedings of 2017 IEEE International Conference on Information and Automation. Macau, China, 2017.
[51] PITT J, MAMDANI A. Communication protocols in multi-agent systems: a development method and reference architecture[M]//DIGNUM F, GREAVES M. Issues in Agent Communication. Berlin: Springer, 2000: 160-177.
[52] YOKOO M, SUZUKI K. Secure multi-agent dynamic programming based on homomorphic encryption and its application to combinatorial auctions[C]//Proceedings of the 1st International Joint Conference on Autonomous Agents and Multiagent Systems. Bologna, Italy, 2002.
[53] ZHU Liehuang, CAO Yuanda, LIAO Lejian, et al. SKQML: a secure multi-agent communication language [C]//Proceedings of the 5th WSEAS International Conference on Applied Computer Science. Hangzhou, China, 2006.
[54] ABDENNADHER S, ABDALLAH M, MUELLER H J. DCP-MASR: a dynamic communication protocol for robot-based multi-agent systems[C]//Proceedings of 2010 5th International Conference on Systems. Menuires, France, 2010: 12-17.
[55] SHEHADA D, YEUN C Y, ZEMERLY M J, et al. BROSMAP: a novel broadcast based secure mobile agent protocol for distributed service applications[J]. Security and communication networks, 2017, 2017: 3606424.
[56] ELSHAAFI H, VINYALS M, GRIMALDI I, et al. Secure automated home energy management in multi-agent smart grid architecture[J]. Technology and economics of smart grids and sustainable energy, 2018, 3(1): 4.
[57] HU Y J, TANG Chaowei. Agent-oriented public key infrastructure for multi-agent e-service[C]//Proceedings of the 7th International Conference on Knowledge-Based Intelligent Information and Engineering Systems. Oxford, UK, 2003: 1215-1221.
[58] LU Feng, HUANG Mei. Research and design of security in Multi-agent System[C]//Proceedings of 2016 IET International Conference on Wireless, Mobile and Multimedia Networks. Hangzhou, China, 2006: 1-4.
[59] TEKBACAK F, TUGLULAR T, DIKENELLI O. An architecture for verification of access control policies with multi agent system ontologies[C]//Proceedings of the 2009 33rd Annual IEEE International Computer Software and Applications Conference. Seattle, USA, 2009: 52-55.
[60] VITABILE S, MILICI G, SCOLARO S, et al. A MAS security framework implementing reputation based policies and owners access control[C]//Proceedings of the 20th International Conference on Advanced Information NETWORKING and Applications. Vienna, Austria, 2006: 746-752.
[61] WANG Xuan, YAN Jinglong. Research on security mechanism for Multi-Agent system[C]//Proceedings of the 2011 IEEE 3rd International Conference on Communication Software and Networks. Xi’an, China, 2011: 345-348.
[62] XIAO Liang, HU Bo. Towards adaptive and secure multi-agent systems[C]//Proceedings of the 2007 2nd International Conference on Pervasive Computing and Applications. Birmingham, UK, 2007: 56-61.
[63] ROBLES S, BORRELL J, BIGHAM J, et al. Design of a trust model for a secure multi-agent marketplace[C]//Proceedings of the 5th International Conference on Autonomous agents. Montreal, Canada, 2001: 77-78.
[64] POGGI A, TOMAIUOLO M, VITAGLIONE G. Security and trust in agent-oriented middleware[C]//Proceedings of 2003 on the Move to Meaningful Internet Systems 2003: OTM 2003 Workshops. Sicily, Italy, 2003: 989-1003.
[65] VAN ‘T NOORDENDE G J, BRAZIER F M T, TANENBAUM A S. Security in a mobile agent system[C]//Proceedings of the IEEE 1st Symposium on Multi-Agent Security and Survivability. Drexel, USA, 2004: 35-45.
[66] CHAE C J, CHOI K N, CHOI K. Information interoperability system using multi-agent with security[J]. Wireless personal communications, 2016, 89(3): 819-832.
[67] JONES K I. A trust based approach to mobile multi-agent systems[D]. Leicester: De Montfort University, 2010.
[68] FAGIOLINI A, VALENTI G, PALLOTTINO L, et al. Decentralized intrusion detection for secure cooperative multi-agent systems[C]//Proceedings of the 2007 46th IEEE Conference on Decision and Control. New Orleans, USA, 2007.
[69] SUCH J M, ALBEROLA J M, ESPINOSA A, et al. A group-oriented secure multiagent platform[J]. Software: practice and experience, 2011, 41(11): 1289-1302.
[70] RASHVAND H F, SALAH K, CALERO J M A, et al. Distributed security for multi-agent systems-review and applications[J]. IET information security, 2010, 4(4): 188-201.

相似文献/References:

[1]沈 晶,顾国昌,刘海波.基于多智能体的Option自动生成算法[J].智能系统学报,2006,1(01):84.
 SHEN Jing,GU Guo-chang,LIU Hai-bo.Algorithm for automatic constructing Option based on multi-agent[J].CAAI Transactions on Intelligent Systems,2006,1(3):84.
[2]李宗刚,贾英民.一类具有群体LEADER的多智能体系统的聚集行为[J].智能系统学报,2006,1(02):26.
 LI Zong-gang,JIA Ying-min.Aggregation of MultiAgent systems with group leaders[J].CAAI Transactions on Intelligent Systems,2006,1(3):26.
[3]王建春,谢广明.含有障碍物环境下多智能体系统的聚集行为[J].智能系统学报,2007,2(05):78.
 WANG Jian-chun,XIE Guang-ming.Aggregation behaviors of multiAgent systems in an environment with obstacles[J].CAAI Transactions on Intelligent Systems,2007,2(3):78.
[4]王 龙,伏 锋,陈小杰,等.复杂网络上的群体决策[J].智能系统学报,2008,3(02):95.
 WANG Long,FU Feng,CHEN Xiao-jie,et al.Collective decision-making over complex networks[J].CAAI Transactions on Intelligent Systems,2008,3(3):95.
[5]连传强,徐昕,吴军,等.面向资源分配问题的Q-CF多智能体强化学习[J].智能系统学报,2011,6(02):95.
 LIAN Chuanqiang,XU Xin,WU Jun,et al.Q-CF multiAgent reinforcement learningfor resource allocation problems[J].CAAI Transactions on Intelligent Systems,2011,6(3):95.
[6]王冬梅,方华京.反馈控制策略的自适应群集运动[J].智能系统学报,2011,6(02):141.
 WANG Dongmei,FANG Huajing.An adaptive flocking motion with a leader based on a feedback control scheme[J].CAAI Transactions on Intelligent Systems,2011,6(3):141.
[7]董洁,纪志坚,王晓晓.多智能体网络系统的能控性代数条件[J].智能系统学报,2015,10(5):747.[doi:10.11992/tis.201411030]
 DONG Jie,JI Zhijian,WANG Xiaoxiao.Algebraic conditions for the controllability of multi-agent systems[J].CAAI Transactions on Intelligent Systems,2015,10(3):747.[doi:10.11992/tis.201411030]
[8]王中林,刘忠信,陈增强,等.一种多智能体领航跟随编队新型控制器的设计[J].智能系统学报,2014,9(03):298.[doi:10.3969/j.issn.1673-4785.]
 WANG Zhonglin,LIU Zhongxin,CHEN Zengqiang,et al.A kind of new type controller for multi-agent leader-follower formation[J].CAAI Transactions on Intelligent Systems,2014,9(3):298.[doi:10.3969/j.issn.1673-4785.]
[9]王晓晓,纪志坚.广播信号下非一致多智能体系统的能控性[J].智能系统学报,2014,9(04):401.[doi:10.3969/j.issn.1673-4785.201401011]
 WANG Xiaoxiao,JI Zhijian.Controllability of non-identical multi-agent systems under a broadcasting control signal[J].CAAI Transactions on Intelligent Systems,2014,9(3):401.[doi:10.3969/j.issn.1673-4785.201401011]
[10]马晨,陈雪波.基于包含原理的多智能体一致性协调控制[J].智能系统学报,2014,9(04):468.[doi:10.3969/j.issn.1673-4785.201306024]
 MA Chen,CHEN Xuebo.Coordinated control of the consensus of a multi-agent system based on the inclusion principle[J].CAAI Transactions on Intelligent Systems,2014,9(3):468.[doi:10.3969/j.issn.1673-4785.201306024]

备注/Memo

备注/Memo:
收稿日期:2018-12-13。
基金项目:国家重点研发计划项目(2018YFB0904900)
作者简介:丁俐夫,博士研究生,主要研究方向为多智能体系统、分布式优化算法;颜钢锋,教授,博士生导师,中国系统工程学会理事,主要研究方向为多智能体系统。1999年被评为浙江省有特殊贡献的中青年专家,2000年获国家政府特殊津贴。发表学术论文100余篇
通讯作者:丁俐夫.E-mail:dinglifu@zju.edu.cn
更新日期/Last Update: 1900-01-01