[1]丁俐夫,颜钢锋.多智能体系统安全性问题及防御机制综述[J].智能系统学报,2020,15(3):425-434.[doi:10.11992/tis.201812015]
　DING Lifu,YAN Gangfeng.A survey of the security issues and defense mechanisms of multi-agent systems[J].CAAI Transactions on Intelligent Systems,2020,15(3):425-434.[doi:10.11992/tis.201812015]
点击复制

多智能体系统安全性问题及防御机制综述

《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷: 15 期数: 2020年第3期 页码: 425-434 栏目: 综述 出版日期: 2020-05-05
Title:
A survey of the security issues and defense mechanisms of multi-agent systems
作者:
丁俐夫1, 颜钢锋2
1. 浙江大学 电气工程学院，浙江 杭州 310027;
2. 浙江大学 华南工业技术研究院，广东 广州 510760
Author(s):
DING Lifu1, YAN Gangfeng2
1. College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China;
2. Huanan Industrial Technology Research Institute, Zhejiang University, Guangzhou 510760, China
关键词:
多智能体系统; 分布式人工智能; 安全威胁; 防御机制; 网络安全; 通信协议; 访问控制; 协调机制
Keywords:
multi-agent system; distributed artificial intelligence; security threat; defense mechanism; network security; communication protocol; access control; coordination mechanism
分类号:
TP18;TP309
DOI:
10.11992/tis.201812015
摘要:
多智能体系统作为分布式人工智能的重要分支，已成为解决大型、复杂、分布式及难预测问题的重要手段。在开放网络中，多智能体系统仍面临许多安全问题，潜在的安全威胁很可能影响其实际应用的稳定性、快速性和准确性。基于目前已知的多智能体系统通用模型，介绍了多智能体系统通信协议、访问控制和协调机制中潜在的安全问题，规范了多智能体系统安全性问题的研究体系，总结了系统设计过程中可行的防御技术和隐私保护技术，最后展望了多智能体系统安全研究的发展方向。
Abstract:
Currently, multi-agent systems are an important branch of distributed artificial intelligence and an important means of solving large, complex, distributed, and difficult-to-predict problems. However, in an open network, multi-agent systems still face many security issues. Potential security threats are likely to affect the stability, speed, and accuracy of the practical applications of multi-agent systems. Based on the general models of currently known multi-agent systems, the potential security problems in multi-agent system communication protocol, access control, and coordination mechanism are introduced. Research on multi-agent system security problems is standardized, and feasible defense and privacy protection technologies in the system design process are summarized. Finally, the development trend of multi-agent system security research is analyzed.
参考文献/References:
[1] GREENBERG M S, BYINGTON J C, HARPER D G. Mobile agents and security[J]. IEEE communications magazine, 1998, 36(7): 76-85.
[2] CENTENO R, FAGUNDES M, BILLHARDT H, et al. Supporting medical emergencies by MAS[C]//Proceedings of the 3rd Kes International Symposium on Agent and Multi-Agent Systems: Technologies and Applications. Uppsala, Sweden, 2009: 823-833.
[3] DIOUBATE M, TAN Guanzheng, MOHAMED L T. An artificial immune system based multi-agent model and its application to robot cooperation problem[C]//Proceedings of the 2008 7th World Congress on Intelligent Control and Automation. Chongqing, China, 2008: 3033-3039.
[4] SZYMAK P, PRACZYK T. Control systems of underwater vehicles in multi-agent system of underwater inspection[C]//Proceedings of the 11th WSEAS International Conference on Automatic Control, Modelling and Simulation. Istanbul, Turkey, 2009: 153-156.
[5] HUANG Xiao, ZHANG Qi, WANG Yu. Research on multi-agent traffic signal control system based on VANET information[C]//Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems. Yokohama, Japan, 2018: 1-6.
[6] 姚尧, 卢淑娟, 徐德民. IMMAS算法在复杂环境下路径规划中的应用[J]. 计算机仿真, 2007, 24(12): 148-151
YAO Yao, LU Shujuan, XU Demin. IMMAS for robot path planning in complex environment[J]. Computer simulation, 2007, 24(12): 148-151
[7] IQBAL S, ALTAF W, ASLAM M, et al. Application of intelligent agents in health-care: review[J]. Artificial intelligence review, 2016, 46(1): 83-112.
[8] BIJANI S, ROBERTSON D. A review of attacks and security approaches in open multi-agent systems[J]. Artificial intelligence review, 2014, 42(4): 607-636.
[9] HUYNH T D, JENNINGS N R, SHADBOLT N R. An integrated trust and reputation model for open multi-agent systems[J]. Autonomous agents and multi-agent systems, 2006, 13(2): 119-154.
[10] WANG S, HU J, LIU A, et al. Security frame and evaluation in mobile agent system[C]. International Conference on Mobile Technology Applications and Systems. Guangzhou, China, 2005: 1-6.
[11] JUNG Y, KIM M, MASOUMZADEH A, et al. A survey of security issue in multi-agent systems[J]. Artificial intelligence review, 2012, 37(3): 239-260.
[12] Foundation for Intelligent Physical Agents (FIPA). FIPA Abstract Architecture Specification 2002 [EB/OL]. (2002-02-25)[2018-12-01]http://www.fipa.org/specs/fipaoo001/SC00001L.pdf.
[13] Foundation for Intelligent Physical Agents (FIPA). FIPA ACL Message Structure Specification [EB/OL]. (2000-01-08)[2018-12-01]http//www.fipa.org/specs/fipa00061/SC00061G.html.
[14] Foundation for Intelligent Physical Agents (FIPA). FIPA agent management specification [EB/OL]. (2004-03-18) [2018-12-01]http://www.fipa.org/specs/fipaO0023/SCO0023K.pdf.
[15] MOURATIDIS H, GIORGINI P, MANSON G. Modelling secure multiagent systems[C]//Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multiagent Systems. Melbourne, Australia, 2003: 859-866.
[16] BRESCIANI P, GIORGINI P, MOURATIDIS H, et al. Multi-agent systems and security requirements analysis[M]//LUCENA C, GARCIA A, ROMANOVSKY A, et al. Software Engineering for Multi-Agent Systems II. Berlin, Germany: Springer, 2004: 35-48.
[17] POSLAD S, CHARLTON P, CALISTI M. Specifying standard security mechanisms in multi-agent systems[M]//FALCONE R, BARBER S, KORBA L, et al. Trust, Reputation, and Security: Theories and Practice. Berlin, Germany: Springer, 2003: 163-176.
[18] BRESCIANI P, PERINI A, GIORGINI P, et al. Tropos: an agent-oriented software development methodology[J]. Autonomous agents and multi-agent systems, 2004, 8(3): 203-236.
[19] MOURATIDIS H, GIORGINI P. Secure tropos: a security-oriented extension of the tropos methodology[J]. International journal of software engineering and knowledge engineering, 2007, 17(2): 285-309.
[20] MOURATIDIS H, GIORGINI P. Enhancing secure tropos to effectively deal with security requirements in the development of multiagent systems[M]//BARLEY M, MOURATIDIS H, UNRUH A, et al. Safety and Security in Multiagent Systems. Berlin: Springer, 2009: 8-26.
[21] BEYDOUN G, LOW G, MOURATIDIS H, et al. A security-aware metamodel for multi-agent systems (MAS)[J]. Information and software technology, 2009, 51(5): 832-845.
[22] MORADIAN E. Security of E-commerce software systems[M]//HAKANSSON A, HARTUNG R. Agent and Multi-Agent Systems in Distributed Systems-Digital Economy and E-Commerce. Berlin: Springer, 2013: 95-103.
[23] BASHEER G S, AHMAD M S, TANG A Y C, et al. Certainty, trust and evidence: towards an integrative model of confidence in multi-agent systems[J]. Computers in human behavior, 2015, 45: 307-315.
[24] ZIKRATOV I, MASLENNIKOV O, LEBEDEV I, et al. Dynamic trust management framework for robotic multi-agent systems[C]//Proceedings of the 16th International Conference on Internet of Things, Smart Spaces, and Next Generation Networks and Systems. Petersburg, Russia, 2016.
[25] CHERIBI H, KHOLLADI M K. A security model for complex applications based on normative multi-agents system[C]//Proceedings of the 2015 2nd International Conference on Information Security and Cyber Forensics. Cape Town, South Africa, 2015:41-46.
[26] MAJD E, BALAKRISHNAN V. A trust model for recommender agent systems[J]. Soft computing — a fusion of foundations, methodologies and applications, 2017, 21(2): 417-433.
[27] BüRKLE A, HERTEL A, MüLLER W, et al. Evaluating the security of mobile agent platforms[J]. Autonomous agents and multi-agent systems, 2009, 18(2): 295-311.
[28] VILA X, SCHUSTER A, RIERA A. Security for a multi-agent system based on JADE[J]. Computers & security, 2007, 26(5): 391-400.
[29] VITABILE S, CONTI V, MILITELLO C, et al. An extended JADE-S based framework for developing secure Multi-Agent Systems[J]. Computer standards & interfaces, 2009, 31(5): 913-930.
[30] JANSEN W A. Countermeasures for mobile agent security[J]. Computer communications, 2000, 23(17): 1667-1676.
[31] ROTH V. Secure recording of itineraries through co-operating agents[C]//Proceedings of 1998 European Conference on Object-Oriented Programming. Belgium, Germany, 1998: 297-298.
[32] FARMER W M, GUTTMAN J D, SWARUP V. Security for mobile agents: authentication and state appraisal[C]//Proceedings of the 4th European Symposium on Research in Computer Security. Rome, Italy, 1996: 118-130.
[33] NECULA G C, LEE P. Safe, untrusted agents using proof-carrying code[M]//VIGNA G. Mobile Agents and Security. Berlin: Springer, 1998: 61-91.
[34] PARUCHURI P, TAMBE M, ORDó?EZ F, et al. Security in multiagent systems by policy randomization[C]//Proceedings of the 5th International Joint Conference on Autonomous Agents and Multiagent Systems. Hakodate, Japan, 2006.
[35] OLFATI-SABER R, FAX J A, MURRAY R M. Consensus and cooperation in networked multi-agent systems[J]. Proceedings of the IEEE, 2007, 95(1): 215-233.
[36] HE Wangli, GAO Xiaoyang, ZHONG Weimin, et al. Secure impulsive synchronization control of multi-agent systems under deception attacks[J]. Information sciences, 2018, 459: 354-368.
[37] AMULLEN E M, SHETTY S, KEEL L H. Secured formation control for multi-agent systems under DoS attacks[C]//Proceedings of 2016 IEEE Symposium on Technologies for Homeland Security. Waltham, USA, 2016.
[38] DE LA TORRE G, YUCELEN T, PETERSON J D. Resilient networked multiagent systems: a distributed adaptive control approachy[C]//Proceedings of the 53rd IEEE Conference on Decision and Control. Los Angeles, USA, 2014.
[39] 伍益明. 恶意攻击下的多智能体系统安全一致性问题研究[D]. 杭州: 浙江工业大学, 2016.
WU Yiming. Research on secure consensus for multi-agent systems under malicious attacks[D]. Hangzhou: Zhejiang University of Technology, 2016.
[40] LIU S, XIE L, ZHANG H, et al. Distributed consensus for multi-agent systems with delays and noises in transmission channels[J]. Automatica, 2011, 47(5): 920-934.
[41] WU Yiming, HE Xiongxiong. Secure consensus control for multiagent systems with attacks and communication delays[J]. IEEE/CAA journal of automatica sinica, 2017, 4(1): 136-142.
[42] MENG Deyuan, JIA Yingmin, DU Junping. Finite-time consensus for multiagent systems with cooperative and antagonistic interactions[J]. IEEE transactions on neural networks and learning systems, 2016, 27(4): 762-770.
[43] MENG Deyuan, ZUO Zongyu. Signed-average consensus for networks of agents: a nonlinear fixed-time convergence protocol[J]. Nonlinear dynamics, 2016, 85(1): 155-165.
[44] LIU Xiaoyang, LAM J, YU Wenwu, et al. Finite-time consensus of multiagent systems with a switching protocol[J]. IEEE transactions on neural networks and learning systems, 2016, 27(4): 853-862.
[45] LI Jinsha, LI Junmin. Adaptive iterative learning control for coordination of second-order multi-agent systems[J]. International journal of robust and nonlinear control, 2014, 24(18): 3282-3299.
[46] WU Yiming, XU Ming, ZHENG Ning, et al. Attack tolerant finite-time consensus for multi-agent networks[C]//Proceedings of the 2017 13th IEEE International Conference on Control & Automation. Ohrid, Macedonia, 2017.
[47] WU Yiming, HE Xiongxiong, LIU Shuai. Resilient consensus for multi-agent systems with quantized communication[C]//Proceedings of 2016 American Control Conference. Boston, USA, 2016.
[48] FENG Zhi, HU Guoqiang, WEN Guanghui. Distributed consensus tracking for multi-agent systems under two types of attacks[J]. International journal of robust and nonlinear control, 2016, 26(5): 896-918.
[49] FENG Zhi, WEN Guanghui, HU Guoqiang. Distributed secure coordinated control for multiagent systems under strategic attacks[J]. IEEE transactions on cybernetics, 2017, 47(5): 1273-1284.
[50] FENG Zhi, HU Guoqiang. Distributed secure leader-following consensus of multi-agent systems under DoS attacks and directed topology[C]//Proceedings of 2017 IEEE International Conference on Information and Automation. Macau, China, 2017.
[51] PITT J, MAMDANI A. Communication protocols in multi-agent systems: a development method and reference architecture[M]//DIGNUM F, GREAVES M. Issues in Agent Communication. Berlin: Springer, 2000: 160-177.
[52] YOKOO M, SUZUKI K. Secure multi-agent dynamic programming based on homomorphic encryption and its application to combinatorial auctions[C]//Proceedings of the 1st International Joint Conference on Autonomous Agents and Multiagent Systems. Bologna, Italy, 2002.
[53] ZHU Liehuang, CAO Yuanda, LIAO Lejian, et al. SKQML: a secure multi-agent communication language [C]//Proceedings of the 5th WSEAS International Conference on Applied Computer Science. Hangzhou, China, 2006.
[54] ABDENNADHER S, ABDALLAH M, MUELLER H J. DCP-MASR: a dynamic communication protocol for robot-based multi-agent systems[C]//Proceedings of 2010 5th International Conference on Systems. Menuires, France, 2010: 12-17.
[55] SHEHADA D, YEUN C Y, ZEMERLY M J, et al. BROSMAP: a novel broadcast based secure mobile agent protocol for distributed service applications[J]. Security and communication networks, 2017, 2017: 3606424.
[56] ELSHAAFI H, VINYALS M, GRIMALDI I, et al. Secure automated home energy management in multi-agent smart grid architecture[J]. Technology and economics of smart grids and sustainable energy, 2018, 3(1): 4.
[57] HU Y J, TANG Chaowei. Agent-oriented public key infrastructure for multi-agent e-service[C]//Proceedings of the 7th International Conference on Knowledge-Based Intelligent Information and Engineering Systems. Oxford, UK, 2003: 1215-1221.
[58] LU Feng, HUANG Mei. Research and design of security in Multi-agent System[C]//Proceedings of 2016 IET International Conference on Wireless, Mobile and Multimedia Networks. Hangzhou, China, 2006: 1-4.
[59] TEKBACAK F, TUGLULAR T, DIKENELLI O. An architecture for verification of access control policies with multi agent system ontologies[C]//Proceedings of the 2009 33rd Annual IEEE International Computer Software and Applications Conference. Seattle, USA, 2009: 52-55.
[60] VITABILE S, MILICI G, SCOLARO S, et al. A MAS security framework implementing reputation based policies and owners access control[C]//Proceedings of the 20th International Conference on Advanced Information NETWORKING and Applications. Vienna, Austria, 2006: 746-752.
[61] WANG Xuan, YAN Jinglong. Research on security mechanism for Multi-Agent system[C]//Proceedings of the 2011 IEEE 3rd International Conference on Communication Software and Networks. Xi’an, China, 2011: 345-348.
[62] XIAO Liang, HU Bo. Towards adaptive and secure multi-agent systems[C]//Proceedings of the 2007 2nd International Conference on Pervasive Computing and Applications. Birmingham, UK, 2007: 56-61.
[63] ROBLES S, BORRELL J, BIGHAM J, et al. Design of a trust model for a secure multi-agent marketplace[C]//Proceedings of the 5th International Conference on Autonomous agents. Montreal, Canada, 2001: 77-78.
[64] POGGI A, TOMAIUOLO M, VITAGLIONE G. Security and trust in agent-oriented middleware[C]//Proceedings of 2003 on the Move to Meaningful Internet Systems 2003: OTM 2003 Workshops. Sicily, Italy, 2003: 989-1003.
[65] VAN ‘T NOORDENDE G J, BRAZIER F M T, TANENBAUM A S. Security in a mobile agent system[C]//Proceedings of the IEEE 1st Symposium on Multi-Agent Security and Survivability. Drexel, USA, 2004: 35-45.
[66] CHAE C J, CHOI K N, CHOI K. Information interoperability system using multi-agent with security[J]. Wireless personal communications, 2016, 89(3): 819-832.
[67] JONES K I. A trust based approach to mobile multi-agent systems[D]. Leicester: De Montfort University, 2010.
[68] FAGIOLINI A, VALENTI G, PALLOTTINO L, et al. Decentralized intrusion detection for secure cooperative multi-agent systems[C]//Proceedings of the 2007 46th IEEE Conference on Decision and Control. New Orleans, USA, 2007.
[69] SUCH J M, ALBEROLA J M, ESPINOSA A, et al. A group-oriented secure multiagent platform[J]. Software: practice and experience, 2011, 41(11): 1289-1302.
[70] RASHVAND H F, SALAH K, CALERO J M A, et al. Distributed security for multi-agent systems-review and applications[J]. IET information security, 2010, 4(4): 188-201.
相似文献/References:
[1]沈 晶,顾国昌,刘海波.基于多智能体的Option自动生成算法[J].智能系统学报,2006,1(1):84.
　SHEN Jing,GU Guo-chang,LIU Hai-bo.Algorithm for automatic constructing Option based on multi-agent[J].CAAI Transactions on Intelligent Systems,2006,1():84.
[2]李宗刚,贾英民.一类具有群体LEADER的多智能体系统的聚集行为[J].智能系统学报,2006,1(2):26.
　LI Zong-gang,JIA Ying-min.Aggregation of MultiAgent systems with group leaders[J].CAAI Transactions on Intelligent Systems,2006,1():26.
[3]王建春,谢广明.含有障碍物环境下多智能体系统的聚集行为[J].智能系统学报,2007,2(5):78.
　WANG Jian-chun,XIE Guang-ming.Aggregation behaviors of multiAgent systems in an environment with obstacles[J].CAAI Transactions on Intelligent Systems,2007,2():78.
[4]王 龙,伏 锋,陈小杰,等.复杂网络上的群体决策[J].智能系统学报,2008,3(2):95.
　WANG Long,FU Feng,CHEN Xiao-jie,et al.Collective decision-making over complex networks[J].CAAI Transactions on Intelligent Systems,2008,3():95.
[5]连传强,徐昕,吴军,等.面向资源分配问题的Q-CF多智能体强化学习[J].智能系统学报,2011,6(2):95.
　LIAN Chuanqiang,XU Xin,WU Jun,et al.Q-CF multiAgent reinforcement learningfor resource allocation problems[J].CAAI Transactions on Intelligent Systems,2011,6():95.
[6]王冬梅,方华京.反馈控制策略的自适应群集运动[J].智能系统学报,2011,6(2):141.
　WANG Dongmei,FANG Huajing.An adaptive flocking motion with a leader based on a feedback control scheme[J].CAAI Transactions on Intelligent Systems,2011,6():141.
[7]董洁,纪志坚,王晓晓.多智能体网络系统的能控性代数条件[J].智能系统学报,2015,10(5):747.[doi:10.11992/tis.201411030]
　DONG Jie,JI Zhijian,WANG Xiaoxiao.Algebraic conditions for the controllability of multi-agent systems[J].CAAI Transactions on Intelligent Systems,2015,10():747.[doi:10.11992/tis.201411030]
[8]王中林,刘忠信,陈增强,等.一种多智能体领航跟随编队新型控制器的设计[J].智能系统学报,2014,9(3):298.[doi:10.3969/j.issn.1673-4785.]
　WANG Zhonglin,LIU Zhongxin,CHEN Zengqiang,et al.A kind of new type controller for multi-agent leader-follower formation[J].CAAI Transactions on Intelligent Systems,2014,9():298.[doi:10.3969/j.issn.1673-4785.]
[9]王晓晓,纪志坚.广播信号下非一致多智能体系统的能控性[J].智能系统学报,2014,9(4):401.[doi:10.3969/j.issn.1673-4785.201401011]
　WANG Xiaoxiao,JI Zhijian.Controllability of non-identical multi-agent systems under a broadcasting control signal[J].CAAI Transactions on Intelligent Systems,2014,9():401.[doi:10.3969/j.issn.1673-4785.201401011]
[10]马晨,陈雪波.基于包含原理的多智能体一致性协调控制[J].智能系统学报,2014,9(4):468.[doi:10.3969/j.issn.1673-4785.201306024]
　MA Chen,CHEN Xuebo.Coordinated control of the consensus of a multi-agent system based on the inclusion principle[J].CAAI Transactions on Intelligent Systems,2014,9():468.[doi:10.3969/j.issn.1673-4785.201306024]

备注/Memo

收稿日期:2018-12-13。
基金项目:国家重点研发计划项目(2018YFB0904900)
作者简介:丁俐夫，博士研究生，主要研究方向为多智能体系统、分布式优化算法;颜钢锋，教授，博士生导师，中国系统工程学会理事，主要研究方向为多智能体系统。发表学术论文100余篇.
通讯作者:丁俐夫.E-mail:dinglifu@zju.edu.cn

更新日期/Last Update: 1900-01-01
Copyright © 《 智能系统学报》 编辑部
地址：（150001）黑龙江省哈尔滨市南岗区南通大街145-1号楼 电话：0451- 82534001、82518134　邮箱：tis@vip.sina.com