第三期
威胁三:多智能体与协议层威胁
总的来说,多智能体系统引入了源于标准化通信、互操作性和分布式任务执行需求的独特攻击向量。与威胁主要局限于提示注入或不安全工具使用的单智能体场景不同,多智能体生态系统通过协议介导的交互放大了风险。消息篡改、角色伪造和协议利用为攻击者创造了不仅攻陷单个智能体,而且破坏整个协同工作流的机会[26][79]。
3.1协议层 MCP 与 A2A 攻击
近期几项研究调查了新兴协议层标准所带来的安全威胁。值得注意的是,Ferrag等人[80]研究了流行的模型上下文协议(Model Context Protocol, MCP)[81]和智能体间(Agent-to-Agent, A2A)协议[82]框架中的漏洞,以及更广泛的协议家族,如智能体网络协议(Agent Network Protocol, ANP)[83]和智能体通信协议(Agent Communication Protocol, ACP)[84]。鉴于MCP和A2A在实现智能体-工具集成和多智能体生态系统中的突出地位和普及性,我们将下面的讨论限定于这两类威胁。
MCP引发的攻击:MCP引发的攻击利用了模型上下文协议自身的设计,该协议采用客户端-服务器架构将语言模型与文件系统、API或数据库等外部资源连接起来[81]。通过将访问与模型逻辑分离,MCP扩展了智能体的功能,但同时也暴露了多个协议层面的攻击向量。
最常见的攻击是泛洪攻击和重放攻击,攻击者利用请求泛洪或无限循环破坏操作,导致拒绝服务(Denial of Service, DoS)[85]。一种相邻但不同的攻击包括凭据泄露,这通过不安全的代理或泄露的令牌发生,并使攻击者能够冒充智能体[80]。
其他MCP攻击源于智能体与MCP中介资源的交互方式,而不仅仅是协议结构本身。例如,嵌入式后门是隐藏在提示或模型参数中的恶意触发器,它们滥用MCP支持的工具访问权限[80][86]。其他关于检索污染和联邦训练操纵的研究,探讨了攻击者如何扭曲检索增强生成(Retrieval-Augmented Generation, RAG)流程中的上下文输入[87],或在联邦学习设置中注入恶意更新以污染分布式训练数据[88]。此外,机密性破坏是指通过MCP接口暴露的敏感上下文数据所引发的推断攻击和侧信道计时攻击[89][90]。
A2A引发的攻击:A2A引发的攻击利用了智能体间(A2A)协议中的漏洞,该协议通过JSON交换来协调MCP、能力识别和任务委托等交互[82]。A2A的可扩展设计引入了几个协议特有的威胁,例如虚假智能体广播和未经授权的注册,这些允许攻击者冒充智能体或接管被委托的任务[91],以及递归式DoS攻击,由重复的任务委托触发死锁或无限循环而引起[92]。
A2A框架下其他由智能体引发的攻击利用了智能体间通信的动态特性,例如传递性提示注入(transitive prompt injection),即有害输入通过互连的智能体工作流传播[47];上下文篡改(context tampering),涉及操纵载荷以误导执行或泄露敏感信息[80];以及记忆操纵(memory manipulation),即恶意输入破坏AI智能体的内部状态或持久记忆[93]。
这些智能体协议中的漏洞凸显了多智能体生态系统中协议层面假设的脆弱性。通过破坏发现、认证或任务编排,攻击者可以从局部操纵升级为系统级的入侵。这些风险强调了进行协议加固、实施安全的智能体身份管理以及部署强健的监控机制的必要性,以减轻多AI智能体系统中的级联失效。
3.2威胁行为分类
身份伪造与角色滥用:攻击者可以利用缺乏集中式身份和信任管理的现状,假扮虚假角色或覆盖既定的权限。通过伪造智能体的身份或伪装成受信任的协作者,攻击者得以进入原本受限的工作流[94][95]。一旦嵌入,恶意智能体可以相互勾结形成隐藏的共识,放大其影响力直至合法的防护措施失效[96][97]。跨领域/组织(例如,相互竞争的企业利益)的激励冲突进一步为对抗性智能体提供了掩护,使其能够在组织目标的伪装下隐藏真实意图[79]。
协调操纵:跨领域/组织系统将智能体动态分组为面向特定任务的团队,通常未经事先审查[98]。虽然这种能力提高了系统的适应性和效率,但它也削弱了清晰的信任边界。恶意的或未经验证的智能体可以在运行时作为特定任务团队的一部分被引入[99][100][101][102]。递归委托放大了攻击效果:一个被攻陷的智能体可以将子任务卸载给其他智能体,将对抗性影响扩散到工作流的更深处[79]。与静态攻击(如传统恶意软件)不同,这些攻击利用了多智能体系统本身的框架[103][104]。
知识与学习操纵:跨域多智能体LLM允许智能体通过共享学习和分布式微调进行自我改进[79][105]。在缺乏统一监督的情况下,攻击者可以巧妙地操纵一个智能体的奖励信号,导致行为失调在智能体间传播[104][106]。正反馈循环可能放大不安全的目标,导致策略漂移和权限越界。与提示或记忆注入不同,这种攻击利用了学习过程本身,并且可能在跨域场景中保持不被发现[79]。
推理与策略规避:联邦式多智能体LLM系统正在变革企业运营,它使得能够访问一个组织中敏感数据的智能体可以与由外部合作伙伴管理的智能体进行交互。然而,一旦信息跨越组织边界流动,即使内部策略(例如,“不披露个人薪资”或“仅共享聚合统计数据”)仍然有效,也没有哪个单一实体能保持对交互的完全监督。这使得威胁行为者能够执行跨智能体推理(cross-agent inference),通过对多个单独看似无害的部分输出进行推理,重构出敏感数据。传统的安全机制假设单个智能体内的提示具有完全可见性[26][95],但在联邦设置中,上下文被分散到多个智能体及其各自的提示中。这种碎片化带来了策略规避的机会。例如,攻击者可能从一个智能体请求某个部门的最高薪资,然后向另一个智能体询问获得该薪资的人员姓名,从而有效地重构出受限制的信息。类似地,向不同智能体发出的独立查询可以有策略地组合起来,以完成原本被禁止的请求。传统的防御措施,如静态关键词过滤和基于角色的访问控制[107],无法捕捉这些分布式推理攻击。零知识证明[108]或差分隐私[109]等技术在应对自然语言交互的动态性和组合性方面也面临局限性[110]。尽管近期研究表明,多轮对话攻击可以规避在孤立环境中成功的保护措施,但大多数企业级工具仍然基于独立假设来处理智能体交互[111]。
责任追溯混淆:从威胁行为者的角度来看,跨越多组织领域的多智能体系统为隐藏恶意行为的责任提供了重要机会。每个领域通常执行独立的日志记录、留存和审计策略,从而阻止了统一的活动追踪[79]。一旦输入数据被LLM处理,它就会被转换为分布式的潜在表示,从而消除了能够将行动与其来源关联起来的持久标识符[112]。与可以实现污染检查或显式信息流追踪的传统软件系统[113]不同,这些潜在表示使得追踪变得不可行[114]。攻击者可以通过攻陷领域A中的一个智能体、注入具有欺骗性但看似合理的指令,并间接触发领域B中的有害行为来利用这一点,同时隐藏其身份和意图[115][116][117]。即使在智能体层面进行审计,智能体间的关系和跨域的因果关系仍然隐藏,这使得责任追溯变得极其复杂。当前的可解释性工具,如影响函数[118][119]、激活追踪[120]或模型来源属性查询[121][122][123],提供的可见性有限,并且常常在这些复杂的多智能体环境中失效。
机密数据篡改/泄露:威胁行为者还可以针对多智能体系统的密码学和工作流限制进行攻击。即使在输入和输出保持加密的隐私保护部署中[124][125][126],攻击者也可能利用跨域完全明文可见性的缺失。例如,在云托管的医疗影像处理流程中,加密的患者扫描由不同供应商管理的多个智能体处理,中间输出从不暴露[127]。虽然这种设计减少了直接暴露的风险,但攻击者仍可能试图修改解密后的结果或干扰中间计算,以注入有害的结果。与具有集中式日志记录和签名的本地系统不同,分布式加密工作流使所有参与者都暴露在攻击之下。诸如全同态加密[128]、多方计算[129]和零知识证明[130]等技术提供了部分缓解措施,但它们计算成本高昂,并且不能完全防止大规模、多域智能体网络中的操纵行为。因此,机密性和完整性方面的漏洞仍然是真实世界多智能体部署中攻击者可以利用的途径。
向上滑动查看参考文献
[1]. How Llms Work. Ai What Do Large Language Models "Understand"? Image, 21:1, 2024.
[2]. Andrei Kucharavy. Fundamental Limitations Of Generative Llms. In Large Language Models In Cybersecurity: Threats, Exposure And Mitigation, Pages 55-64. Springer Nature Switzerland Cham, 2024.
[3]. Thomas Kwa, Ben West, Joel Becker, Amy Deng, Kathryn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx, Et Al. Measuring Ai Ability To Complete Long Tasks. Arxiv Preprint Arxiv:2503.14499, 2025.
[4]. Palo Alto Networks (Unit 42). Ai Agents Are Here. So Are The Threats., 2025.
[5]. Langchain. Langchain Documentation. Https://Python.Langchain.Com/, 2024.
[6]. Toran Bruce Richards. Autogpt: An Autonomous Gpt Experiment. Https://Github.Com/Torantulino/Auto-Gpt, 2024.
[7]. Openclaw. Https://Openclaw.Im
[8]. Guanzhi Wang et al. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023.
[9]. Paolo Dal Cin, Daniel Kendzior, Yusof Seedat, and Renato Marinho. Three essentials for agentic ai security. MIT Sloan Management Review (Online), pages 1-4, 2025.
[10]. Reuters. Just in time? manufacturers turn to ai to weather tariff storm, 2025. URL https://www.reuters.com/business/just-time-manufacturers-turn-ai-weather-tariff-storm-2025-08-13/.
[11]. Wired. Forget chatbots. ai agents are the future, 2025. URL https://www.wired.com/story/fast-forward-forget-chatbots-ai-agents-are-the-future/. Accessed: 2025-08-16.
[12]. Joon Sung Park et al. Generative agents: Interactive simulacra of human behavior. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), 2023.
[13]. Shanghua Gao, Ada Fang, Yepeng Huang, Valentina Giunchiglia, Ayush Noori, Jonathan Richard Schwarz, Yasha Ektefaie, Jovana Kondic, and Marinka Zitnik. Empowering biomedical discovery with ai agents. Cell, 187 (22):6125-6151, 2024.
[14]. Mourad Gridach, Jay Nanavati, Khaldoun Zine El Abidine, Lenon Mendes, and Christina Mack. Agentic ai for scientific discovery: A survey of progress, challenges, and future directions. arXiv preprint arXiv:2503.08979, 2025.
[15]. Verge. Inside the automated warehouse where robots are packing your groceries, 2025. URL https://www.theverge.com/robot/719880/ocado-online-grocery-automation-krogers-luton-ogrp-robot-grid. Accessed: 2025-08-16.
[16]. Zihan Chen, Yixin Wu, et al. Autoagents: A framework for automatic agent generation. arXiv preprint arXiv:2309.17288, 2023. URL https://arxiv.org/abs/2309.17288.
[17]. Reuters. Amazon's delivery, logistics get ai boost, 2025. URL https://www.reuters.com/business/retail-consumer/amazons-delivery-logistics-will-get-an-ai-boost-2025-06-04/. Accessed: 2025-08-16.
[18]. Subash Neupane, Sudip Mittal, and Shahram Rahimi. Towards a hipaa compliant agentic ai system in healthcare. arXiv preprint arXiv:2504.17669, 2025.
[19]. Ken Huang. Ai agents in healthcare. In Agentic AI: Theories and Practices, pages 303-321. Springer, 2025.
[20]. Nalan Karunanayake. Next-generation agentic ai for transforming healthcare. Informatics and Health, 2(2): 73-83, 2025.
[21]. Michael Moritz, Eric Topol, and Pranav Rajpurkar. Coordinated ai agents for advancing healthcare. Nature Biomedical Engineering, pages 1-7, 2025.
[22]. James Zou and Eric J Topol. The rise of agentic ai teammates in medicine. The Lancet, 405(10477):457, 2025.
[23]. Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you've signed up for: Compromising real-world llm-integrated Applications with indirect prompt injection. Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 2023. URL https://api.semanticscholar.org/CorpusID:258546941.
[24]. Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. ArXiv, abs/2211.09527, 2022. URL https://api.semanticscholar.org/CorpusID:253581710.
[25]. Luca Beurer-Kellner, Beat Buesser, Ana-Maria Cre?tu, Edoardo Debenedetti, Daniel Dobos, Daniel Fabian, Marc Fischer, David Froelicher, Kathrin Grosse, Daniel Naeff, et al. Design Patterns for Securing LLM Agents against Prompt Injections. arXiv preprint arXiv:2506.08837, 2025.
[26]. Donghyun Lee and Mo Tiwari. Prompt infection: Llm-to-llm prompt injection within multi-agent systems. arXiv preprint arXiv:2410.07283, 2024.
[27]. OWASP GenAI Project. Owasp genai llm01: Prompt injection, 2025.
[28]. Jeremy McHugh, Kristina Sekrst, and Jonathan Rodriguez Cefalu. Prompt injection 2.0: Hybrid ai threats. ArXiv, abs/2507.13169, 2025. URL https://api.semanticscholar.org/CorpusID:280296803.
[29]. Yulin Chen, Haoran Li, Yuan Sui, Yufei He, Yue Liu, Yangqiu Song, and Bryan Hooi. Can indirect prompt injection attacks be detected and removed? arXiv preprint arXiv:2502.16580, 2025.
[30]. Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. InjeAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 10471-10506, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi:10.18653/v1/2024. findings-acl.624. URL https://aclanthology.org/2024. findings-acl.624/.
[31]. Qiusi Zhan, Richard Fang, Henil Shalin Panchal, and Daniel Kang. Adaptive attacks break defenses against indirect prompt injection attacks on llm agents. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 7101–7117, 2025.
[32]. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023. URL https://arxiv.org/abs/2307.15043.
[33]. Xiaogeng Liu, Zhiyuan Yu, Yizhe Zhang, Ning Zhang, and Chaowei Xiao. Automatic and universal prompt injection attacks against large language models. arXiv preprint arXiv:2403.04957, 2024.
[34]. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614, 2023.
[35]. Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. Autodan: interpretable gradient-based adversarial attacks on large language models. arXiv preprint arXiv:2310.15140, 2023.
[36]. Zexuan Zhong, Ziqing Huang, Alexander Wettig, and Danqi Chen. Poisoning retrieval corpora by injecting adversarial passages. arXiv preprint arXiv:2310.19156, 2023.
[37]. Zeyi Liao, Lingbo Mo, Chejian Xu, Mintong Kang, Jiawei Zhang, Chaowei Xiao, Yuan Tian, Bo Li, and Huan Sun. Eia: Environmental injection attack on generalist web agents for privacy leakage. arXiv preprint arXiv:2409.11295, 2024.
[38]. Chejian Xu, Mintong Kang, Jiawei Zhang, Zeyi Liao, Lingbo Mo, Mengqi Yuan, Huan Sun, and Bo Li. Advagent: Controllable blackbox red-teaming on web agents. arXiv preprint arXiv:2410.17401, 2024.
[39]. Chen Henry Wu, Rishi Shah, Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried, and Aditi Raghunathan. Dissecting adversarial robustness of multimodal lm agents. arXiv preprint arXiv:2406.12814, 2024.
[40]. Kaijie Zhu, Xianjun Yang, Jindong Wang, Wenbo Guo, and William Yang Wang. Melon: Provable defense against indirect prompt injection attacks in ai agents. arXiv preprint arXiv:2502.05174, 2025.
[41]. Yanzhe Zhang, Tao Yu, and Diyi Yang. Attacking vision-language computer agents via pop-ups. arXiv preprint arXiv:2411.02391, 2024.
[42]. Sam Johnson, Viet Pham, and Thai Le. Manipulating llm web agents with indirect prompt injection attack via html accessibility tree. arXiv preprint arXiv:2507.14799, 2025.
[43]. Junhyuk Choi, Yeseon Hong, Minju Kim, and Bugeun Kim. Examining identity drift in conversations of llm agents, 2025. URL https://arxiv.org/abs/2412.00804.
[44]. Jiawei Guo and Haipeng Cai. System prompt poisoning: Persistent attacks on large language models beyond user injection. arXiv preprint, 2025. URL https://arxiv.org/abs/2505.06493.
[45]. Quan Zhang, Binqi Zeng, Chijin Zhou, Gwhwan Go, Heyuan Shi, and Yu Jiang. Human-imperceptible retrieval poisoning attacks in llm-powered applications. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering, pages 502-506, 2024.
[46]. Cody Clop and Yannick Teglia. Backdoored retrievers for prompt injection attacks on retrieval augmented generation of large language models. arXiv preprint arXiv:2410.14479, 2024.
[47]. Le Wang, Zonghao Ying, Tianyuan Zhang, Siyuan Liang, Shengshan Hu, Mingchuan Zhang, Aishan Liu, and Xianglong Liu. Manipulating multimodal agents via cross-modal prompt injection. arXiv preprint arXiv:2504.14348, 2025.
[48]. Sean Park. Unveiling ai agent vulnerabilities part ii: Code execution. Trend Micro Research Report, 2025. URL https://www.trendmicro.com/vinfo/br/security/news/cybercrime-and-digital-threats/unveiling-ai-agent-vulnerabilities-code-execution.
[49]. Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi, and Vitaly Shmatikov. Abusing images and sounds for indirect instruction injection in multi-modal llms. arXiv preprint arXiv:2307.10490, 2023.
[50]. Rodrigo Pedro, Daniel Castro, Paulo Carreira, and Nuno Santos. From prompt injections to sql injection attacks: How protected is your llm-integrated web application? arXiv preprint arXiv:2308.01990, 2023.
[51]. Richard Fang, Rohan Bindu, Akul Gupta, Qiushi Zhan, and Daniel Kang. LLM agents can autonomously hack websites. arXiv preprint arXiv:2402.06664, 2024.
[52]. Rodrigo Pedro, Miguel E. Coimbra, Daniel Castro, Paulo Carreira, and Nuno Santos. Prompt-to-sql injections in llm-integrated web applications: Risks and defenses. 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE), pages 1768-1780, 2025. URL https://api.semanticscholar.org/CorpusID:272856332.
[53]. MITRE Corporation. CVE-2024-5565: Vanna.AI Remote Code Execution Vulnerability, 2024. URL https://cve.org/CVErecord?id=CVe-2024-5565.
[54]. Anshuman Chhabra, Kartik Patwari, Chandana Kuntala, Deepak Kumar Sharma, Prasant Mohapatra, et al. Towards fair video summarization. Transactions on Machine Learning Research, 2023.
[55]. Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. Transferable adversarial attacks for image and video object detection. arXiv preprint arXiv:1811.12641, 2018.
[56]. Zhipeng Wei, Jingjing Chen, Xingxing Wei, Linxi Jiang, Tat-Seng Chua, Fengfeng Zhou, and Yu-Gang Jiang. Heuristic black-box adversarial attacks on video recognition models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 12338-12345, 2020.
[57]. Linxi Jiang, Xingjun Ma, Shaoxiang Chen, James Bailey, and Yu-Gang Jiang. Black-box adversarial attacks on video recognition models. In Proceedings of the 27th ACM International Conference on Multimedia, pages 864-872, 2019.
[58]. Guangke Chen, Fu Song, Zhe Zhao, Xiaojun Jia, Yang Liu, Yanchen Qiao, and Weizhe Zhang. Audiojailbreak: Jailbreak attacks against end-to-end large audio-language models. arXiv preprint arXiv:2505.14103, 2025.
[59]. Eugene Bagdasaryan, Rishi Jha, Vitaly Shmatikov, and Tingwei Zhang. Adversarial illusions in multi-modal embeddings. In 33rd USENIX Security Symposium (USENIX Security 24), pages 3009-3025, 2024.
[60]. Lukas Aichberger, Alasdair Paren, Yarin Gal, Philip Torr, and Adel Bibi. Attacking multimodal os agents with malicious image patches, 2025. URL https://arxiv.org/abs/2503.10809.
[61]. Cristian Pinzon, Juan F. De Paz, Javier Bajo, Alvaro Herrero, and Emilio Corchado. Aida-sql: An adaptive intelligent intrusion detector agent for detecting sql injection attacks. In 2010 10th International Conference on Hybrid Intelligent Systems, pages 73-78, 2010. doi:10.1109/HIS.2010.5600026.
[62]. Johann Rehberger. Deepseek ai: From prompt injection to account takeover, 2024. URL https://embracehered.com/blog/posts/2024/deepseek-ai-prompt-injection-to-xss-and-account-takeover/.
[63]. Sander Schulhoff, Jeremy Pinto, Anam Khan, L-F Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Liu Kost, Christopher Carnahan, and Jordan Boyd-Graber. Ignore this title and hackaprompt: Exposing systemic vulnerabilities of llms through a global scale prompt hacking competition. In The 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2023.
[64]. Stav Cohen, Ron Bitton, and Ben Nassi. Here comes the ai worm: Unleashing zero-click worms that target genai-powered applications. arXiv preprint arXiv:2403.02817, 2024.
[65]. Diego Gosmar, Deborah A Dahl, and Dario Gosmar. Prompt injection detection and mitigation via ai multi-agent nlp frameworks. arXiv preprint arXiv:2503.11517, 2025.
[66]. Sippo Rossi, Alisia Marianne Michel, Raghava Rao Mukkamala, and Jason Bennett Thatcher. An early categorization of prompt injection attacks on large language models, 2024. URL https://arxiv.org/abs/2402.00898.
[67]. U.S. AI Safety Institute. Technical blog: Strengthening ai agent hijacking evaluations. https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations, January 2025. Accessed: January 29, 2025.
[68]. Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. Llm agents can autonomously exploit one-day vulnerabilities. arXiv preprint arXiv:2404.08144, 2024.
[69]. Simon Bennetts. Owasp zed attack proxy. AppSec USA, 2013.
[70]. David Kennedy, Jim O'gorman, Devon Kearns, and Mati Aharoni. Metasploit: the penetration tester's guide. No Starch Press, 2011.
[71]. Nathalie Muehlberger. Csrf and xss: Practical examples using burp suite. Seminararbeit, Ausgew?hlte Kapitel der IT-Security, 2020. URL https://wiki.elvis.science/images/b/b3/Thesis.pdf. Accessed: 2026-02-05.
[72]. Rushi Mamtora, DP Sharma, and Jatin Patel. Server-side template injection with custom exploit. International Journal of Scientific Research in Science, Engineering and Technology, 2021.
[73]. Jean Rosemond Dora, Ladislav Hluchy, and Karol Nemoga. Ontology for blind sql injection. Computing and Informatics, 42(2):480-500, 2023.
[74]. Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, and Zhaochun Ren. Learning to use tools via cooperative and interactive agents. arXiv preprint arXiv:2403.03031, 2024.
[75]. Renxi Wang, Xudong Han, Lei Ji, Shu Wang, Timothy Baldwin, and Haonan Li. Toolgen: Unified tool retrieval and calling via generation. arXiv preprint arXiv:2410.03439, 2024.
[76]. Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClaw,https://arxiv.org/html/2603.12644
[77]. Trojan's Whisper: Stealthy Manipulation of OpenClaw through Injected Bootstrapped Guidance,https://browse-export.arxiv.org/abs/2603.19974
[78]. OpenClaw Vulnerability: Website-to-Local Agent Takeover,https://www.oasis.security/blog/openclaw-vulnerability
[79]. Ronny Ko, Jiseong Jeong, Shuyuan Zheng, Chuan Xiao, Tae-Wan Kim, Makoto Onizuka, and Won-Yong Shin. Seven security challenges that must be solved in cross-domain multi-agent llm systems. arXiv preprint arXiv:2505.23847, 2025.
[80]. Mohamed Amine Ferrag, Norbert Tihanyi, Djallel Hamouda, Leandros Maglaras, and Merouane Debbah. From prompt injections to protocol exploits: Threats in llm-powered ai agents workflows. arXiv preprint arXiv:2506.23260, 2025.
[81]. Introduction to mcp, 2025. URL https://modelcontextprotocol.io/introduction. Accessed: 2025-06-04.
[82]. Rao Surapaneni, Miku Jha, Michael Vakoc, and Todd Segal. A2a: A new era of agent interoperability, April 2025. URL https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/. Accessed: 2025-06-04.
[83]. Gaowei Chang. Agentnetworkprotocol (anp) github repository. https://github.com/agent-network-protocol/AgentNetworkProtocol, 2024. Accessed: 2025-06-04.
[84]. Agent communication protocol: Welcome, 2024. URL https://agentcommunicationprotocol.dev/introduction/welcome. Accessed: 2025-06-04.
[85]. Saman Taghavi Zargar, James Joshi, and David Tipper. A survey of defense mechanisms against distributed denial of service (ddos) flooding attacks. IEEE communications surveys & tutorials, 15(4):2046-2069, 2013.
[86]. Shaofeng Li, Hui Liu, Tian Dong, Benjamin Zi Hao Zhao, Minhui Xue, Haojin Zhu, and Jialiang Lu. Hidden backdoors in human-centric language models. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 3123-3140, 2021.
[87]. Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. Poisonedrag: Knowledge poisoning attacks to retrieval-augmented generation of large language models. ArXiv, abs/2402.07867, 2024. URL https://api.semanticscholar.org/CorpusID:267626957.
[88]. Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, and Ling Liu. Data poisoning attacks against federated learning systems. In European symposium on research in computer security, pages 480-501. Springer, 2020.
[89]. Shubhi Shukla, Manaar Alam, Sarani Bhattacharya, Pabitra Mitra, and Dehdeep Mukhopadhyay. "whispering mlaas": Exploiting timing channels to compromise user privacy in deep neural networks. IACR Transactions on Cryptographic Hardware and Embedded Systems, pages 587-613, 2023.
[90]. Edoardo Debenedetti, Giorgio Severi, Nicholas Carlini, Christopher A Choquette-Choo, Matthew Jagielski, Milad Nasr, Eric Wallace, and Florian Tramer. Privacy side channels in machine learning systems. In 33rd USENIX Security Symposium (USENIX Security 24), pages 6861-6848, 2024.
[91]. Venkatraman Renganathan and Tyler Holt Summers. Spoof resilient coordination for distributed multi-robot systems. 2017 International Symposium on Multi-Robot and Multi-Agent Systems (MRS), pages 135-141, 2017. URL https://api.semanticscholar.org/CorpusID:7897062.
[92]. Richard M. Chang, Guofei Jiang, Franjo Ivancic, Sriram Sankaranarayanan, and Vitaly Shmatikov. Inputs of coma: Static detection of denial-of-service vulnerabilities. 2009 22nd IEEE Computer Security Foundations Symposium, pages 186-199, 2009. URL https://api.semanticscholar.org/CorpusID:6355518.
[93]. Shiyi Yang, Zhibo Hu, Xinshu Li, Chen Wang, Tong Yu, Xiwei Xu, Liming Zhu, and Lina Yao. Drunkagent: Stealthy memory corruption in llm-powered recommender agents. arXiv preprint arXiv:2503.23804, 2025.
[94]. Sumeet Motwani, Mikhail Baranchuk, Martin Strohmeier, Vijay Bolina, Philip Torr, Lewis Hammond, and Christian Schroeder de Witt. Secret collusion among ai agents: Multi-agent deception via steganography. Advances in Neural Information Processing Systems, 37:73439-73486, 2024.
[95]. Rana Shahroz, Zhen Tan, Sukwon Yun, Charles Fleming, and Tianlong Chen. Agents under siege: Breaking pragmatic multi-agent llm systems with optimized prompt attacks. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9661-9674, 2025.
[96]. Yang Li, Wenhao Zhang, Jianhong Wang, Shao Zhang, Yali Du, Ying Wen, and Wei Pan. Aligning individual and collective objectives in multi-agent cooperation. Advances in Neural Information Processing Systems, 37: 44735-44760, 2024.
[97]. Bei Chen, Gaolei Li, Xi Lin, Zheng Wang, and Jianhua Li. Blockagents: Towards byzantine-robust llm-based multi-agent coordination via blockchain. In Proceedings of the ACM Turing Award Celebration Conference-China 2024, pages 187-192, 2024.
[98]. Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. A dynamic llm-powered agent network for task-oriented agent collaboration. In First Conference on Language Modeling, 2024.
[99]. Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Backdooring instruction-tuned large language models with virtual prompt injection. arXiv preprint arXiv:2307.16888, 2023.
[100]. Jinyuan Jia, Yupei Liu, and Neil Zhenqiang Gong. Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning. In 2022 IEEE Symposium on Security and Privacy (SP), pages 2043–2059. IEEE, 2022.
[101]. Rui Zeng, Xi Chen, Yuwen Pu, Xuhong Zhang, Tianyu Du, and Shouling Ji. Clibe: detecting dynamic backdoors in transformer-based nlp models. arXiv preprint arXiv:2409.01193, 2024.
[102]. JFrog Security Research Team. Jfrog and hugging face join forces to expose malicious ml models, March 2025. URL https://jfrog.com/community/ai/jfrog-and-hugging-face-join-forces-to-expose-malicious-ml-models/. Accessed: 2025-06-04.
[103]. Wei Duan, Jie Lu, and Junyu Xuan. Group-aware coordination graph for multi-agent reinforcement learning. arXiv preprint arXiv:2404.10976, 2024.
[104]. Mert Cemri, Melissa Z Pan, Shuyi Yang, Lakshya A Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, et al. Why do multi-agent llm systems fail? arXiv preprint arXiv:2503.13657, 2025.
[105]. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen llm applications via multi-agent conversations. In First Conference on Language Modeling, 2024.
[106]. Xuezhou Zhang, Yuzhe Ma, Adish Singla, and Xiaojin Zhu. Adaptive reward-poisoning attacks against reinforcement learning. In International Conference on Machine Learning, pages 11225–11234. PMLR, 2020.
[107]. David F Ferraiolo, Ravi Sandhu, Serban Gavrila, D Richard Kuhn, and Ramaswamy Chandramouli. Proposed nist standard for role-based access control. ACM Transactions on Information and System Security (TISSEC), 4 (3):224–274, 2001.
[108]. Boyi Zeng, Lizheng Wang, Yuncong Hu, Yi Xu, Chenghu Zhou, Xinbing Wang, Yu Yu, and Zhouhan Lin. Hurfe: Human-readable fingerprint for large language models. Advances in Neural Information Processing Systems, 37: 126332–126362, 2024.
[109]. Timour Igamberdiev, Thomas Arnold, and Ivan Habernal. Dp-rewrite: Towards reproducibility and transparency in differentially private text rewriting. arXiv preprint arXiv:2208.10400, 2022.
[110]. Weiyan Shi, Ryan Shea, Si Chen, Chiyuan Zhang, Ruoxi Jia, and Zhou Yu. Just fine-tune twice: Selective differential privacy for large language models. arXiv preprint arXiv:2204.07667, 2022.
[111]. Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. Are large pre-trained language models leaking your personal information? arXiv preprint arXiv:2205.12628, 2022.
[112]. Feng He, Tianqing Zhu, Dayong Ye, Bo Liu, Wanlei Zhou, and Philip S Yu. The emerged security and privacy of llm agent: A survey with case studies. arXiv preprint arXiv:2407.19354, 2024.
[113]. William Enck, Peter Gilbert, Seungyeop Han, Vasant Tendulkar, Byung-Gon Chun, Landon P Cox, Jaeyeon Jung, Patrick McDaniel, and Anmol N Sheth. Taintdroid: an information-flow tracking system for realtime privacy monitoring on smartphones. ACM Transactions on Computer Systems (TOCS), 32(2):1–29, 2014.
[114]. Shoaib Ahmed Siddiqui, Radhika Gaonkar, Boris K?pf, David Krueger, Andrew Paverd, Ahmed Salem, Shruti Tople, Lukas Wutschitz, Menglin Xia, and Santiago Zanella-Béguelin. Permissive information-flow analysis for large language models. arXiv preprint arXiv:2410.03055, 2024.
[115]. Pierre Peigne, Mikolaj Kniejski, Filip Sondej, Matthieu David, Jason Hoelscher-Obermaier, Christian Schroeder de Witt, and Esben Kran. Multi-agent security tax: Trading off security and collaboration capabilities in multi-agent systems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27573–27581, 2025.
[116]. Satbir Singh. Llm-based agents: The benefits and the risks. https://www.enkryptai.com/blog/llm&agents&benefits&&srisks, February 2025. Accessed: 2025-08-21.
[117]. Christian Schroeder de Witt. Open challenges in multi-agent security: Towards secure systems of interacting ai agents. arXiv preprint arXiv:2505.02077, 2025.
[118]. Anshuman Chhabra, Peizhao Li, Prasant Mohapatra, and Hongfu Liu. " what data benefits my classifier?" enhancing model performance and interpretability through influence-based data selection. In ICLR, 2024.
[119]. Anshuman Chhabra, Bo Li, Jian Chen, Prasant Mohapatra, and Hongfu Liu. Outlier gradient analysis: Efficiently identifying detrimental training samples for deep learning models. In ICML, 2025.
[120]. Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin, Ahmed Salem, Mario Fritz, and Andrew Paverd. Get my drift? catching llm task drift with activation deltas. 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 43–67, 2024. URL https://api.semanticscholar.org/CorpusID:270211056.
[121]. Dami Choi, Yonadav Shavit, and David K Duvenaud. Tools for verifying neural models' training data. Advances in Neural Information Processing Systems, 36:1154-1188, 2023.
[122]. Dan Petrovic. Advanced interpretability techniques for tracing llm activations. https://dejan.ai/blog/advanced-interpretability-techniques-for-tracing-llm-activations/, March 2025. Accessed: 2025-08-21.
[123]. Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. Watermarking makes language models radioactive. Advances in Neural Information Processing Systems, 37:21079-21113, 2024.
[124]. Meng Hao, Hongwei Li, Hanxiao Chen, Pengzhi Xing, Guowen Xu, and Tianwei Zhang. Iron: Private inference on transformers. Advances in neural information processing systems, 35:15718-15731, 2022.
[125]. Georgios A Kaissis, Marcus R Makowski, Daniel Rückert, and Rickmer F Braren. Secure, privacy-preserving and federated machine learning in medical imaging. Nature Machine Intelligence, 2(6):305-311, 2020.
[126]. Francis Dutil, Alexandre See, Lisa Di Jorio, and Florent Chandelier. Application of homomorphic encryption in medical imaging. arXiv preprint arXiv:2110.07768, 2021.
[127]. Harshal Tapsamudre, Arun Kumar, Vikas Agarwal, Nisha Gupta, and Sneha Mondal. Ai-assisted controls change management for cybersecurity in the cloud. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12629-12635, 2022.
[128]. Leo de Castro, Antigoni Polychroniadou, and Daniel Escudero. Privacy-preserving large language model inference via gpu-accelerated fully homomorphic encryption. In Neurips Safe Generative AI Workshop 2024, 2024.
[129]. Deevashwer Rathee, Dacheng Li, Ion Stoica, Hao Zhang, and Raluca Popa. Mpc-minimized secure llm inference. arXiv preprint arXiv:2408.03561, 2024.
[130]. Tao Lu, Haoyu Wang, Wenjie Qu, Zonghui Wang, Jinye He, Tianyang Tao, Wenzhi Chen, and Jiaheng Zhang. An efficient and extensible zero-knowledge proof framework for neural networks. Cryptology ePrint Archive, 2024.
[131]. Yurun Chen, Xavier Hu, Keting Yin, Juncheng Li, and Shengyu Zhang. Evaluating the robustness of multimodal agents against active environmental injection attacks. arXiv preprint arXiv:2502.13053, 2025.
[132]. Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023.
[133]. De Chezelles, Thibault Le Sellier, Sahar Omidi Shayegan, Lawrence Keunho Jang, Xing Han Lu, Ori Yoran, Dehan Kong, Frank F Xu, Siva Reddy, Quentin Cappart, et al. The browsergym ecosystem for web agent research. arXiv preprint arXiv:2412.05467, 2024.
[134]. Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala. Agentocam: A simple yet strong baseline for llm-based web agents. arXiv preprint arXiv:2410.13825, 2024.
[135]. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[136]. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web, 2023. URL https://arxiv.org/abs/2306.06070.
[137]. Yaxin Luo, Zhaoyi Li, Jiacheng Liu, Jiacheng Cui, Xiaohan Zhao, and Zhiqiang Shen. Open captcha world: A comprehensive web-based platform for testing and benchmarking multimodal llm agents. arXiv preprint arXiv:2505.24878, 2025.
下期预告:“威胁四:接口与环境风险”,敬请期待。
AI智能体安全现状最新进展|威胁篇(第一期)AI智能体安全现状最新进展|威胁篇(第二期)双A领航?智优芯生|2026年白金合作伙伴走进山石网科
山石网科是中国网络安全行业的技术创新领导厂商,由一批知名网络安全技术骨干于2007年创立,并以首批网络安全企业的身份,于2019年9月登陆科创板(股票简称:山石网科,股票代码:688030)。现阶段,山石网科掌握30项自主研发核心技术,申请560多项国内外专利。山石网科于2019年起,积极布局信创领域,致力于推动国内信息技术创新,并于2021年正式启动安全芯片战略。2023年进行自研ASIC安全芯片的技术研发,旨在通过自主创新,为用户提供更高效、更安全的网络安全保障。目前,山石网科已形成了具备“全息、量化、智能、协同”四大技术特点的涉及基础设施安全、云安全、数据安全、应用安全、安全运营、工业互联网安全、信息技术应用创新、AI安全、安全服务、安全教育等10大类产品及服务,50余个行业和场景的完整解决方案。
股票
板块
用户
资讯
评论 (0)