Research direction

AI Agent Security

Mengyuan Li’s research on AI agent security uses trusted execution environments (TEEs) and runtime monitoring to observe agent execution and to build trusted infrastructure for LLM systems and AI agents. The goal is agent backends that remain observable, trustworthy, and resilient when they run on shared cloud infrastructure and handle sensitive data.

Topics: secure AI agents · agent runtime monitoring · TEEs for AI agents · trusted infrastructure for LLM systems · trustworthy agent execution

Perspective

This direction focuses on monitoring and protecting agent execution rather than only verifying final outputs. It brings together TEEs, confidential computing, and runtime observation to support trustworthy LLM systems and agentic infrastructure.

Papers

  1. ASPLOS 2026 WAVE: Leveraging Architecture Observation for Privacy-Preserving Model Oversight Haoxuan Xu*, Chen Gong*, Beijie Liu*, Haizhong Zheng, Beidi Chen, Mengyuan Li 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2026)

    Explores how system-level observation can support privacy-preserving monitoring of model execution, which aligns with runtime monitoring for trustworthy LLM systems and agents.

    DOI · Code · BibTeX
  2. HPCA 2026 SCALE: Tackling Communication Bottlenecks in Confidential Distributed Machine Learning Joongun Park, Yongqin Wang, Huan Xu, Hanjiang Wu, Mengyuan Li, Tushar Krishna 2026 IEEE International Symposium on High-Performance Computer Architecture (HPCA 2026)

    Contributes infrastructure for secure and efficient confidential multi-GPU execution, which is important for building trusted backends for large-scale LLM systems and agents.

    DOI · BibTeX
  3. arXiv 2026 Preprint CatchBench: When Can an Agent Failure Be Caught? Yue Zhao, Mengyuan Li, Ruolin Li, Prince Zizhuang Wang, Shuli Jiang, Linsey Pang, Xiongye Xiao, Xiyang Hu arXiv preprint arXiv:2608.22808

    Benchmarks when an agent failure can be caught by putting one auditor’s question to three information states: a run’s declared configuration, a growing prefix of its trace, and the finished trace.

    PDF · arXiv · Code · BibTeX
  4. arXiv 2026 Preprint Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skills Peichun Hua, Haoxuan Xu, Mengyuan Li arXiv preprint arXiv:2608.04192

    Shows that keeping an LLM agent skill’s files secret does not keep its functionality secret: through ordinary use alone, the black-box attack SkillClone recovers several of 30 tested skills exactly or partially.

    PDF · arXiv · BibTeX
  5. arXiv 2026 Preprint Infrastructure for Valuable, Tradable, and Verifiable Agent Memory Mengyuan Li, Lei Gao, Haoxuan Xu, Jiate Li, Potung Yu, Lingke Cheng, Yue Zhao, Murali Annavaram arXiv preprint arXiv:2603.24564

    Proposes infrastructure that makes agent memory verifiable and tradable: ClawGang binds memory to verifiable computational provenance through TEE remote attestation, and MeowTrade is a market layer for listing, transferring, and governing certified memory artifacts.

    PDF · arXiv · Code · BibTeX

* Equal contribution.

Projects

Open source

Foundations

TEE and Confidential Computing provide mechanisms for isolated execution, protection of agent state, and secure multi-tenant infrastructure.

Verifiable AI is related because agent systems also need execution integrity, auditability, and trustworthy model-serving pipelines.

Challenges

Key open problems include:

  • observing agent execution with TEEs or runtime monitoring;
  • isolating tool execution;
  • protecting long-lived memory and secrets;
  • building trusted infrastructure for LLM systems and agents;
  • defending agent backends against cloud- and hardware-level leakage.