Scan Before You Install: Detecting and Mitigating Local and Composite Vulnerabilities in Skills
Parth A. Gandhi, Avital Aviv, Yuval Elovici, Asaf Shabtai and colleagues (Ben-Gurion University of the Negev) build a six-module scanner for unintended vulnerabilities in agent skills, including risks that appear only when skills interact.
Ask this paper
Wild results. In 1,587 skills from the 75 most-starred GitHub repositories containing skills, analysts confirmed 53 of 59 findings (89.8% precision) across 23 repositories, with no evidence of malicious intent.
Local analysis. It reaches 95.90% recall at a 4.22% false-positive rate on SkillTrustBench and an F1 of 0.947 on 1,000 MalSkillBench skills, ahead of Cisco Skill Scanner, NVIDIA SkillSpector, AgentSkillsScanner and SkillSieve.
Composite analysis. Modeling dependencies through shared resources, invocations and instructions gives an F1 of 0.991 on SCR-Bench against 0.816-0.858 for three SkillsGuard configurations.
Mitigation. For 47 of 55 GitHub findings, recommended mitigations stopped proof-of-concept attacks in all trials and preserved tested functionality.
Abstract
An agent skill packages natural-language instructions, scripts, and reference files that an LLM-based agent loads for a task. Skills may access the file system, shell, and network under user-authorized permissions; weaknesses can lead to unsafe execution. Existing scanners primarily target malicious skills, not unintended vulnerabilities in public repositories, and analyze skills in isolation, overlooking multi-skill interactions. To address these gaps, we developed a six-module scanner: Content Extraction, Local Analysis, Composite Analysis, Validation, Reporting, and Mitigation. Local Analysis detects unintended vulnerabilities in benign skills. On a 556-case leaderboard subset of SkillTrustBench, it achieved 95.90% recall with a 4.22% false positive rate; on a balanced sample of 1,000 MalSkillBench skills, its F1 was 0.947, outperforming Cisco's Skill Scanner, NVIDIA's SkillSpector, AgentSkillsScanner, and SkillSieve. In 1,587 skills from the 75 most-starred GitHub repositories, three analysts confirmed 53 of 59 findings (89.8% precision) across 23 repositories. We developed proof-of-concept (PoC) attacks for each vulnerability, disclosed 10 medium- and high-severity cases to maintainers, and tested mitigations on the remaining cases. Composite Analysis models dependencies among skills through shared resources, invocations, and instruction-mediated interactions. On SCR-Bench, its F1 score was 0.991 (98.7% precision, 99.6% recall), versus 0.816-0.858 for three SkillsGuard configurations released with CompoSkill. It identified two composite findings on GitHub, validated by PoC attacks. The Mitigation module recommends skill substitution, patches, and configuration changes. For 47 of 55 GitHub findings, mitigations prevented PoC attacks in all trials while preserving tested legitimate functionality.