Microsoft has unveiled two new open-source tools called fort wall And clarity Helping developers better test the security of artificial intelligence (AI) agents.
Rampart, short for Risk Assessment and Measurement Platform for Agent Red Teaming, serves as a PyTest-native security and protection testing framework for writing and running security and protection tests for AI agents, covering both adversarial and benign issues as well as various pitfall categories.
Users can write test cases to attack or investigate the AI agent to detect potential security violations such as cross-prompt injection, where untrusted data indirectly accesses the AI system through a data source (for example, email, file, or web page) processed by the AI system, or unintended behavior regression and data intrusion.
RAMPART then evaluates the results of those tests and reports the results. It simply requires an adapter that connects an agent to the test suite. The tool is built on PyRIT (short for Python Risk Identification Tool), which Microsoft released more than two years ago as a way to test AI systems.
Clarity, on the other hand, is described by the tech giant as a “structured sounding board” that helps developers arrive at the right approach before even writing a line of code. It’s an “AI thinking partner that pushes back,” guiding them through problem clarification, solution exploration, failure analysis, and decision tracking.
In publicly releasing these tools, Microsoft said the idea is to figure out why certain decisions are included in the early stages of software development so that any potential issues – for example, an agent’s access to tools – can be addressed well before the system is built.
“We wanted to give product managers and engineers a way to pressure-test their assumptions early in a project, when it’s cheaper to change course and the right conversation can save months of work,” Ram Shankar Shiv Kumar, data cowboy and founder of Microsoft’s AI Red Team, said in a blog shared with The Hacker News.
Microsoft noted that a secondary motivation behind investment in these tools is to make incidents reproducible and mitigations verifiable and to turn learnings from red teaming exercises into driveable engineering assets.
“Whereas PyRIT is optimized for black-box discovery by security researchers after the system is built, RAMPART is designed for engineers while the system is being built,” said Shiv Kumar. “Clarity helps teams articulate design intent and capture assumptions. Together, these approaches take AI security from a one-time review to a set of living artifacts that developers can use throughout the lifecycle.”
