Cities & local government
Continuity of essential services and public trust when systems, budgets, or information environments are under strain.
- Ransomware on 911/CAD
- Utility outage
- Election disinformation
AI-Augmented Stress Testing
Find your blind spots before reality does.
Countervail helps cities, hospitals, crisis services, universities, emergency managers, startups, boards, and militaries turn red teaming, stress testing, and tabletop exercises from once-a-year events into a standing capability. We use frontier models, agentic tools, MCP, and engineered AI subject-matter experts, and accountable humans make every call.
What improves with Countervail
- Practice
Once a yearbecomes every weekShort drills fire whenever something changes: a new system, a key departure, a storm on the forecast.- What-ifs
A handfulbecomes hundredsAI drafts hundreds of ways things could go wrong. Your experts keep the ones that matter.- Perspectives
Whoever's in the roombecomes a full councilClinicians, residents, regulators, investors, adversaries, and a required skeptic, on demand.- Lessons
Weeks laterbecomes same dayThe after-action report is drafted within hours, with every finding traced to evidence.
The problem
Stress testing is bounded by three limits. AI attacks all three at once.
- 01
Bandwidth
People who can design a good exercise or poke holes in a plan are scarce. Every scenario they build is one they can't build elsewhere.
- 02
Imagination
Teams share the mental models of the plans they test. Groupthink, hierarchy, and familiarity bias shrink the space of what could go wrong.
- 03
Cadence
Exercises are costly, so they're rare. Findings go stale as systems, threats, people, and conditions change.
A traditional exercise is a scheduled sparring match with one opponent. An AI-augmented program is a gym full of tireless sparring partners, each fighting in a different style. The human coach still decides what the team needs to work on.
| Dimension | Traditional model | With Countervail |
|---|---|---|
| Cadence | Annual or semi-annual events | Continuous, with micro-exercises weekly or on change |
| Scenario generation | Handcrafted by a few experts | Hundreds of variants generated, human-curated |
| Perspectives | Whoever is in the room | Your team plus an AI SME council and stakeholder personas |
| Tooling | Generic templates or long dev cycles | Purpose-built tools for each scenario, built with AI-assisted coding |
| Data access | Manual exports and screenshots | MCP-connected agents with scoped, logged access |
| Evidence | Anecdotes from a single run | Comparable data across repeated runs, mapping the range of outcomes |
| After-action | Weeks to produce a report | Draft AAR within hours, traceable to evidence |
| Risk register | Static document, updated quarterly | Living register fed by continuous discovery |
Who we work with
Every plan has a blind spot. Every sector has an adversary.
The adversary isn't always a hacker. It might be a hurricane, a patient surge, a funding cliff, a viral rumor, or a quiet assumption buried in the plan itself. We bring the same disciplined, human-adjudicated approach to all of them.
Healthcare systems
Clinical operations that keep running through downtime, surge, and shortage, with patient safety as the test that matters.
- EHR downtime
- Mass-casualty surge
- Drug shortage
Mental health & crisis services
Crisis lines, mobile teams, and behavioral health programs tested for the moments when demand spikes and staff are stretched.
- Crisis line surge
- Staffing & burnout gaps
- Sensitive record exposure
Higher education & research
Campus safety, research security, and institutional resilience across a community that turns over every year.
- Campus emergency
- Research security
- Enrollment or funding shock
Emergency management & disaster preparedness
HSEEP-aligned exercises with branching injects, cascading infrastructure failures, and multi-agency role-players.
- Hurricane + grid failure
- Wildfire evacuation
- Comms blackout
Startups & scaling companies
Pre-mortems and red teams sized for small teams who can't afford to learn the expensive way.
- Launch pre-mortem
- Key-person loss
- Single-vendor dependency
Boards & executive leadership
Decision-grade crisis simulations and strategy red teaming that sharpen oversight before the real thing.
- Crisis decision drills
- Strategy red team
- AI governance oversight
Defense & national security
Adversary emulation, mission assurance, wargaming, and CMMC readiness, delivered in accredited environments.
- Contested logistics
- Supply chain compromise
- Insider threat
Nonprofits & community organizations
Practical resilience for mission-driven teams serving people who depend on them, on budgets that leave no slack.
- Funding cliff
- Volunteer data breach
- Reputational crisis
Capabilities
Four practice areas. One adversarial engine.
Each capability is rated by how much it expands coverage, compresses timelines, or surfaces risk that would otherwise stay hidden.
Red Teaming
Think like the opponent, whether that's a fraudster, a competitor, a ransomware crew, a skeptical regulator, or a nation-state. We red team systems, plans, and strategies, including your own AI agents.
Discuss a red team engagement →- High
Strategy & plan red teaming
Business plans, emergency plans, policies, and pitch decks attacked from every angle before they meet reality.
- High
Adversary emulation planning
Campaign plans built from threat reporting and mapped to ATT&CK, tailored to your environment.
- High
Attack path & graph reasoning
Identity, network, and asset data ingested via MCP; multi-hop paths to what matters most.
- High
Hypothesis generation at scale
Hundreds of candidate attack and failure hypotheses, ranked by feasibility and consequence.
- High
AI system red teaming
Prompt injection, tool poisoning, data leakage, and goal hijacking against your agents (MITRE ATLAS, OWASP LLM Top 10).
- Medium
Fraud, misinformation & social engineering
Authorized pretext scenarios, phishing simulations, and rumor-response drills, within rules of engagement.
- Medium
Purple team detection engineering
Each emulated technique paired with detection logic, test cases, and coverage gaps.
Resilience & Continuity
Find single points of failure across people, process, technology, facilities, and suppliers, then explore the second- and third-order effects before they happen for real.
Discuss a resilience assessment →- High
Failure mode generation
FMEA-style failure modes across people, process, technology, facilities, and supply chain.
- High
Critical function dependency mapping
Essential services traced to systems, data, vendors, buildings, and people to expose single points of failure.
- High
Cascading failure modeling
Second- and third-order effects of an outage, disaster, compromise, or loss of key staff.
- High
Surge & capacity modeling
Patient surges, call spikes, evacuations, enrollment swings, and viral growth tested against real capacity.
- Medium
Chaos scenario design
Chaos engineering experiments with blast-radius limits and rollback criteria.
- Medium
Supply chain & vendor stress
Vendor loss, compromised updates, and shortages modeled against operational timelines.
Tabletop Exercises
Branching exercises that respond to your players' decisions, with AI role-players, realistic injects, and an after-action report drafted while the room is still warm. HSEEP-aligned when you need it to be.
Plan an AI-augmented TTX →- High
Dynamic, branching scenarios
Injects adapt in real time to player decisions instead of following a fixed script.
- High
AI role-players
Adversaries, reporters, regulators, executives, patients and families, residents, students, investors, or partner agencies, on demand.
- High
Live outside-world feed
AI-driven simulations generate news, social posts, calls, alerts, and press queries in real time, reacting to what players do.
- High
Automated after-action reports
AAR drafted from transcripts with decisions, gaps, and recommendations traced to evidence.
- Medium
Real-time adjudication support
Plausibility of player actions assessed against your plans, protocols, and real-world constraints.
- Medium
Tested scenario library
Proven scenarios reused and refreshed on a regular cadence, so exercises start fast and stay timely.
- High
Built-in measurement
Pre- and post-exercise assessments, decision logs, and transcript analysis captured the same way every run.
- Medium
Micro-exercises
Focused 20-minute drills on demand for individuals, shifts, or small teams.
Risk Discovery
Continuous threat modeling, control gap analysis, and assumption hunting that feed a living risk register, not a static document updated once a quarter.
Discuss risk discovery →- High
Automated pre-mortems
Assume the plan failed; generate plausible causal narratives for why.
- High
Assumption hunting
Hidden assumptions extracted from plans, contracts, protocols, and strategy documents, and each one tested.
- High
Control & compliance gap analysis
Controls mapped to the frameworks you answer to, such as HIPAA, FERPA, SOC 2, ISO 27001, NIST CSF, 800-53, and CMMC, with gaps flagged.
- High
Continuous threat modeling
STRIDE or LINDDUN models generated from architecture diagrams, code, and design docs.
- Medium
Weak signal detection
Tickets, near-misses, incident reports, audit notes, and complaints correlated to spot emerging risk patterns.
- Medium
Risk register maintenance
The register stays current as systems, threats, people, and findings change.
Operating principle
AI proposes. Humans dispose.
Any action that touches live systems, is irreversible, or affects people requires explicit human authorization. Agents run on least privilege, and every tool call is logged. Humans hold the decision gates at scoping, authorization of active testing, adjudication of findings, and acceptance of risk.
Exercise program
Exercises that produce evidence, not just experience.
Most exercises end with a hot wash and a few anecdotes. We run them as a research program: each one answers a defined question, produces data you can compare across runs, and ends up in front of the people who can act on it.
From questions to findings
One exercise tells you what happened once. A program tells you what tends to happen, how often, and what changes the outcome.
- 01
Set the research agenda
Agree on the questions your exercises should answer, the scenarios that will answer them, and the audience for each.
- 02
Design runs as studies
Define what each run tests, and standardize scenarios and data capture so results are comparable. Repeated play maps the full range of outcomes, not just one story.
- 03
Measure everything
Pre- and post-exercise assessments, decision logs, transcript analysis, and after-action reports, captured the same way every time.
- 04
Turn results into findings
Publish what the exercises show and brief senior leaders inside and outside your organization on the gaps they expose.
Built to scale beyond us
- Reuse
Tested scenario library
A library of proven scenarios the team reuses most of the time, refreshed on a regular cadence so they stay timely.
- Scale
TTX in a box
A facilitator kit with scenarios, injects, data capture templates, and guides so other teams and partners can run exercises to the same standard.
- Realism
Live outside-world feed
AI-driven simulations that generate news, social media, calls, and stakeholder reactions during play, responding to what players decide.
- People
Team & facilitator bench
We help you hire, train, and develop an exercise team and a bench of facilitators who can run the program without us.
Signature methods
Approaches that weren't practical before AI.
Beyond doing existing work faster, these methods were previously out of reach because of cost, staffing, or cognitive load.
Adversarial Multi-Agent Debate
Red, blue, and judge agents argue a scenario through multiple rounds. Like a moot court for your plan, the value lies in the arguments the judge can't dismiss. Your experts review the surviving arguments instead of generating them from scratch.
The Thousand Pre-Mortems
Hundreds of independent pre-mortems, each seeded with a different persona, threat, or failure. Clustered results turn intuition into something closer to a statistical distribution of failure.
Persona-Based SME Councils
Engineered AI SMEs review a plan from their lens, with a required dissent step against consensus.
Digital Twin Wargaming
Agents run attack, disaster, and failure scenarios against a mirror of your systems or operations, continuously, with no risk to the real thing.
Change-Triggered Exercises
A focused micro-exercise fires when a system goes live, a key person leaves, a vendor changes, a regulation shifts, or a new threat emerges.
Repeated Play
Run the same scenario dozens of times with different players and variations. The spread of outcomes shows what is luck and what is a real gap.
Red Teaming the AI Itself
Prompt injection via documents and web content, compromised MCP servers, excessive agent permissions, exfiltration through outputs.
Plan Assassination
One objective: find the single assumption that, if false, collapses the plan. Iterate until the plan survives.
AI SME Council
More perspectives in every room.
An AI SME is an engineered persona: a defined role, a grounding corpus of authoritative sources, an analytic standard suited to the domain (intelligence tradecraft, clinical evidence, engineering safety), and explicit limits on what it may assert. It behaves less like a chatbot and more like a structured analytic instrument.
Think of it as a reference librarian who has read the entire specialty library and can argue from it, but has never been in the field. Excellent for breadth and structured critique. No substitute for lived experience. We build each council around your sector and the people your decisions affect.
Adversary Emulator
Thinks and plans as a specific threat: a criminal group, fraudster, activist, competitor, or nation-state.
Threat intel · ATT&CK groups · incident historyOperations Owner
Evaluates impact on essential services, outcomes, and operational tempo.
Continuity plans · essential functions · BIAClinical & Care Safety Lead
Weighs every option against patient and client safety, duty of care, and clinical protocol.
Clinical guidelines · safety event data · care protocolsEmergency Manager
Tests plans against incident command, mutual aid, and the realities of a disaster footprint.
NIMS/ICS · HSEEP · hazard mitigation plansCommunity & Stakeholder Voice
Represents residents, students, patients, and vulnerable groups who bear the impact.
Community feedback · accessibility & equity guidanceBoard Director & Investor
Probes fiduciary exposure, strategy, runway, and how a decision will look in hindsight.
Governance charters · financials · audit findingsCompliance Assessor
Tests findings against the control frameworks and regulations you answer to.
HIPAA · FERPA · SOC 2 · NIST · CMMCDetection & Response Engineer
Assesses whether trouble would be seen, escalated, and contained in time.
Detection rules · log schemas · runbooksSystems Safety Engineer
Examines hazards, failure modes, and cascading effects.
FMEA · STPA · safety standardsLegal & Communications
Surfaces disclosure, regulatory, and reputational consequences.
Notification rules · crisis comms plansContrarian / Devil's Advocate
Required to argue against consensus in every review.
The plan itself and all prior outputs
How we keep AI SMEs credible
- Grounding over recall
- Factual claims cite the grounding corpus.
- Calibrated confidence
- Explicit confidence levels; evidence kept separate from inference.
- Structured dissent
- At least one opposing view per review, to counter model agreeableness.
- Bounded authority
- AI SMEs advise. They never approve, authorize, or accept risk.
- Model diversity
- Personas run on different models to reduce correlated blind spots.
- Calibrated before use
- Tested against golden scenarios and human SMEs, like a new analyst on probation.
Consulting services
Ways to work with us.
Start with a single exercise, or stand up a standing capability inside your organization. Every engagement is scoped to your data-handling rules, regulatory obligations, and rules of engagement.
Capability build
Stand up AI-augmented assurance in 180 days
We build the capability with your team, then hand it over: governance, approved models, AI SME personas, MCP integrations, tooling, and metrics.
- Crawl · Days 1–30
Research agenda: the questions to answer and the audience for each. Governance, approved models, data handling, and AI rules of engagement. Two or three calibrated AI SMEs. Low-risk uses: pre-mortems, threat modeling, TTX drafting, AAR drafting.
- Walk · Days 31–90
Read-only MCP connections to your operational systems, such as ticketing, incident logs, asset inventories, and security tooling. First measured, AI-augmented TTX with pre- and post-assessments. Scenario library started. Multi-agent debate for plan and strategy reviews.
- Run · Days 91–180
Agents on a digital twin or test range for continuous scenarios. Change-triggered micro-exercises and a living risk register. AI system red teaming. TTX-in-a-box kit and a trained facilitator bench. Findings briefed to leadership.
Program
Exercise Program Leadership
We set the research agenda, design and measure your exercises, build the scenario library, and develop your team and facilitator bench until you can run it yourselves.
Discuss program leadership →Kit
TTX in a Box
A facilitator kit with tested scenarios, injects, live-feed tooling, measurement templates, and facilitator training, so your teams and partners can run exercises to the same standard.
Get the kit →Assessment
Assurance Readiness Assessment
Where you stand today: exercise cadence, plans, tooling, data access, governance, and team skills, with a prioritized path to continuous assurance.
Request an assessment →Exercise
Turnkey AI-Augmented Tabletop
Branching scenario, AI role-players, realistic injects, and a traceable draft AAR within hours of the final inject. Cyber, clinical, disaster, or business continuity.
Plan a tabletop →Leadership
Board & Executive Crisis Simulation
A half-day decision drill for boards and leadership teams, with realistic pressure from AI-played media, regulators, investors, and the public.
Plan a simulation →Assessment
AI Agent & MCP Red Team
Adversarial testing of your deployed agents and integrations for prompt injection, tool poisoning, permission sprawl, and data leakage.
Test your agents →Ongoing
Continuous Assurance Retainer
Change-triggered micro-exercises, living risk register upkeep, and periodic plan assassination reviews, with leadership reporting.
Discuss a retainer →Training
Team Upskilling
Context engineering, agent orchestration, evaluation discipline, AI security literacy, and the most important skill: critical adjudication.
Plan a workshop →Reference operating model
Seven phases. Human gates where they matter.
- 1
Agenda & scope
AI drafts the questions to answer, maps critical functions, proposes scenarios.
Human sets the agenda, audience, and rules of engagement.
- 2
Intelligence & context
AI summarizes threat reporting, incident history, and sector guidance.
Human validates relevance and sourcing.
- 3
Hypothesis generation
AI generates and ranks attack and failure hypotheses.
Human curates and prioritizes.
- 4
Scenario & tool build
AI writes scenarios, injects, and tooling.
Human reviews code and scenario realism.
- 5
Execution
AI runs approved tests in a sandbox or twin; plays TTX roles.
Human authorizes actions and holds the kill switch.
- 6
Analysis
AI analyzes decision logs and transcripts, compares runs, drafts findings and AAR.
Human adjudicates findings and severity.
- 7
Findings & action
AI drafts briefings, fixes, and plan updates.
Human briefs leaders, accepts risk, and assigns owners.
Mandatory human decision gate
Enabling stack
Five layers that compound.
Each is useful alone. Together they create the capability. For sensitive data, we work with locally hosted models or accredited environments that match your obligations, from HIPAA-eligible services to FedRAMP and DoD IL levels.
- ModelsReasoning, generation, critiqueFrontier LLMs, reasoning models, local open-weight models
- AgentsAutonomous multi-step executionPlanner/executor loops, multi-agent debate, coding agents
- MCPStandardized, auditable tool and data accessTicketing, incident logs, asset inventories, security tooling, GIS, cloud APIs
- AI-assisted codingRapid custom toolingParsers, inject engines, dashboards, simulators, test harnesses
- GroundingAuthoritative knowledgeATT&CK, ATLAS, NIST, HSEEP, NIMS/ICS, sector regulations, clinical guidance, prior AARs
Governance, safety & guardrails
Governed like any high-stakes capability.
Aligned with the NIST AI Risk Management Framework and ISO/IEC 42001, the rules of your sector (from HIPAA, 42 CFR Part 2, and FERPA to DoD AI policy), and your existing rules of engagement.
Agent exceeds scope
Machine-readable rules of engagement, allow-listed targets, least-privilege MCP credentials.
Impact on live operations
Human approval gates, sandbox or twin execution by default, and a kill switch.
Sensitive data exposure
De-identification by default; local or accredited models for PHI, student records, CUI, and other protected data.
Prompt injection & tool compromise
Retrieved content treated as untrusted; vetted, pinned, isolated MCP servers; output filtering.
Unverified findings
Mandatory human adjudication, evidence traceability, and full logging of prompts, tool calls, and approvals.
Over-reliance on AI
Periodic human-only exercises to maintain team skill and benchmark AI value.
Participant wellbeing
Scenarios involving trauma, self-harm, or loss are designed with clinicians, flagged in advance, and include opt-outs.
Harm to vulnerable groups
Scenarios and findings reviewed for disparate impact on people with the least margin for error.
Measuring effectiveness
We report on signal, not volume.
- Unique validated findings per cycleIs AI expanding discovery, not just volume?
- Time from hypothesis to validated findingSpeed gained from AI and purpose-built tooling
- Detection & response coverage deltaHow much more you'd see, and how much faster you'd act
- AI output precisionValidated ÷ proposed hypotheses
- Pre- to post-exercise improvementWhat participants actually learned and changed
- Outcome spread across repeated runsWhich results are robust and which were luck
- Mean time to after-action reportHow fast exercises become learning
- Remediation closure rateWhether findings actually drive change
- Novel risk ratioShare of findings absent from prior registers
Minimum viable team
Four people can stand this up.
The AI-augmented team is smaller than you'd expect and more cross-trained. We help you staff and train it.
- Exercise & red team leadStrong adjudication skills
- AI / agent engineerPersonas, evaluation harnesses, guardrails
- Exercise designer & facilitatorScenario library, live-feed direction, facilitator training
- Domain SMEClinical, emergency, academic, or operational realism
A bench of trained facilitators extends the program to more teams and partners. AI SMEs fill remaining perspective gaps, and human specialists validate high-stakes findings.
Start a conversation
The advantage isn't who has the biggest team.
It's who can generate, test, and learn from the most well-adjudicated hypotheses per week. Tell us what you're protecting and we'll propose a scoped first engagement.
- Scoping call within one week
- Rules of engagement and data handling agreed before any work
- Delivery inside your secure or accredited environment when required