Skip to content
View bbrookhart's full-sized avatar

Block or report bbrookhart

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
bbrookhart/README.md
AI safety and security research across foundation models, autonomous agents, and cyber-physical systems

AI Safety & Security Researcher · Foundation Model Evaluation · Secure Autonomous Systems

LinkedIn Research Method

As AI systems move from generating information to influencing decisions and taking action, safety and security become one problem: keeping increasingly capable systems aligned with human intent under uncertainty, pressure, and adversarial conditions.

I study how advanced AI systems fail — from foundation-model behaviors that can distort beliefs, values, or decisions to agentic misalignment, adversarial compromise, and unauthorized autonomous action — and how rigorous evaluation, human oversight, adversarial validation, and independently enforced controls can reduce those risks.

My research asks:

How can we measure when AI systems undermine human agency, trace how failures propagate across trust boundaries, and build technical boundaries that keep those failures from becoming consequential actions?

Foundation-model evaluation · societal-impact measurement · human agency · agentic AI safety · AI red teaming · attack-path analysis · runtime enforcement · cyber-physical resilience


Research Program

My portfolio is organized around one end-to-end research thesis:

Observe → Evaluate → Adversarially Test → Control → Measure Consequence

Research pillar Core questions Representative work
Foundation Model Safety & Societal Impact When do model behaviors undermine epistemic, value, or decision autonomy? How should persuasion, sycophancy, overreliance, and disempowerment potential be measured? AUTONOMY EVALS — Preserving Human Agency (active) · AI persuasion study (planned, ethics-dependent)
Agentic AI Safety & Control What happens when models can act rather than only advise? Which approval, monitoring, capability, and policy mechanisms constrain misaligned or compromised agents? CERBERUS NULL · VIGIL (private implementation) · human oversight under agentic misalignment (planned)
AI Security & Adversarial Evaluation How do weaknesses in AI, tools, identity, software, and cloud infrastructure compose into consequential attack paths, which controls actually break those paths, and what may a defensive system change on its own authority? FAULTLINE (private implementation) · MERIDIAN ATLAS SECURITY · NIGHTGLASS · FALSEPROXY · GHOSTLEDGER
Cyber-Physical & High-Consequence AI How do autonomous decisions propagate into operational and physical consequence, and how can independent safety boundaries preserve mission integrity? BLACKSTART · critical-infrastructure and software-integrity research

Emerging Safety Research Track

These studies extend the existing security portfolio upward from protecting systems from AI failure to also measuring how foundation-model behavior can affect people and human decision-making.

Status: active research

Research question:

Under what conversational conditions do foundation-model assistants exhibit behaviors that may undermine a user's epistemic, value, or action autonomy, and can targeted interventions reduce those behaviors without materially degrading helpfulness?

The claim boundary is explicit: these evaluations measure model behaviors associated with autonomy-preserving or autonomy-undermining interaction patterns; they do not establish psychological or societal harm to people.

AI Persuasion & Human Influence

Status: planned follow-on study; human-subject work contingent on appropriate ethics review

Planned question:

When does AI-generated or personalized communication change human beliefs, confidence, and source trust, and which design interventions preserve autonomy while retaining legitimate decision support?

The study will begin with replication or reanalysis of existing public persuasion datasets before any original participant recruitment.

Human Oversight Under Agentic Misalignment

Status: planned experimental evaluation

Planned question:

Which human-oversight and technical-control mechanisms most effectively prevent policy-violating actions by goal-directed agents, and what operational cost do those safeguards impose?

The evaluation will use simulated organizational environments, synthetic data, controlled goal conflict, approval policies, permission boundaries, and measurable safety–utility tradeoffs.


Flagship Research // Start Here

Preserving Human Agency — foundation model autonomy evaluation

Foundation Model Safety · Human Agency · Behavioral Evaluation · Measurement Validation

A research-grade evaluation framework for determining whether conversational AI remains evidence-grounded and user-directed under confidence pressure, validation seeking, counterevidence, value revelation, and consequential recommendation requests. The project explicitly tests the failure mode where an anti-sycophancy intervention becomes reflexively skeptical: unsupported agreement should fall while warranted agreement and useful directness remain intact.

Current evidence: 30 authored base scenarios → 720 controlled v0.1 variants; three autonomy domains; reproducible model/grader/human-annotation pipeline; v0.2 development branch adds positive controls, evidence-sensitive agreement, multiple trajectory families, independent scenario review, and measurement-validation gates. Real-model v0.2 findings remain pending calibration.


Faultline — continuous adversarial validation across AI and cloud

FAULTLINE · Private Implementation · Public Recruiter Dossier

AI & Cloud Adversarial Validation · Attack-Path Analysis · MCP & Coding-Agent Security · Control Effectiveness · Authorized Autonomous Response

Faultline asks whether a weakness in AI, software, identity, or infrastructure can propagate across trust boundaries into material security impact — and what is the smallest defensive change that breaks that path. It combines probabilistic adversarial evaluation, security-graph reasoning, replay, differential remediation testing, purple-team validation, control-effectiveness analysis, and minimum-control-cut reasoning across AI and cloud systems.

It now closes the loop. The same minimum-control-cut reasoning produces a typed remediation proposal that an external authority must sign before anything executes: five registered operations, no shell, no generic patch, no cluster-admin; autonomy tiers derived from the operation registry rather than from model confidence; containment proven by independent re-observation and a recomputed attack-path delta, with UNKNOWN that no code path converts into success; and rollback pinned to the resource version the change left behind.

Current evidence: synthetic AI-agent lab baseline 236/720 successful attack trials → 0/720 after hardening; MCP lab 371/600 → 0/600; coding-agent arena 381/630 → 0/630, with legitimate-work checks retained. The response plane is 5 registered operations, an 8-row verification contract resolving to 5 verdicts, R0–R5 autonomy tiers, and capabilities that are single-use with a TTL of ≤30 minutes — covered by 1,775 automated tests and 60/60 OPA policy tests. The loop is driven end to end against a synthetic cyber range and a real PostgreSQL journal, including forged/replayed/expired-capability refusals, single-use redemption under twelve racing connections, and crash resumption at every transition. These are controlled evaluations, not production-world effectiveness claims: the loop has not yet been run against a real Kubernetes cluster, a real identity provider, or a real incident.

Find the path. Prove the risk. Break the chain.


VIGIL — runtime safety and security control plane for autonomous AI agents

VIGIL · Private Implementation · Public Recruiter Dossier

Agent Runtime Security · Zero-Trust Authority · Capability Enforcement · Pre-Execution Control

VIGIL is a local runtime safety and security control plane that treats autonomous agents as untrusted principals. It mediates process, filesystem, network, tool, and credential authority through deterministic policy, signed capabilities, provenance and taint tracking, action budgets, approval gates, and tamper-evident audit before protected execution.

Measured comparison: across 14 attack and benign cases in seven families, scored on observed side effects rather than transcripts, the unprotected arm completed 10/10 attacks, a static pattern filter 6/10, and VIGIL 0/10 — while VIGIL completed 3 of 4 benign tasks outright and escalated the fourth to human approval rather than denying it. The pattern filter falsely denied one outright. Removing VIGIL's workspace boundary alone reopened 5 of the 10, attributing those preventions to that mechanism rather than to the system in general. This is a scripted adversary, not a language model: it measures the enforcement layer, not model susceptibility.

Current generated evidence: inventory of 1,056 Rust, 199 Swift, and 11 Python source test entry points; 25 adversarial harness tests; 21 named attack paths; 14 fuzz targets; 67 ADRs; 19 workspace crates; and 0 unsafe Rust constructs. CI now executes the full suite — including both macOS adapters, a kind + Cilium non-bypassability run, and the fuzz campaign. This is implementation, test, and bounded-experiment evidence, not a claim of production-world safety. Apple entitlement-dependent activated-device validation remains.

Stop unsafe action before it becomes consequence.


CERBERUS NULL — formally constrained agentic cyber defense

Formal Methods · Agent Authority · Deterministic Policy · Protected Execution

CERBERUS NULL is a research-grade control plane that treats the AI planner as untrusted and independently evaluates identity, mission, capability, provenance, policy, safety, consequence, approval, and emergency state before protected execution.

Measured v0.1 evidence: 0/24 unsafe actions escaped; the naive comparator executed 13/14 unsafe actions versus 0/14 with CERBERUS NULL; bounded TLC model checking found no F1–F7 counterexample across 221 reachable states. These results are bounded to the evaluated model and experiment rather than a claim of complete system security.

Reason probabilistically. Act deterministically.


BLACKSTART — consequence-driven cyber-physical resilience

Cyber-Physical Resilience · OT/ICS · Safety Invariants · Mission Consequence

BLACKSTART is a reproducible research range that connects cyber events to control state, physical-process behavior, explicit safety invariants, causal evidence, and mission consequence so defensive mechanisms can be evaluated against what actually matters operationally.

Measured evidence: in the frozen synthetic experiment, the unprotected run accumulated 639.5 seconds of unsafe duration; the independent engineering backstop reduced unsafe duration to 0.0 seconds with independently recalculated metrics. This is a controlled synthetic experiment, not a production-world effectiveness claim.

Assume compromise. Preserve the mission.


Research Method

Research QuestionConstruct / Threat ModelControlled EvaluationInterventionMeasurementUncertaintyReproduction

Across both safety and security research, I use the same methodological commitments:

  • Define the claim boundary first. Model behavior, human impact, system compromise, and physical consequence are related but distinct constructs and should not be conflated.
  • Paired and adversarial evaluation. Baselines, matched controls, negative cases, counterfactual pairs, failure attribution, attack-path composition, and safety–utility tradeoffs remain visible.
  • Evidence over demos. Claims are tied to code, tests, raw artifacts, evaluation logs, statistical analysis, limitations, and reproducibility instructions.
  • Validate the measurement. Automated graders are treated as measurement instruments rather than unquestioned ground truth; human agreement and grader sensitivity should be tested where relevant.
  • Test the control, not just the attack. A blocked attack is not enough: remediation should be replayed, utility should remain visible, and prevention, detection, alerting, and telemetry gaps should be distinguished.
  • Authority outside the model. For autonomous-action systems, models may reason and propose; independently enforced controls decide what may execute.
  • Consequence-aware analysis. Evaluation follows what model behavior can change — beliefs, decisions, identities, digital state, cloud access, or physical state — rather than stopping at surface-level outputs.

Active AI Safety & Security Research

Foundation-model safety and societal impact

Project Research question Current public evidence
AUTONOMY EVALS When do foundation-model assistants remain evidence-grounded and user-directed, and when do they drift toward sycophancy, value substitution, or premature action pressure? 30 authored v0.1 bases, 720 controlled variants, paired multi-turn design, blinded grader/human annotation workflows; v0.2 validity upgrade adds positive controls and evidence-sensitive agreement

Agent behavior, adversarial evaluation, and trust-boundary security

Project Research question Current evidence
FAULTLINE (private implementation; public dossier) Can weaknesses in AI, software, identity, or cloud infrastructure compose into consequential attack paths, what minimum control set breaks those paths, and can that change be externally authorized, narrowly executed, independently verified, and undone? Synthetic AI-agent, MCP, and coding-agent baseline→hardened evaluations; replay, differential testing, minimum-control-cut analysis, control-effectiveness and purple-team measurement; a closed response loop over a synthetic cyber range with signed capabilities, an eight-row verification contract, rollback, and journalled crash recovery; not yet run against a real cluster
MERIDIAN ATLAS SECURITY How should adversarial AI evaluation connect attack replay, telemetry, detection, and assurance rather than stopping at jailbreak counts? 45 attack replays + 65 benign sessions; N=20 retests with Wilson 95% CIs; measured detection failures including a 13.5%-precision rule and silent telemetry defects
CRUCIBLE How should AI cyber capability and defensive uplift be measured without confusing benchmark success with real-world capability? Functional v0.1 defensive harness, bounded tools, scoring isolation, contamination controls; real-model and human-uplift studies remain planned
NIGHTGLASS Can malicious influence persist across agent sessions and trigger a delayed unauthorized action? 40 enterprise scenarios, 12 attack mechanisms, paired design, deterministic evidence explorer
FALSEPROXY Can identity, provenance, scope, audience, and revocation survive MCP/A2A delegation? 96 declarative scenarios across 18 attack classes with matched benign controls and a reproducible preview pipeline
GHOSTLEDGER Can an agent complete the visible task while quietly degrading mission integrity? 48 paired cases, 18-class sabotage taxonomy, 96 reproducible technical-preview bundles

Critical infrastructure, trust, and strategic resilience

Project Focus
IRONVEIL End-to-end software/firmware provenance, release authorization, update security, and source-to-installed-state assurance
HARVEST//ZERO Cryptographic discovery, dependency-aware post-quantum migration, CBOM, measured PQC microbenchmarks, and crypto agility
DEADFALL Temporal campaign graphs for detecting persistent access established for possible future critical-infrastructure disruption

Applied AI security and governance

Project Focus
SECURITY-FIRST MULTI-AGENTIC SOC Least privilege, human approval, proposal-only containment, deterministic routing, and auditable local-LLM SOC workflows
NORTHSTAR MEDICAL AI DEPLOYMENT Secure deployment and lifecycle assurance for constrained agentic AI in a high-stakes healthcare environment

Technical & Research Depth

Foundation-model safety & evaluation AI security & control Systems & quantitative methods
Behavioral evaluation · multi-turn evals · construct definition · paired/counterfactual controls · grader validation · evidence-sensitive agreement · safety–utility tradeoffs · societal-impact measurement · TEVV Agent authority · prompt injection · MCP security · coding-agent security · memory poisoning · tool-use security · RAG authorization · delegation · provenance · attack-path analysis · minimum control cuts · runtime monitoring · human approval Python · Go · Rust · FastAPI · ConnectRPC · Temporal · Neo4j · PostgreSQL · Docker · OpenTelemetry · OPA/Rego · GitHub Actions · Linux · experimental design · confidence intervals · differential testing · reproducibility

Frameworks and standards: NIST AI RMF · NIST SP 800-53 · NIST SP 800-82 · ISO/IEC 42001 · ISO/IEC 27001 · OWASP GenAI guidance · MITRE ATT&CK / ATLAS


Education & Credentials

PhD, Information Technology (in progress) Research focus: AI safety and security across agentic AI, foundation-model evaluation, and cyber-physical systems
M.S., Cybersecurity & Information Assurance Completed
BBA, Business Analytics Completed
CompTIA PenTest+ · CySA+ · ISC2 CC Certified

Research Direction

The attack surface is no longer only the infrastructure. It is also the model behavior, the human decision, the trust boundary, the authority behind an action, and the consequence that follows.

My long-term research program connects three questions that are often studied separately:

  1. What do increasingly capable AI systems do to human judgment and agency?
  2. How do failures propagate across agents, tools, identities, and infrastructure?
  3. What happens when those systems gain the authority to act?

The objective is to build evaluation, adversarial-validation, and control methods that remain useful across that entire path — from model behavior and societal impact to cross-layer compromise and autonomous execution.

Evaluating and securing AI systems from model behavior to real-world action.

LinkedIn

@bbrookhart's activity is private