Case study in progress Verified evidence, incomplete publication scope

nullstate

nullstate is a purple-team sandbox for infrastructure code. It attacks Terraform-defined resources in local cloud emulators, applies a deterministic fix, and replays the attack to prove the fix holds. I built it in 48 hours for the AMD x lablab.ai hackathon and kept developing it as open source.

Repository Watch demo Submission PDF
Blocked post-fix AWS replay The public S3 attack worked before the fix and failed after it.
Private self-hosted models vLLM on an AMD MI300X, reached through an SSH tunnel.
48h original hackathon build AMD x lablab.ai, then continued as open source.

Current publication status

The core AWS path is proven. The full case study is not finished.

nullstate tests whether infrastructure security validation can move beyond a static finding. It reads Terraform, targets controlled cloud emulators, asks self-hosted models to reason about red and blue paths, applies deterministic remediation, repeats the attack, and writes evidence a reviewer can inspect.

The acting models ran through ROCm and vLLM on an AMD Instinct MI300X. The endpoint was reached through a secure SSH tunnel rather than exposed publicly. The model improved reasoning and reporting, while deterministic code remained the source of truth for findings, fixes, and the final pass or fail verdict.

Measured architecture

A constrained red and blue loop around reproducible infrastructure.

The original Terraform source is never modified. Every run operates in an isolated workspace and preserves events, findings, attack code, remediation, metrics, and the final report.

  1. 01 Terraform input

    Read an IaC project and create an isolated run workspace.

  2. 02 Deterministic detection

    Identify supported cloud-storage exposure from configuration and plan evidence.

  3. 03 Local sandbox

    Target LocalStack-backed AWS or Azure-style infrastructure rather than production accounts.

  4. 04 Private model endpoint

    Call self-hosted red and blue models through an SSH-tunneled OpenAI-compatible vLLM API.

  5. 05 Constrained attack

    Execute only the generated run-directory attack script and preserve command evidence.

  6. 06 Deterministic repair

    Apply a versioned Terraform patch rather than accepting model output as the security verdict.

  7. 07 Replay and report

    Repeat the attack, record the outcome, and write structured evidence artifacts.

Proven today

AWS LocalStack replay and repository discipline

  • The public S3 path was live-validated against LocalStack.
  • The attack succeeded before remediation and was blocked after the deterministic Terraform fix.
  • Self-hosted Gemma 4 26B produced 104.45 tok/s (red) and 160.94 tok/s (blue) in the final AWS run.
  • CI, CodeQL, dependency review, security documentation, and release evidence are public.

Not claimed yet

Live Azure validation and broad production readiness

  • The Azure Blob path has deterministic finding and remediation evidence but not live emulator validation.
  • Kubernetes, Docker Compose, and on-prem targets remain scaffolded unless a page states otherwise.
  • The current benchmark does not yet execute Checkov, tfsec, Terrascan, or a human-review baseline.
  • This is an open-source alpha and should not be treated as a production security control.

Retained evidence

The screenshots support the story, but do not replace the run artifacts.

Open any image at full size. The repository remains the canonical source for the security model, technical walkthrough, run contracts, and current readiness limits.

nullstate CLI launch screen showing the autonomous purple-team sandbox workflow and next commands
CLI entry point The local-first workflow keeps sandbox status, execution, reports, and cleanup visible from one command surface.
Terminal evidence showing nullstate red and blue vLLM containers and AMD Instinct MI300X GPU information
MI300X and vLLM runtime Retained Docker and amd-smi evidence from the self-hosted red and blue model services.
nullstate Azure public blob offline run showing a finding before remediation and the attack path blocked after remediation
Azure-style deterministic run An offline deterministic Azure-style run. Live LocalStack Azure validation remains blocked by emulator entitlement and is not claimed.
GitHub pull request for nullstate scenario inference and sandbox probes with checks and verification commands
PR-gated engineering Repository work used reviewable branches, checks, verification commands, and explicit evidence requirements.

Security model

The model never receives an unrestricted shell.

The red model returns attack reasoning. nullstate executes only the generated run-directory attack.py through a constrained runner and records the command, stdout, stderr, return code, target, stage, and duration.

Production cloud access is out of scope by default. Model endpoints, LocalStack, cloud tokens, Terraform state, and environment files are not exposed as public services.

Evidence required before completion

What still has to be assembled

Source and release evidence

The demo is large and opens from the GitHub release rather than autoplaying on this page.

Repository Media release Project record Case-study index