case study · September 2026
Mini ISE
Simulated devices ask for network access. A decision service answers allow, deny or quarantine. Admins manage the policies in a React console, where Claude turns a plain-English rule into a draft that a person has to approve before it can decide anything.
The split that matters
The tempting design is to let the model decide. Ask it whether this device, on this network, at this hour, should get access, and let it answer. It would often be right.
Three things make that the wrong shape. Access decisions get audited, which means the same inputs must produce the same answer next quarter as they do today. Access cannot depend on someone else’s API being reachable. And device data is attacker-influenced input, so putting it in a prompt hands an attacker a channel into the thing that decides whether they get in.
So the model drafts, a person approves, and from that point on plain Python decides. The language model is a drafting tool for the admin, not a participant in the request path. Everything the service does at request time is a first-match walk down an ordered list of policies that a human being signed off on.
The approval gate is the product
A draft only becomes a policy if it validates against the policy vocabulary: known fields, known operators, known actions. Anything the model invents is rejected before it reaches a human, and the console says why in plain language rather than showing a schema error. A rule the model cannot express, a catch-all for instance, comes back marked infeasible with a suggestion of what to ask for instead.
The same gate runs on the public site, so a visitor can type a rule, watch the draft come back, and watch the validator refuse the ones that do not hold up.
Keeping two engines honest
There are two implementations of the policy walk: Python in the cluster, and TypeScript in the browser so the site can demonstrate it without a backend. Two implementations of the same rules drift, and the drift stays invisible until someone is on stage.
One JSON fixture of policy cases is the shared contract, executed by both the pytest suite and the vitest suite. A rule change that lands on one side fails a test on the other. The prompt, the schema and the validation cases are shared the same way.
Failure behavior
The decision service holds the policy set in memory, refreshes it every three seconds, and writes its decision log in batches rather than per request. It therefore keeps deciding correctly while Postgres is unavailable: it stops seeing new policy changes and buffers its log, but it does not stop answering. Losing the database should degrade the audit trail, not the network.
What I would do next
Two things are unfinished. Policies are ordered by an integer and first match wins, which is how the real product behaves and also how people write rules that silently shadow each other, so a shadowing check at approval time is the obvious next guardrail. And the drafting endpoint runs on in-memory rate limits per instance, which is honest for a demo and wrong for anything more.