Evaluating Phalanx
Follow the action all the way to its outcome.
To judge an execution-control system, trace one action end to end: what the agent proposed, what authority it had, what the gateway decided and what actually happened. Phalanx records each stage, so the boundary can be examined, not just described.
Five questions for a meaningful evaluation.
| Question | What to inspect |
|---|---|
| Who authorized the task? | The authenticated actor, task scope, allowed actions, and limits supplied by the customer application. |
| What action was proposed? | The exact tool, target, destination, arguments, and relevant system state. |
| Why did it proceed or stop? | The recorded PERMIT, HOLD, or BLOCK decision and the applicable policy or unresolved condition. |
| Could it execute another way? | Credential isolation, and whether any other route can perform the same action without Phalanx. |
| What happened afterward? | The connector result, the verification outcome, and any uncertainty or partial completion, as recorded in the receipt. |
Test useful work and the conditions that should stop it.
A useful evaluation includes ordinary authorized actions as well as the boundaries around them. These are evaluation questions, not published pass-rate claims.
| Scenario | Required observation |
|---|---|
| An allowed action with valid state | It can proceed through the protected route and its outcome is recorded. |
| An action outside the task's resource or destination scope | It receives no execution permit. |
| An action or shared workflow limit has been consumed | Recorded usage remains part of the decision for later calls. Participating tools in a configured shared workflow draw from the same task budget. |
| Approval is required | The action stays on hold until the required authorized resolution and re-evaluation. |
| Required enrolled state is absent or unavailable | The action fails closed under its enrolled contract without a silent permissive fallback. |
| State changes before execution | Required execution-time checks detect the relevant change and apply the contract. |
| A permit or operation is repeated | The test distinguishes rejected permit reuse from connector idempotency and uncertain-outcome handling. |
| An external system times out | The result is recorded as known or uncertain according to verification evidence; a timeout is not reported as success. |
| The conversation names a different customer | Phalanx uses the identity your application signed. A customer ID typed into the chat can't change which resource the action touches. |
What we tested before releasing Phalanx 1.9.
Production release 1.9.0 was approved on 28 September 2026 for Linux/Docker and Render, after four release gates: specify, implement, qualify the signed artifacts, and deliver the exact accepted build.
| Test | Result |
|---|---|
| Source and security test suite | 2,169 cases passing, plus schema checks and SDK build and tests |
| Boundary tests on the installed release | 32 cases covering receipt chains, durable activation and workflow operations, uncertain connector execution, and backup and recovery |
| Upgrade and rollback | Upgraded from preserved 1.5.1 state, and rolled back to the matching earlier image, with keys and receipt history intact |
| Multi-tool state rehearsal | 17 integrations in one shared workflow, with 212 existing receipts and unchanged keys |
| Receipt verification at scale | Oldest, middle and newest receipts verified in a 99,990-receipt chain with one million authenticated events, inside the 120-second verification window, at 0.5 CPU and 512 MB |
| Live delivery | Customer CLI updated and downloaded from production; the Meridian demo upgraded in place with all 17 integrations reactivated, and a receipt-bound protected read served through its chatbot |
These are release-engineering results, not an attack-blocking rate or a success rate for every deployment. Failure cases were injected in isolated tests of the installed release, not run against the public demo. Every customer deployment is qualified on its own routes.
Keep the result attached to the system that produced it.
A meaningful result names the Phalanx release, the protected workflow, the connector, the authority and policy configuration, the test conditions and the observed outcome. Customer qualification also checks the infrastructure and routes that make the boundary enforceable.
Meridian, the public demo, is a synthetic business operated by Invarra. Its introduction lists the active Phalanx controls. A customer evaluation establishes coverage for the workflow actually being deployed.
Earlier research.
Before Phalanx became an execution gateway, it screened written chat for jailbreaks and prompt injection. Those results concern that earlier runtime and different test contracts. They are kept as a historical record and are not performance results for Phalanx 1.9.
Decide what success must look like in your environment.
Start with an action that should work, a boundary that must hold and an outcome you need to verify. We'll scope an evaluation around those three.