Response reuse
Return a prior answer only inside relevant tenant, app, subject, and freshness boundaries. Semantic candidates remain shadow-only when the plan or evidence does not allow serving.
// from traffic to proof
Observe first. Change nothing.
Point one compatible agent at Punk with an observe-mode key. The live provider still answers. Punk records the run, maps the opportunity, and shows what deserves deeper testing.
Observe mode estimates opportunities as ghost savings. It does not count them as realized savings or serve an optimized response.
// the first loop
One workload. Five decisions.
A good first workload repeats, produces reviewable output, and has low or controlled side effects. Support classification, internal record summaries, and read-only research are practical starting points.
Connect one agent
Use the OpenAI-compatible endpoint, Anthropic-compatible endpoint, or Punk SDK. Add stable app, agent, and pseudonymous subject identity so caches and evidence stay within safe boundaries.
Keep the live provider in control
Observe mode serves the configured live model while recording costs, latency, route alternatives, policy results, and tools. The production answer does not come from a shortcut.
Find repeated work
Punk looks for identical requests, semantically similar requests, repeated read-only tool calls, stable tool plans, and structured outputs that may be expressible as a deterministic workflow.
Prove a candidate
Replay checks the candidate against historical traces. Shadow runs it beside fresh live traffic without serving it. Side effects are suppressed or dry-run during both.
Choose the next mode
Stay observe-only, gather more evidence, canary one low-risk route, or stop. Novel, ambiguous, stale, or policy-sensitive requests continue to the live provider.
// route choices
Reuse only what has earned reuse.
Work reuse
Reuse a safe read or a stable plan while still fetching fresh data when freshness matters. Writes are never treated like ordinary cached reads.
Workflow reuse
Execute a declarative tested workflow, or combine deterministic fields with a live-model fallback for the parts that still need judgment.
// what proof means
Bounded evidence, not a blanket guarantee.
Proof is scoped to a workload and route. It establishes what was tested, against which traffic, under which policy, and with what mismatches.
| Gate | Question | Evidence |
|---|---|---|
| Replay | Would the candidate have matched acceptable historical behavior? | Case count, passes, skips, mismatch severity, structural or semantic checks. |
| Shadow | Does it still match current live behavior? | Fresh sample count, agreement, high-severity mismatches, schema and policy state. |
| Policy | May this route run for this tenant, app, agent, subject, and action? | Policy version, verdict, approval state, side-effect level. |
| Operations | Can the route fail safely? | Fallback reason, canary scope, rollback path, route explanation. |
// compatible by design
The first connection is deliberately small.
const client = new OpenAI({
baseURL: "https://app.punktechnologies.com/v1",
apiKey: process.env.PUNK_API_KEY,
defaultHeaders: {
"X-Punk-App": "support-triage",
"X-Punk-Agent": "triage-bot",
"X-Punk-Subject": accountId
}
});Compatibility has a boundary. A base URL change covers compatible model traffic. Tool tracing, side-effect declarations, feedback, semantic web sessions, and deeper learning surfaces may require the SDK or additional integration.
Get the answer before changing behavior.
Start observe-only and review the opportunity, evidence gaps, and next decision in one report.