With a gateway
Use existing provider access and resilience where it meets your needs. Place Punk where governed observation and verified reuse belong in the request path.
// infrastructure comparison
Gateway, observability, or adaptive runtime?
A gateway moves and controls AI traffic. Observability helps teams inspect what happened. An adaptive runtime uses governed evidence from prior execution to improve what can run next. Many production stacks need more than one layer.
Short answer
Choose by job, not label: access and traffic control; visibility and evaluation; or evidence-gated adaptation. Product boundaries differ, so verify the capabilities of each vendor you evaluate.
// primary jobs
Similar inputs. Different responsibilities.
| Layer | Primary question | Typical capabilities | Important boundary |
|---|---|---|---|
| AI gateway | How should this model request reach a provider? | Compatible APIs, credentials, provider routing, retries, limits, and sometimes caching | A gateway does not inherently learn reusable multi-step execution |
| Agent observability | What happened, and was it acceptable? | Traces, spans, evaluations, dashboards, datasets, alerts, and debugging | Visibility does not inherently change the next execution path |
| Adaptive runtime | What has earned a better way to run? | Observation, policy, pattern learning, replay, shadow comparison, reusable routes, fallback, explanations | Adaptation is only as good as its scope, evidence, policies, and outcome signals |
// how Punk fits
Keep the layers that already work.
Punk includes an OpenAI- and Anthropic-compatible traffic surface, but its defining job is the adaptive loop around execution. It can complement a gateway or observability platform instead of forcing a wholesale replacement.
With observability
Keep deep trace analysis and evaluation workflows. Use Punk to connect eligible production evidence to a future route.
On its own
For a narrow workload, compatible model traffic can begin in observe-only mode. Richer tool and outcome context requires corresponding integration.
// evaluation questions
Ask every vendor the same hard questions.
Execution
- What can change a live response?
- What happens when the system is uncertain or unavailable?
- Are optimized routes scoped by tenant, app, and subject?
- Can every served response explain its route?
Evidence
- How are candidates tested against historical traffic?
- Can fresh traffic be compared without serving the candidate?
- How are mismatches classified?
- Are forecast and realized savings reported separately?
// frequently asked
Common questions.
Does an adaptive runtime replace observability?
No. Teams may still need dedicated trace exploration, evaluation, alerting, and debugging. The adaptive runtime's distinct role is to use bounded evidence to govern and improve future execution.
Is Punk also an AI gateway?
Punk exposes compatible gateway surfaces for model traffic and routing, but “adaptive runtime” describes the broader product: observation, governance, learning, verification, reuse, fallback, and route explanation.
Can one endpoint change integrate a full agent?
It can redirect compatible model traffic. Complete tool traces, identity, outcomes, browser actions, or approvals may require SDK, header, connector, or application-level integration.
Add the layer your stack is missing.
Start with observe-only model traffic or bring one narrow workload to the Punk team.