Route every inference
to the right node.
Proximarun places each request on the node that meets its latency and cost target, learning traffic patterns to keep AI workloads fast without over-provisioning.
Static allocation wastes GPU budget on every off-peak hour.
You provisioned nodes for peak load. Most of the time, 60% sits idle. Then a surge hits and the queue builds anyway because your hardcoded endpoint points at one node, and it is saturated, and your script does not know about the other two.
More nodes do not fix this. A routing layer that knows each node's current load, latency, and cost, and that places each request accordingly, fixes this. That is what Proximarun does. Not a general-purpose load balancer. Specifically an inference router that understands latency budgets and cost ceilings.
// hardcoded endpoint, no fallback
const res = await fetch(
'https://gpu-node-01.internal/v1/chat',
{ method: 'POST', body: payload }
);
// node-01 saturated? you wait.
// node-02 idle? too bad.const result = await client.infer({
prompt: messages,
latencyBudgetMs: 400,
costCeilingUsd: 0.002
});
// Proximarun selects the node
// that fits both targets.Three steps from request to routed result.
Declare your SLA targets
Pass latencyBudgetMs and costCeilingUsd with each inference call. No config files to keep in sync.
Engine scores live nodes
The routing engine scores each healthy node on latency estimate, cost, and current load factor. Weighted sum picks the winner in microseconds.
Request dispatched, pattern learned
Request goes to the winning node. Response returns unchanged. Routing decision is logged, sharpening the pattern model for next time.
Six things the routing layer handles so you don't have to.
Latency-budget routing
Set p50, p95, and p99 targets. Proximarun only routes to nodes currently capable of hitting them, based on rolling latency history from the last 500 requests.
Cost-ceiling enforcement
Per-request cost caps. Spill to cheaper nodes when budget is tight, never silently over-spend.
Traffic pattern learning
After 7 days of live traffic, the engine adjusts node weights based on your actual request distribution, with a 48-hour exponential decay so recent behavior matters more than last month's.
Multi-backend support
vLLM, TGI, Triton, and any OpenAI-compatible endpoint in a single pool. No adapter layers, no format translation. The response schema your app already handles comes back unchanged.
Failover and health checks
Health endpoints polled every 5 seconds. A node that fails its check or exceeds its declared queue depth is pulled from the pool before the next request arrives. Re-admission requires two consecutive passing checks.
Real-time telemetry
Every routing decision logged with the full scoring context: which nodes were eligible, what each scored, the actual latency and cost. Scrape via Prometheus, push via webhook, or query the /v1/telemetry endpoint directly.
npm install @proximarun/clientimport { ProximaClient } from '@proximarun/client';
const client = new ProximaClient({
apiKey: process.env.PROXIMA_KEY
});const result = await client.infer({
prompt: messages,
latencyBudgetMs: 400,
costCeilingUsd: 0.002
});Replace one fetch() call. Keep every other line.
Install the client, initialize with your API key, and swap your hardcoded endpoint for client.infer() with inline SLA params. The response object is identical to what your backend already returns. We are not a new API format to learn; we are routing intelligence inserted in front of the one you already use.
Early access teams. Real infrastructure. Specific results.
"We were over-provisioning two A100 nodes for peak that hit maybe 3 times a week. Proximarun cut our idle spend by half without touching the SLA."
"The routing decision latency is genuinely under 10ms in practice. I was skeptical. It's real."
"One config file, one SDK call. We were routing across 4 different inference backends in an afternoon."
Stop paying for idle GPUs.
Get early access and route your first inference request in under 15 minutes.