Quickstart

This guide gets you from zero to a working routed inference call in under 15 minutes. You will install the client SDK, define a two-node pool in a YAML config, make your first client.infer() call with explicit SLA targets, and verify the routing telemetry. Prerequisites: Node.js 18+ or Python 3.9+, an active Proximarun account, and at least one inference backend endpoint (vLLM, TGI, Triton, or any OpenAI-compatible API).

Step 1: Install the SDK

Install the official Proximarun client library:

npm install @proximarun/client

Or for Python:

pip install proximarun

Step 2: Get your API key

After creating your account, navigate to Settings in the Proximarun dashboard and generate an API key. Store it as an environment variable:

export PROXIMA_KEY="prx_live_..."

Step 3: Configure your first node pool

Create a proxima.yaml file in your project root. This file defines the inference nodes Proximarun will route across:

# proxima.yaml
nodes:
  - id: primary-edge
    endpoint: https://your-edge-node.internal/v1
    backend: vllm
    costPerToken: 0.0000012
    latencyClass: fast
    standby: hot

  - id: cloud-backup
    endpoint: https://api.openai.com/v1
    backend: openai-compat
    costPerToken: 0.000015
    latencyClass: standard
    standby: hot

routing:
  defaultLatencyBudgetMs: 500
  defaultCostCeilingUsd: 0.005

Step 4: Initialize the client

In your application code, initialize the Proximarun client. The client reads your proxima.yaml automatically if it is present in the current working directory:

import { ProximaClient } from '@proximarun/client';

const client = new ProximaClient({
  apiKey: process.env.PROXIMA_KEY,
  configPath: './proxima.yaml'
});

Step 5: Make your first routed call

Replace your existing hardcoded inference call with a routed call. Pass your SLA targets inline:

const result = await client.infer({
  prompt: [
    { role: 'user', content: 'Summarize this document...' }
  ],
  latencyBudgetMs: 400,
  costCeilingUsd: 0.002
});

console.log(result.content);
console.log('Routed to:', result.meta.nodeId);
console.log('Latency:', result.meta.latencyMs, 'ms');

Step 6: Verify telemetry

After your first successful call, you can verify the routing decision was logged by querying the telemetry endpoint:

curl -H "Authorization: Bearer $PROXIMA_KEY" \
  https://api.proximarun.com/v1/telemetry?limit=1

You should see a JSON response containing the routing event with node selection, scores, latency, and cost fields. This confirms the routing engine is operating correctly.

Next steps

  • Read the Concepts guide to understand how routing policy weights, spill logic, and the traffic pattern learning model interact in production.
  • Explore the API Reference for the full parameter set on /infer, including reliabilityMin and allowColdStandby options.
  • Configure Prometheus scraping at /metrics or set a webhook endpoint to receive per-decision telemetry events before you put production traffic through the router.