When we say Proximarun "learns your traffic patterns," that phrase needs to be made concrete or it sounds like marketing language. This post explains the actual mechanism: what data we collect per routing decision, how we update routing weights over time, what the learning rate looks like in practice, and how the system handles sudden shifts in traffic shape that invalidate what it has learned.
What Gets Observed
Every routing decision Proximarun makes produces a telemetry record. That record contains the node selected, the latency target passed in, the actual latency observed on completion, the estimated cost at dispatch time, the actual cost on completion, and a timestamp. Over a rolling window of the last 500 requests (configurable), we maintain an aggregate view per node of: median observed latency, p95 observed latency, cost accuracy (how often our estimate was within 10% of actual), and request success rate.
We also track temporal distribution. Not just "node A has a median latency of 220ms" but "node A has a median latency of 220ms between 9am and noon UTC, and 180ms from midnight to 6am." That temporal dimension is what separates useful pattern learning from a simple moving average.
We are not doing anything exotic here. This is not a neural network or an RL agent. It is a weighted scoring function whose weights update based on recent observed outcomes. The sophistication is in the window management and the decay function, not in the base algorithm.
The Decay Function
When a new observation comes in, we do not simply replace old observations with new ones. We apply exponential decay, which means recent observations have more influence on current weights than older ones, but older observations do not disappear instantly. The decay half-life defaults to six hours, meaning an observation from six hours ago has half the weight of an observation from right now. An observation from 24 hours ago has about 6% of current weight.
This matters for two reasons. First, it prevents the system from over-indexing on a single unusual event. If a node had a temporary latency spike at 2am, one bad observation should not tank its score for the whole day. Second, it ensures the system adapts to sustained changes reasonably quickly. If a node's latency has genuinely degraded, the decay means the improved historical scores age out and the current reality dominates within a few hours.
The half-life can be configured. Six hours works well for traffic that has a consistent daily pattern. If your traffic distribution changes more rapidly, a shorter half-life makes the routing more reactive at the cost of higher variance. If your traffic is very stable with infrequent changes, a longer half-life reduces noise. We settled on six hours as the default after observing that most workloads we have seen have meaningful diurnal structure.
How Weights Get Updated
The base scoring formula for a routing decision is: latency score (weight 0.4) + cost score (weight 0.35) + reliability score (weight 0.25). Those base weights are fixed. What learning changes is the inputs to that formula.
Specifically, the latency estimate for a node is not a static configured value. It is the exponentially weighted moving average of observed latencies for that node over the rolling window, segmented by time-of-day bucket. The cost estimate is updated similarly based on actual versus estimated cost over the window. The reliability score is updated based on success rate over the window.
This means the system starts with initial estimates that you configure at setup time, then gradually replaces those estimates with empirically observed values as traffic flows through. In the first few hundred requests, you are largely relying on your initial configuration. By the time you have seen a few thousand requests across a typical traffic day, the estimates are primarily data-driven.
The Cold Start Period
The first 24-48 hours after setup are the most important to understand. The system has no learned estimates yet, so it falls back to your configured values and then to uniform distribution if no estimates are configured. During this period, routing is essentially round-robin with a light bias toward whatever latency and cost estimates you provided at setup.
We recommend that teams configure reasonably accurate initial latency and cost estimates when they register their nodes. You do not need exact numbers. An order-of-magnitude estimate of your typical latency and your hourly node cost gets you to a sensible starting state. The learning will correct inaccuracies quickly, but wildly wrong initial estimates can cause the cold start period to route poorly for longer than necessary.
We are not saying the cold start period is a serious problem. For most teams, the learning converges to good routing behavior within 24-72 hours. We are saying it is worth being aware of if you are evaluating behavior immediately after integration.
What Happens When Traffic Shifts Suddenly
The scenario that most concerns teams hearing about pattern learning is a sudden traffic shift. Imagine you launch a new feature that doubles request volume, or you change your prompt structure in a way that significantly increases average context length. The routing weights the system has learned are now calibrated to a pattern that no longer applies.
The decay function handles gradual shifts well. The weights adapt at a rate proportional to how much the current observations deviate from the recent history. A 20% increase in latency on a node will move the weights noticeably within an hour or two.
For abrupt large shifts, the system transitions through a brief period of suboptimal routing while the new pattern accumulates enough observations to dominate the weighted history. With a six-hour half-life, a pattern that has been stable for 24 hours will take two to four hours before new observations substantially override it. During that window, you might see some routing decisions that would have been better calibrated to the new reality.
There are two mitigations built in. First, the health check system runs independently of pattern learning. If a node is genuinely overwhelmed and failing health checks, it gets pulled from the routing pool immediately, regardless of its historical score. Second, you can manually trigger a weight reset from the API if you know a major traffic pattern change is coming. A reset drops accumulated history and puts the system back into the initial estimation mode, which adapts more aggressively to new observations. We use this ourselves when we know a significant behavioral change is planned.
An Honest View of the Limitations
Pattern learning works well for traffic that has repeatable structure across days: workloads with a consistent diurnal cycle, consistent request type distribution, and nodes whose latency characteristics are relatively stable. For workloads that are highly irregular or bursty in ways that do not repeat, the learned weights may not be meaningfully better than well-configured static weights.
We also want to be clear that this is not a forecasting system. The routing engine observes what has happened and uses that to inform current decisions. It does not predict future load or pre-emptively adjust node allocation. Capacity planning and autoscaling are upstream of routing. What pattern learning does is make the allocation you have work more efficiently, not tell you how much allocation you need.
The practical result for teams with typical workloads is routing that gets noticeably better over the first week and then maintains that quality with low maintenance overhead. The learning does not keep dramatically improving after the initial convergence period. It stabilizes to match the current pattern and updates as that pattern evolves.