When we talk about how Proximarun routes requests, people often ask: is it a rule-based system or a machine learning system? The honest answer is that it is both, applied at different layers and for different purposes. The interesting design question is not which one is better in the abstract. It is understanding the failure modes of each well enough to know when to reach for which tool.
We have spent considerable time on this. Early versions of our routing engine were predominantly rule-based. We added learned policy components incrementally, and each addition came with a lesson about where learning helps and where it creates problems. This is our current thinking.
What Rules Are Good At
Rule-based routing is predictable and debuggable. When a request lands on Node B instead of Node A, you can trace exactly why: load factor on Node A exceeded the configured threshold of 0.8, so the routing rule excluded it from consideration. The decision is a deterministic function of observable state. You can reproduce it, explain it in a postmortem, and modify it with confidence about what changes.
Rules are also fast to deploy. When you need to route all requests tagged as batch-workload to cloud nodes and keep edge nodes reserved for interactive traffic, you write one rule and deploy it. No training data required. No waiting for a model to converge. The rule is live in under a minute.
Rules handle known, stable constraints especially well. Examples: "never route to a node in a region that is under a geographic compliance restriction," "always apply a minimum reliability score threshold before considering a node," "if a node has not sent a heartbeat in 30 seconds, exclude it." These constraints do not benefit from learning. They are invariants, and expressing them as rules makes the invariant explicit and auditable.
Where Rules Break Down
Rules require you to specify the decision boundary ahead of time. When you write a rule like "if load factor exceeds 0.8, reduce routing weight by 50%," you have decided that 0.8 is the right threshold and 50% is the right adjustment. Both of those choices may be wrong for your specific node profile, or for a specific traffic pattern, or during a specific time of day.
The deeper problem is that rule thresholds interact. If you have 15 routing rules each with their own thresholds, the emergent behavior of the combined rule set is hard to reason about, especially under unusual traffic conditions. You end up with rules that contradict each other, rules that only trigger together under specific combinations of node states, and edge cases that no individual rule covers.
We see this pattern frequently in teams that have been maintaining hand-tuned routing configurations for 6-12 months. The config file is dense with conditionals. Nobody is confident what happens under load spikes because the combined rule behavior has never been tested at those states. Rules work well when there are few of them; they become brittle as the rule set grows.
What Learned Policies Add
Learned policies can discover relationships that are hard to express as rules. The classic example in inference routing is the relationship between prompt token count, model size, and expected queuing delay at different load levels. Writing a rule that captures this three-way interaction requires significant manual calibration per node type. A learned policy can discover this relationship from observed data without you having to specify the functional form.
Proximarun's traffic pattern learning works on a rolling window of request outcomes. It adjusts routing weights in the direction that improves P99 latency given the observed request distribution. If your traffic has a strong diurnal pattern (heavier load in the afternoon, lighter at night), the learned policy picks up on that pattern and pre-adjusts routing weights ahead of the ramp, rather than reacting after load factor thresholds are triggered.
That anticipatory adjustment is the core value proposition of learned routing over reactive rules. Rules respond to the state they observe now. A learned policy can respond to the state it predicts is coming in the next few minutes based on historical patterns.
Where Learned Policies Fail
Learned policies fail silently in ways that rules do not. When a routing rule misconfigures, you usually see an error or obviously wrong behavior that is traceable. When a learned policy misconfigures, it may behave plausibly for most traffic while making systematically poor decisions for a minority of request types. The damage is harder to detect.
Distribution shift is the specific failure mode we watch for. A learned policy trained on your traffic from the last two weeks is optimized for that traffic pattern. If your traffic pattern changes suddenly (a new product launch, a change in how your application calls the inference API, a shift in prompt length distribution), the policy's decisions may be incorrect for a window of time until it re-adapts. Rules do not have this problem: they apply as-configured regardless of whether the current traffic pattern matches the pattern you designed them for.
We are not saying learned policies are unreliable. We are saying they require monitoring for distribution shift in ways that rule-based systems do not. If you cannot tell when your traffic pattern has changed, you cannot tell when your learned policy may have drifted from optimal.
The Architecture We Landed On
Proximarun uses rules for constraints and learned policies for weights. The distinction: rules define what is not allowed (compliance, hard exclusions, safety rails). Learned policies determine how to allocate within the allowed set (which nodes get what fraction of traffic, how to weight the scoring signals for the current traffic pattern).
This layered architecture means the safety properties are preserved regardless of what the learned policy does. If the learned policy starts over-weighting a node for reasons we do not understand, the hard rule layer still prevents it from sending traffic to an excluded node, still enforces minimum reliability thresholds, still respects compliance boundaries.
Within the allowed set, learned policies have freedom to optimize. They are operating in a bounded space, not an unbounded one. The combination is more robust than either approach alone: you get the predictability of rules where it matters and the adaptability of learning where it helps.
The practical implication for teams using Proximarun: if you have constraints that are truly invariant ("never route to Node X"), express them as rules. If you have preferences that should adapt to traffic conditions ("prefer Node Y but not always"), let the learned policy handle the weight allocation. Mixing them by writing rules that try to emulate what a learned policy would do anyway tends to produce configurations that are hard to maintain and easy to mis-tune.