Delay-Aware Reinforcement Coding for Multi-Hop Narrowband Wireless Mesh Networks: Joint RTT, Redundancy, and Forwarding Optimization

Abstract

Multi-hop narrowband wireless mesh networks remain useful for telemetry and supervisory control, but tail round-trip time (RTT) is strongly coupled to packet-loss probability, queue pressure, hop count, message size, retransmission, and coding overhead. In this paper Delay-Aware Reinforcement Coding (DARC), a finite Markov decision process controller that jointly selects fast-forward or store-and-forward relaying, random linear network coding (RLNC) generation size, and coded-packet redundancy, is proposed. The state includes four three-level features — end-to-end uncoded packet-loss probability, load/queue pressure, hop count, and message size — yielding 81 contexts and 12 actions (972 Q-values). DARC minimizes a fully normalized objective by combining empirical CVaR95 tail latency, residual loss, coding overhead, and goodput deficit. The load transition is action-dependent through a useful service rate, providing a long-horizon motivation for Q-learning; a contextual UCB bandit, a model-informed greedy controller, an adaptive heuristic, fixed policies, and an exact MDP oracle are included as baselines. Ten independent learning seeds are evaluated over 120,000 transitions with 95% confidence intervals. Under the exact discounted MDP benchmark, DARC achieves a normalized cost of 0.3364, 2.2% below the store-and-forward cost and within 0.29% of the exact oracle. In the representative five-hop, 12% end-to-end packet-loss condition, the learned policy selects uncoded fast-forward switching to preserve 12.25 kb/s goodput, while reliability-biased RLNC lowers p95 RTT at a substantial goodput cost. The results support DARC as a reproducible adaptive-control hypothesis; formal RF calibration remains pending the availability of raw traces.

Keywords

Graphical Abstract

Novelty Statement

DARC jointly adapts forwarding mode, RLNC generation size, and redundancy through an 81-state CVaR95-aware MDP, approaching oracle cost while preserving interpretability in narrowband multi-hop mesh networks.