Minghao Ye @ NYU

[IWQoS ’23] QoS-RL: QoS-Aware Traffic Engineering with Reinforcement Learning

by

One of our recent works entitled “Reinforcement Learning-based Traffic Engineering for QoS Provisioning and Load Balancing” was accepted by the 31st IEEE/ACM International Symposium on Quality of Service (IWQoS 2023) on 3/30/2023. Congratulations!

In this work, we developed a Reinforcement Learning (RL)-based traffic engineering solution to provide good Quality of Service (QoS) for high priority traffic while maintaining promising load balancing performance in the network by rerouting a small portion of low priority traffic with low management overhead. This work was presented at the IWQoS conference in Orlando, FL, USA, on 6/19/2023.

Project Overview:

(1) TE requirements for different applications: QoS provisioning + load balancing

(2) Our idea: Categorize traffic into different priority levels for routing optimization
(3) Apply destination-based routing to forward high/low priority traffic with simplified routing updates
(4) Leverage intelligent RL to select a few forwarding entries for routing updates: Achieve good performance with efficient TCAM usage + accelerated TE optimization + reduced management overhead
(5) Evaluation results: Close-to-optimal delay performance + Optimal load balancing performance

Abstract: 

Emerging applications pose different Quality of Service (QoS) requirements for the network, where Traffic Engineering (TE) plays an important role in QoS provisioning by carefully selecting routing paths and adjusting traffic split ratios on routing paths. To accommodate diverse QoS requirements of traffic flows under network dynamics, TE usually periodically computes an optimal routing strategy and updates a significant number of forwarding entries, which introduces considerable network operation management overhead.

In this work, we propose QoS-RL, a Reinforcement Learning (RL)-based TE solution for QoS provisioning and load balancing with low management overhead and service disruption during routing updates. Given the traffic matrices that represent the traffic demands of high and low priority flows, QoS-RL can intelligently select and update only a few destination-based forwarding entries to satisfy the QoS requirements of high priority traffic while maintaining good load balancing performance by rerouting a small portion of low priority traffic.

Extensive simulation results on four real-world network topologies demonstrate that QoS-RL provides at least 95.5% of optimal end-to-end delay performance on average for high priority flows, and also achieves above 90% of optimal load balancing performance in most cases by updating only 10% of destination-based forwarding entries.

Publications: 

  • [IWQoS ’23] Minghao Ye, Yang Hu (co-first author), Junjie Zhang, Zehua Guo, and H. Jonathan Chao, “Reinforcement Learning-based Traffic Engineering for QoS Provisioning and Load Balancing,” The 31st IEEE/ACM International Symposium on Quality of Service (IWQoS), 2023. (Acceptance rate: 23.5%, 62/264) [Paper URL] [PDF]