Project Overview: 

Next-generation cellular wireless edge networks with embedded computational capabilities and services have become a critical infrastructure that enables secure, robust, and high-performance applications in many domains, including education, business, transportation, healthcare, and entertainment. Meanwhile, the availability, reliability, and resiliency of edge networks are constantly being challenged by both the expected resource and demand variations, e.g., diurnal application traffic patterns, user mobility, and random link/node failures, as well as the unexpected level-shifts of operating conditions, e.g., traffic flash-crowds triggered by emerging events, major infrastructure failures after coordinated malicious attacks and natural disasters.

The project is developing novel hybrid learning solutions for autonomous and resilient wireless edge networks from the angle of joint provisioning, allocation, and scheduling of communication and computation resources. It includes several research thrusts: 1) To achieve high efficiency and robustness in the face of expected demand/resource variations, the project team is investigating robust communication and computation resource provisioning at long time scales. At short time scales, the team is studying adaptive routing under the Model Predictive Control framework coupled with Adaptive Dynamic Programming (ADP). The researchers are also investigating adaptive scheduling of virtual middle-boxes using a domain-knowledge enriched reinforcement learning framework. 2) To recover from major disruptions, the project team is studying how to progressively bring network services back by strategically utilizing backup and external resources. Resilient progressive recovery is achieved through joint provisioning, routing, and scheduling based on hybrid online learning. The stability of the recovery process is analyzed under the Robust ADP framework. 3) Individual research components are being evaluated using mathematical analysis, trace-driven simulations, and experiments on the CloudLab testbed. The project is generating new theories and algorithms on hybrid learning that can be used to study complex systems in other domains. Valuable research opportunities are being created for graduate and undergraduate students, especially women and minority students.

Presentation

People:

• Yong Liu (Principal Investigator, NYU)
• Shivendra Panwar (Co-Principal Investigator, NYU)
• Pei Liu (Co-Principal Investigator, NYU)
• Zhong-Ping Jiang (Co-Principal Investigator, NYU)
• Jian Li (Co-Principal Investigator, Stony Brook University)

Funding:
      
This project is funded by the USA National Science Foundation as part of the RINGS program, under contract Award # 2148309

Publication:  

• “On Scalable Multi-flow and Multi-channel Traffic Steering through Hybrid Learning”,
  Xiaotian Li, Yong Liu, Shivendra Panwar, and Shu-ping Yeh
  ACM SIGMETRICS Performance Evaluation Review 53 (2), 45-50, 2025
• DeeP-TE: Data-enabled Predictive Traffic Engineering
  Zhun Yin, Xiaotian Li, Lifan Mei, Yong Liu, and Zhong-Ping Jiang
  arXiv preprint arXiv:2508.14281, 2025 [Paper]
• Robust Lyapunov Optimization for LEO Satellite Networks Routing Control
  Zhemin Huang, Zhong-Ping Jiang, Zhu Han, and Yong Liu
  IEEE Transactions on Mobile Computing, July 2025,
  Conference Version presented in 12th IEEE International Conference on Wireless for Space and Extreme Environments (WiSEE 2024)
• Decentralized Federated Learning with Model Caching on Mobile Agents
  Xiaoyu Wang, Guojun Xiong, Houwei Cao, Jian Li, and Yong Liu
  Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2025
  [Paper] [Interview]
• On routing optimization in networks with embedded computational services
  Lifan Mei, Jinrui Gou, Jingrui Yang, Yujin Cai, and Yong Liu
  IEEE Transactions on Network and Service Management, Volume: 22, Issue: 1, February 2025
  [Paper]
• “Online Restless Multi-Armed Bandits with Long-Term Fairness Constraints”,
  Shufan Wang, Guojun Xiong and Jian Li,
  Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI), Vancouver, Canada, February 2024.  [Paper]
• “Fast Computation Flow Restoration with Path-based Two-stage Traffic Engineering”,
  Xiaotian Li and Yong Liu,
  ACM/IEEE Symposium on Edge Computing (SEC), Best Paper Shortlist, December, 2023, [Paper]
• “Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation”,  
  Guojun Xiong and Jian Li,
  Proceedings of the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), New Orleans, LA, December 2023. [Paper]
• “Oblivious Routing Using Learning Methods”,
  U. Usubutun, M. Kodialam, T.V. Lakshman and S. Panwar, 
  in IEEE Global Communications Conference (Globecom), December, 2023, [Paper], [Slides]
• “Discrete-Time Distributed Optimization for Linear Uncertain Multi-Agent Systems”, 
  Tong Liu, Michelangelo Bin, Ivano Notarnicola, Thomas Parisini, Zhong-Ping Jiang,
  62nd IEEE Conference on Decision and Control (CDC), Marina Bay Sands, Singapore, December 2023.
• “Reinforcement Learning for Dynamic Dimensioning of Cloud Caches: A Restless Bandit Approach”,
  Guojun Xiong, Shufan Wang, Gang Yan and Jian LI, 
  IEEE/ACM Transactions on Networking (IEEE/ACM TON), vol. 31, no. 5, pp. 2147-2161, October 2023,[Paper]
• “A reinforcement learning look at risk-sensitive linear quadratic Gaussian control”,
  L. Cui, T. Basar and Z. P. Jiang,
  Proceedings of Machine Learning Research, 2023.
• “Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approach”, 
  L. Cui, B. Pang and Z. P. Jiang,
  in IEEE Transactions on Automatic Control, 2023
• “Learning Infinite-Horizon Average-Reward Restless Multi-Action Bandits via Index Awareness”,
  Guojun Xiong, Shufan Wang and Jian Li,
  Proceedings of the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), New Orleans, LA, November 2022, [Paper]
• “Index-Aware Reinforcement Learning for Adaptive Video Streaming at the Wireless Edge”,
  Guojun Xiong, Xudong Qin, Bin Li, Rahul Singh and Jian Li,
  Proceedings of ACM Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing (ACM MobiHoc), Seoul, South Korea, October 2022, [Paper]