Collaborative Orchestration of Microservices and AI Services in Edges: A Dual-Time-Scale Reinforcement Learning Approach

The rapid development of service computing has led to the emergence of scalable and flexible architectures such as microservices and Artificial Intelligence as a Service (AaaS), enabling the orchestration of AI-driven intelligent applications. However, existing work on intelligent applications orchestration overlooked essential microservice components that support AI services, resulting in coarse-grained and incomplete models. To ensure system integrity and enhance QoS, fine-grained collaborative orchestration of microservices and AI services is crucial. However, this poses significant challenges due to complex service dependencies, high request concurrency, and heterogeneous resource demands in edge environments. Moreover, the strong coupling between service deployment and request routing complicates their joint optimization, since effective decisions in one depend on the other. To address these challenges, we propose a collaborative orchestration framework that jointly optimizes the deployment of microservices and AI services along with probabilistic request routing in edge environments. We formulate the problem as a mixed-integer nonlinear program and leverage Jackson queuing networks for accurate delay modeling. To solve this, we develop a dual time scale hybrid greedy proximal policy optimization (DTS-HGPPO) algorithm that performs instance-level deployment and adaptive routing, enhanced with iterative instance planning, action masking and intrinsic motivation mechanisms. Extensive trace-driven experiments demonstrate that our method significantly reduces both response delay and service cost compared to state-of-the-art baselines.