Energy-Aware Service Mesh Deployment and Online Request Routing in Edge: A Hierarchical Deep Reinforcement Learning Approach

Service meshes built upon ubiquitous microservice architectures, as an emerging paradigm, promise to enhance the flexibility, scalability, and portability of energy-consuming and latency-sensitive applications in edge with limited resources. However, due to intricate microservice dependencies, service multiplexing, and parallel distributed instances, microservice deployment and request routing are highly interdependent. To reduce response latency and energy consumption, such collaborative optimization for efficient service mesh orchestration is necessary, but significantly challenging. Besides, strict service level objective (SLO) requirements and f ine-grained latency analysis with multi-nest routing further impose great difficulties to online orchestration. When considering multi instance modeling and multi-hop data communications for numerous microservices, the difficulty is extremely amplified. Nevertheless, most prevailing work failed to design sophisticate models and methods for addressing the above difficulties, and ignored the inherent transmission energy consumption for highly-concurrent multi-hop data interactions. Therefore, this paper investigates the energy aware service mesh deployment and online request routing in edge. First, we establish a multi-instance queuing network model to accurately analyze the end-to-end response latency with complicated dependencies and multi-hop communications, and optimize energy consumption in a fine-grained manner. Then, to boost the overall performance, we design an efficient multi-dimensional hierarchical deep reinforcement learning algorithm, which enables edges and service instances to cooperate with each other to handle massively concurrent requests. Besides, we propose an energy-aware proactive autoscaling algorithm to carefully adapt to exceedingly dynamic scenarios. Finally, extensive experiments are performed to show our superior performance compared to other baselines.