Deep Reinforcement Learning Based Approach for Online Service Placement and Computation Resource Allocation in Edge Computing

Due to the urgent emergence of computation-intensive intelligent applications on end devices, edge computing has been put forward as an extension of cloud computing, to satisfy the low-latency requirements of these applications. To process heterogenous computation tasks on an edge node, the corresponding services should be placed in advance, including installing softwares and caching databases/libraries. Considering the limited storage space and computation resources on the edge node, services should be elaborately selected and deployed on the edge node and its computation resources should be carefully allocated to placed services, according to the arrivals of computation workloads. The joint service placement and computation resource allocation problem is particularly complicated, in terms of considering the stochastic arrivals of tasks, the additional latency incurred by service migration, and the waiting time of unprocessed tasks. Benefiting from deep reinforcement learning, we propose a novel approach based on parameterized deep Q networks to make the joint service placement and computation resource allocation decisions, with the objective of minimizing the total latency of tasks in a long term. Extensive simulations are conducted to evaluate the convergence and performance achieved by our proposed approach.