February 2021 Issue
Research Highlights

Computing in close proximity: Edge intelligence with deep reinforcement learning

Mobile edge computing (MEC) is a promising paradigm to improve the quality of computation experience for mobile devices by providing computing capabilities in close proximity. MEC finds applications in homes, factories, and transport modes including trains and airplanes. However, the design of computation offloading policies for an MEC system, specifically, the decision of executing a computation task at the mobile device or at the remote MEC server, should adapt to the network randomness and uncertainties.

Now, Celimuge Wu at the University of Electro-Communications, Tokyo and colleagues in Finland, USA, and China, report on the Deep-SARL, a double deep Q-network (DQN)-based online strategic computation offloading algorithm to learn the optimal policy without knowing a priori knowledge of network dynamics (Fig. 1).

The computation offloading problem is modeled as a Markov decision process, where its objective is to maximize the long-term utility performance whereby an offloading decision is made based on the task queue state, the energy queue state, and the channel qualities between mobile users and base stations. The researchers describe the adoption of a Q-function decomposition technique to enhance the learning performance.

Numerical experiments based on TensorFlow show that their proposed learning algorithm achieves a significant improvement in computation offloading performance compared with existing baselines, showing an optimal tradeoff among the computation task execution delay, task drops, task queuing delay, task failure penalty, and MEC service payment. Deep-SARL provides a novel and effective approach to facilitate intelligence in edge computing under time-varying network dynamics.

figure
Fig. 1 Deep-SARL-based strategic computation offloading in an MEC system.

References

author
  • Xianfu Chen, Honggang Zhang, Celimuge Wu, Shiwen Mao, Yusheng Ji, Mehdi Bennis, "Optimized Computation Offloading Performance in Virtual Edge Computing Systems via Deep Reinforcement Learning," IEEE Internet of Things Journal, Vol.6, no.3, pp. 4005-4018, June 2019. DOI: 10.1109/JIOT.2018.2876279