Campus Units

Electrical and Computer Engineering

Document Type

Conference Proceeding


28th International Conference on Computer Communication and Networks (ICCCN)

Publication Version

Accepted Manuscript

Link to Published Version

Publication Date


Journal or Book Title

2019 28th International Conference on Computer Communication and Networks (ICCCN)



Conference Title

28th International Conference on Computer Communication and Networks (ICCCN)

Conference Date

July 29-August 1, 2019


Valencia, Spain


Energy harvesting communications systems are able to provide high quality communications services using green energy sources. This paper presents an autonomous energy harvesting communications system that is able to adapt to any environment, and optimize its behavior with experience to maximize the valuable received data. The considered system is a point-to-point energy harvesting communications system consisting of a source and a destination, and working in an unknown and uncertain environment. The source is an energy harvesting node capable of harvesting solar energy and storing it in a finite capacity battery. Energy can be harvested, stored, and used from continuous ranges of energy values. Channel gains can take any value within a continuous range. Since exact information about future channel gains and harvested energy is unavailable, an architecture based on actor-critic reinforcement learning is proposed to learn a close-to-optimal transmission power allocation policy. The actor uses a stochastic parameterized policy to select actions at states stochastically. The policy is modeled by a normal distribution with a parameterized mean and standard deviation. The actor uses policy gradient to optimize the policy’s parameters. The critic uses a three layer neural network to approximate the action-value function, and to evaluate the optimized policy. Simulation results evaluate the proposed architecture for actor-critic learning, and shows its ability to improve its performance with experience.


This is a manuscript of a proceeding published as Masadeh, Ala'eddin, Zhengdao Wang, and Ahmed E. Kamal. "An Actor-Critic Reinforcement Learning Approach for Energy Harvesting Communications Systems." In 2019 28th International Conference on Computer Communication and Networks (ICCCN). DOI: 10.1109/ICCCN.2019.8846912. Posted with permission.


© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Copyright Owner




File Format


Published Version


Article Location