Tomonori Sadamoto Assistant Professor, Department of Mechanical Engineering and Intelligent Systems, UEC Tokyo.

Innovative control theory: Bridging the gap between research on control, reinforcement learning, and power systems

Assistant Professor Tomonori Sadamoto is an expertise in control theory, currently focusing on integrating control theory with reinforcement learning and power engineering.

Reinforcement learning is a key methodology for controlling large-scale complex systems such as power grids and transportation networks. However, the major contemporary learning theories currently used are unsuitable for real-time control because designers must repeat trials just for acquiring data. Instead, it is necessary to develop a methodology that is capable of real-time decision making.

With this background, Sadamoto and his colleagues have proposed a fast real-time learning method for large-scale network systems. For example, it takes 140 seconds to learn when conventional methods are applied to 100-dimensional level power systems. In contrast. Their method completes the task of learning in only 6 seconds— a reduction of the learning time by 95% while maintaining control performance.

The central feature of their method is data compression, where redundant data is eliminated while minimizing information loss by projecting data onto dominantly controllable subspace. Because of the smaller scale and less information loss of the compressed data, the learning process can provide nearly optimal control with lower computational complexity. Moreover, the applications of this approach can be extended to partly observable systems where the availability of sensors is very limited.

These are new reinforcement learning methods, and in the language of control theory, can be interpreted as model-free versions of optimal controller design through model order reduction.

Sadamoto is also working on the development of control-theoretic power engineering. The rapid spread of renewable energy resources has led to rapid and major changes in power grids and it is widely accepted that traditional power engineering approaches are no longer applicable. So, system operators require innovative and systematic methodology based on control theory.

For example, Sadamoto’s group has proposed a new wind turbine architecture to enhance the stability of wind-integrated power systems. It is known that DFIG-type wind turbines can cause frequency oscillations in the grid. Actions to control turbines, however, sometimes fail to dampen oscillations, and it is not clear when actions will fail.

In their research, Sadamoto and his colleagues theoretically showed that control failure is induced by parameter-dependent uncontrollability of turbines. To solve this problem, they proposed adding a compensator to improve the controllability, thereby successfully enhancing the stability of wind-integrated power systems.

In related work, Sadamoto has proposed a new method for designing power flow schedules to enhance the stability of the entire grid. The stability of power systems is known to depend on the power flowing through them. However, current power flow design is based on economic optimality only. The proposed‘ method minimizes the H2 norm, which is a measure of stability used in control theory, while ensuring economic optimality. This research is being conducted in collaboration with Keio University in Tokyo.

These are examples of the innovative control-theoretic solutions being developed by Sadamoto and his colleagues for the realization of next-generation power systems. In 2020 an article published by Sadamoto and his colleagues received the IEEE Control Systems Magazine Outstanding Paper Award for bridging the gap between research communities focused on control and power systems.

Further information

Research Highlight: Innovative automated control systems: Control-theoretic approach for fast online reinforcement learning
Department website: http://www.sc.lab.uec.ac.jp/ts/index.html