Reinforcement Learning for Process Control with Application in Semiconductor Manufacturing

Abstract

Process control is widely discussed in the manufacturing process, especially for semiconductor manufacturing. Due to unavoidable disturbances in manufacturing, different process controllers are proposed to realize variation reduction. Since reinforcement learning (RL) has shown great advantages in learning actions from interactions with a dynamic system, we introduce RL methods for process control and propose a new controller called RL-based controller. Considering the fact that most existing process controllers mainly rely on a linear model assumption for the process input-output relationship, we first propose theoretical properties of RL-based controllers according to the linear model assumption. Then the performance of RL-based controllers and traditional process controllers (e.g., exponentially weighted moving average (EWMA), general harmonic rule (GHR) controllers) are compared for linear processes. Furthermore, we find that the RL-based controllers have potential advantages to deal with other complicated nonlinear processes that are with and without assumed explicit model formulations. The intensive numerical studies validate the advantages of the proposed RL-based controllers.