Learning to Control pH Processes at Multiple Time Scales

Abstract

This article presents a solution to pH control based on model-free learning control (MFLC). The MFLC technique is proposed because the algorithm gives a general solution for acid-base systems, yet is simple enough for implementation in existing control hardware. MFLC is based on reinforcement learning (RL), which is learning by direct interaction with the environment. The MFLC algorithm is model free and satisfying incremental control, input and output constraints. A novel solution of MFLC using multi-step actions (MSA) is presented: actions on multiple time scales consist of several identical primitive actions. This solves the problem of determining a suitable fixed time scale to select control actions so as to trade off accuracy in control against learning complexity. An application of MFLC to a pH process at laboratory scale is presented, showing that the proposed MFLC learns to control adequately the neutralization process, and maintain the process in the goal band. Also, the MFLC controller smoothly manipulates the control signal.