Improving greenhouse fruit-production control by integrating reinforcement learning into short-horizon model predictive control

Bart van Laatum, Salim Msaad, Eldert J. van Henten, Robert D. McAllister, Sjoerd Boersma

Greenhouse fruit-production control aims to maximize the economic performance (fruit revenue minus operating costs) while operating within system constraints under external weather disturbances. Control methods need to balance the delayed economic benefit of fruit yield with current operating costs. For such problems, model predictive control (MPC) can explicitly handle system constraints under future weather disturbances, but can become computationally demanding when using sufficiently long prediction horizons for (relatively large) nonlinear greenhouse fruit production models. In contrast, reinforcement learning (RL) can learn control policies offline while considering longer-term economic performance, but struggles to enforce system constraints, and performance may degrade under unseen weather trajectories. This work proposes trajectory-selection RL-MPC, a framework that incorporates longer-term economic information of fruit yield into a short-horizon MPC optimization problem. The framework uses an RL rollout trajectory to define a terminal region constraint and terminal cost. Next, a nonlinear MPC solves a short-horizon optimization problem with these terminal ingredients to find a local optimum. Finally, the framework selects and executes the first input from the trajectory with the better objective value, either from the MPC-predicted or the RL rollout trajectory. The method is applied to GreenLight, a large-scale greenhouse tomato production model that exhibits stiff dynamics. The simulation results show that trajectory-selection RL-MPC with a one-hour prediction horizon matches the closed-loop performance of a high-performing guiding policy while significantly improving over standalone MPC with the same horizon.