Hybrid additive modeling with partial dependence for supervised regression and dynamical systems forecasting

Learning processes by exploiting restricted domain knowledge is an important
task across a plethora of scientific areas, with more and more hybrid training
methods additively combining data-driven and model-based approaches. Although
the obtained models are more accurate than purely data-driven models, the
optimization process usually comes with sensitive regularization constraints.
Furthermore, while such hybrid methods have been tested in various scientific
applications, they have been mostly tested on dynamical systems, with only
limited study about the influence of each model component on global performance
and parameter identification. In this work, we introduce a new hybrid training
approach based on partial dependence, which removes the need for intricate
regularization. Moreover, we assess the performance of hybrid modeling against
traditional machine learning methods on standard regression problems. We
compare, on both synthetic and real regression problems, several approaches for
training such hybrid models. We focus on hybrid methods that additively combine
a parametric term with a machine learning term and investigate model-agnostic
training procedures. Therefore, experiments are carried out with different
types of machine learning models, including tree-based models and artificial
neural networks. We also extend our partial dependence optimization process for
dynamical systems forecasting and compare it to existing schemes.