Extrapolation by way of the Lens of Distributional Regression

View a PDF of the paper titled Engression: Extrapolation by way of the Lens of Distributional Regression, by Xinwei Shen and Nicolai Meinshausen

View PDF
HTML (experimental)

Summary:Distributional regression goals to estimate the total conditional distribution of a goal variable, given covariates. Common strategies embrace linear and tree-ensemble primarily based quantile regression. We suggest a neural network-based distributional regression methodology known as `engression’. An engression mannequin is generative within the sense that we will pattern from the fitted conditional distribution and can also be appropriate for high-dimensional outcomes. Moreover, we discover that modelling the conditional distribution on coaching information can constrain the fitted operate outdoors of the coaching assist, which affords a brand new perspective to the difficult extrapolation downside in nonlinear regression. Particularly, for `pre-additive noise’ fashions, the place noise is added to the covariates earlier than making use of a nonlinear transformation, we present that engression can efficiently carry out extrapolation beneath some assumptions corresponding to monotonicity, whereas conventional regression approaches corresponding to least-squares or quantile regression fall quick beneath the identical assumptions. Our empirical outcomes, from each simulated and actual information, validate the effectiveness of the engression methodology and point out that the pre-additive noise mannequin is often appropriate for a lot of real-world eventualities. The software program implementations of engression can be found in each R and Python.