Publication
Title
Smart initialisation and approximating loss function for robust regression
Author
Abstract
Two of the most common methods for robust regression are least trimmed squares (LTS) and least median of squares (LMS) regression. Both of these methods require sorting the squared residuals. Because sorting is not a differentiable operation, end-to-end optimalisation with gradient-based methods is not stable. Furthermore, existing algorithms for estimating LTS and LMS regressors rely on multiple random initial starting points. We propose and investigate two potential improvements to LTS and LMS: (1) the use of soft differentiable sorting in the loss functions and (2) deterministic initialisation of the estimators using the wrapping transformation. The first improvement aims to tackle the drawbacks of using hard sorting by introducing an alternative loss function that can be optimised using gradient based optimisation schemes, while the latter improvement aims to remove the need for multiple random initial starting points, leading to both improved accuracy as well as faster convergence. We perform an extensive experiment on both simulated and real world datasets and compare the performance of our introduced methods with well known baseline methods for robust regression. We show that the deterministic initialisation has significant benefits for LTS and LMS, both for predictive accuracy and for computational speed. The soft loss function mostly benefits LMS, as it makes it possible to apply iterative optimisation schemes to the LMS loss function. We also demonstrate the potential application of the Soft LTS loss function to non-linear regression problems using neural networks.
Language
English
Source (journal)
Information sciences. - New York
Publication
New York : 2023
ISSN
0020-0255
DOI
10.1016/J.INS.2023.119715
Volume/pages
651 (2023) , p. 1-13
Article Reference
119715
ISI
001087136000001
Full text (Publisher's DOI)
Full text (open access)
The author-created version that incorporates referee comments and is the accepted for publication version Available from 03.04.2024
Full text (publisher's version - intranet only)
UAntwerpen
Faculty/Department
Research group
Project info
data-driven anomaly detection and cashflow prediction for accountants
Publication type
Subject
Affiliation
Publications with a UAntwerp address
External links
Web of Science
Record
Identifier
Creation 04.12.2023
Last edited 27.02.2024
To cite this reference