arctan_loss_multi_objective#
- openstef_models.utils.loss_functions.arctan_loss_multi_objective(y_true: ndarray[tuple[Any, ...], dtype[floating]], y_pred: ndarray[tuple[Any, ...], dtype[floating]], quantiles: list[Quantile], sample_weight: ndarray[tuple[Any, ...], dtype[floating]] | None = None, s: float = 0.1) tuple[ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]]][source]#
Arctan-smoothed multi-quantile pinball loss objective function for XGBoost.
Computes first and second derivatives of the arctan pinball loss, a smooth approximation of the standard pinball loss that provides non-zero second derivatives essential for XGBoost’s second-order optimization. This formulation addresses the fundamental limitation of standard pinball loss having zero second derivatives.
The arctan pinball loss maintains asymptotic unbiasedness while providing substantial second derivatives in the relevant domain (typically -10 to 10 for standardized targets), making it well-suited for gradient boosting algorithms that rely on second-order information for tree splitting decisions.
- Parameters:
y_true (ndarray[tuple[Any, ...], dtype[floating]]) – True target values, shape (n_samples, n_quantiles)
y_pred (ndarray[tuple[Any, ...], dtype[floating]]) – Predicted values, shape (n_samples, n_quantiles)
quantiles (list[Quantile]) – List of validated quantiles corresponding to predictions
sample_weight (ndarray[tuple[Any, ...], dtype[floating]] | None) – Optional sample weights, shape (n_samples,). If provided, gradients and hessians are scaled by these weights to give different importance to different samples in the loss computation.
s (float) – Smoothing parameter for the arctan function. Smaller values provide closer approximation to standard pinball loss but reduce second derivative magnitude.
- Returns:
(gradient, hessian) arrays in 2D format for XGBoost multi-output
gradient: First derivative of arctan pinball loss, shape (n_samples, n_quantiles)
hessian: Second derivative of arctan pinball loss, shape (n_samples, n_quantiles)
Both arrays are normalized by n_quantiles for consistent optimization dynamics.
- Return type:
- Mathematical Formulation:
Loss function: L^(arctan)_τ,s(u) = (τ - 0.5 + arctan(u/s)/π) * u + s/π
Where:
u = y_true - y_pred (prediction error)
τ = quantile level (0 < τ < 1)
s = smoothing parameter
First derivative (gradient): ∂L/∂u = τ - 0.5 + arctan(u/s)/π + u/(π*s*(1+(u/s)²))
Second derivative (hessian): ∂²L/∂u² = 2/(π*s) * (1+(u/s)²)^(-2)
Key Properties:
Smooth approximation: Continuously differentiable everywhere
Asymptotic unbiasedness: Behaves like standard pinball loss for large
|u|Non-zero hessian: Provides substantial second derivatives for XGBoost optimization
Scale sensitivity: Hessian magnitude inversely proportional to smoothing parameter s
- Gradient Normalization:
Both gradient and hessian are divided by n_quantiles to ensure consistent optimization dynamics regardless of the number of quantiles being predicted. This normalization prevents gradient magnitude from scaling linearly with the number of outputs.
References
Arctan pinball loss: arXiv:2406.02293v1 “Composite Quantile Regression With XGBoost”
Second-order optimization: XGBoost requires non-zero hessian for tree splitting
Implementation: LaurensSluyterman/XGBoost_quantile_regression