Total: 1
In a sequential prediction setting, a decision-maker may be primarily concerned with whether the continuous-valued future observation will increase or decrease compared to the current one, rather than the actual value of the future observation. We introduce the parity calibration framework, where the goal is to provide calibrated uncertainty estimates for the increase-decrease event in a timeseries. While these ``parity" probabilities can be extracted from a distributional forecast, such a strategy does not work as expected and can have poor practical performance. We then observe that although the original task was regression, parity calibration can be expressed as binary calibration. Drawing on this connection, we use a recently proposed online binary calibration method to achieve parity calibration. We demonstrate the effectiveness of our method on real-world case studies in epidemiology, weather forecasting, and model-based control in nuclear fusion.