Towards a single goodness metric of clinically relevant, accurate, fair and unbiased machine learning predictions of health-related quality of life

With the accelerating adoption of artificial intelligence and machine learning in medicine, algorithmic fairness, and the demonstration of the validity, accuracy and clinical relevance of models is becoming increasingly important. EQ-5D-5L is one of the most widely used instrument for measuring health-related quality of life. For the evaluation, selection, and adoption of machine learning models used for the prediction of healt-related quality of life expressed in EQ-5D-5L index scores, we propose a new metric called G as a single measure of model goodness that consolidates measures such as accuracy, bias, and fairness in a value ranging from 0 to 1. Fairness is conceived as the independence of prediction error from EQ-5D-5L, and protected variables, such as sex, age, education, income and work status. The G metric ignores prediction errors that cannot be perceived by patients (e.g., smaller than the minimum clinically important difference (MCID) for EQ-5D-5L). For the computation of G, we estimated the Hungarian MCID for EQ-5D-5L as 0.066617. The proposed metric was tested on simulated prediction errors in a real-world sample of 2000 individuals and synthetic dataset. Consistently with the expectations, in the real-world dataset, highest G values were found with most accurate predictions and independent errors from the protected variables. However, using the same error simulation parameters, the performance of G was inconsistent in the synthetic dataset. Further research is needed to derive a G metric robust to data distribution shifts.