Regression model performance evaluation function: Features and application destinations, respectively
Evaluation function of regression model that can be used with sckit-learnBased on, I will summarize the differences and usage of each. yi is the true y, and the bar above it is the predicted value.
Regression model performance evaluation function
Mean square error (MSE, RMSE)
$$ \ text {MSE} (y, \ hat {y}) = \ frac {1} {n_ \ text {samples}} \ sum_ {i = 0} ^ {n_ \ text {samples} – 1} (y_i – \ Hat {y} _i (predicted value)) ^ 2. $$Mean squared error (MSE: Mean squared error, RMSE: Root mean squared error) adjusts the model to minimize the average sum of squared errors. RMSE takes the square root after square and aligns the dimension with y.
First-choice evaluation function.Focus on large values(The larger the value, the larger the square error)Sensitive to outliers, but useful in many situations where outliers decrease exponentially, such as in the normal distribution.
Mean Absolute Error (MAE)
$$ \ text {MAE} (y, \ hat {y}) = \ frac {1} {n_ {\ text {samples}}} \ sum_ {i = 0} ^ {n_ {\ text {samples}}- 1} \ left | y_i – \ hat {y} _i \ right |. $$Mean absolute error (MAE) adjusts the model to minimize the average sum of absolute values of error.
From MSERobust evaluation method that is resistant to outliers.On the contrary, learning of data with greatly different values is weak.However, there remains a tendency to emphasize large values.For datasets that have a large impact on outliers.
Mean squared logarithmic error
$$ \ text {MSLE} (y, \ hat {y}) = \ frac {1} {n_ \ text {samples}} \ sum_ {i = 0} ^ {n_ \ text {samples} – 1} (\ log_e (1 + y_i) – \ log_e (1 + \ hat {y} _i)) ^ 2. $$The mean square logarithmic error is the average of the sum of squares for the logarithmic y error.It is applied to tasks where y increases exponentially, such as population and annual product sales.There is a greater penalty for the underestimation than for the overestimation.
Median absolute error (MedAE)
$$ R ^ 2 (y, \ hat {y}) = 1 – \ frac {\ sum_ {i = 1} ^ {n} (y_i – \ hat {y} _i) ^ 2} {\ sum_ {i = 1} ^ {n} (y_i – \ bar {y}) ^ 2} $$Minimize the median of the absolute error of each predicted value.The principle is that the more outliers are, the larger the error is, but the median error is hardly affected by it.
Since it is more robust than MAE, it is useful for data with only a small number of samples and data for which outliers are anxious but cannot be excluded.
Mean percentage error (MPE)
The average percentage error works to minimize the average percentage (%: percent) that deviates from the measured value. Not implemented in sckit-learn.It looks good because values in any region are treated as errors of the same weight regardless of their size, but there are some problems in practical use (an error occurs when y is 0, or a very small value). Then% error diverges, etc .:From English wikipedia）
R2 score (coefficient of determination)
Correlation between true and predicted valuesIndicates, which corresponds to the fit of yy plot.If there is an exact match, the R2 score will be 2. Because the RXNUMX score depends on the datasetR² cannot be compared between different datasets.
Summary
BasicallyR2 score and mean squared error (MSE)Or R2 score and mean absolute error (MAE)Is the best to use together.
If you want to emphasize large values and learn outliers well,RMSE.
If you want to reduce the weight of outliers,MAE.
If the prediction target is a value that changes exponentiallyMean square logarithmic errorThere is, but this can also be dealt with by logarithmizing y.
If you have a small amount of data, or if you don't want to exclude outliers that have a large impact,Median absolute error (MedAE)To use.