feat(skore): Add a method to check performance consistency across cv splits

[GaetandeCast]

Closes #1989.

Investigating how to detect cross-validation splits with unexpectedly good or bad performance using the Modified Z-score with Median Absolute Deviation (MAD).

Example usage:

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from skore import CrossValidationReport
X, y = load_breast_cancer(return_X_y=True)
classifier = LogisticRegression(max_iter=10_000)
report = CrossValidationReport(classifier, X=X, y=y, splitter=5)
report.check_split_consistency()

[github-actions]

Documentation preview @ dd45fdd

[github-actions]

Coverage Report for skore/

File	Stmts	Miss	Cover	Missing
skore/src/skore
__init__.py	24	0	100%
_config.py	31	0	100%
exceptions.py	4	4	0%	4, 15, 19, 23
skore/src/skore/_sklearn
__init__.py	6	0	100%
_base.py	199	14	92%	46, 59, 128, 131, 184, 187–188, 190–193, 226, 229–230
feature_names.py	23	0	100%
find_ml_task.py	61	0	100%
types.py	28	1	96%	30
skore/src/skore/_sklearn/_comparison
__init__.py	7	0	100%
feature_importance_accessor.py	32	1	96%	131
metrics_accessor.py	191	2	98%	177, 257
report.py	108	0	100%
utils.py	57	0	100%
skore/src/skore/_sklearn/_cross_validation
__init__.py	9	0	100%
data_accessor.py	45	2	95%	137, 140
feature_importance_accessor.py	21	0	100%
metrics_accessor.py	193	1	99%	250
report.py	171	6	96%	538, 545–547, 549, 618
skore/src/skore/_sklearn/_estimator
__init__.py	9	0	100%
data_accessor.py	66	1	98%	82
feature_importance_accessor.py	130	0	100%
metrics_accessor.py	387	4	98%	425, 429, 444, 479
report.py	166	2	98%	449–450
skore/src/skore/_sklearn/_plot
__init__.py	3	0	100%
base.py	107	6	94%	63–64, 266–268, 272
utils.py	83	0	100%
skore/src/skore/_sklearn/_plot/data
__init__.py	2	0	100%
table_report.py	175	1	99%	657
skore/src/skore/_sklearn/_plot/feature_importance
__init__.py	0	0	100%
coefficients.py	178	0	100%
skore/src/skore/_sklearn/_plot/metrics
__init__.py	6	0	100%
confusion_matrix.py	156	1	99%	543
metrics_summary_display.py	8	0	100%
precision_recall_curve.py	175	0	100%
prediction_error.py	152	0	100%
roc_curve.py	314	9	97%	263, 455, 578, 583, 684, 689, 693, 762, 902
skore/src/skore/_sklearn/train_test_split
__init__.py	0	0	100%
train_test_split.py	58	0	100%
skore/src/skore/_sklearn/train_test_split/warning
__init__.py	8	0	100%
high_class_imbalance_too_few_examples_warning.py	19	1	94%	83
high_class_imbalance_warning.py	20	0	100%
random_state_unset_warning.py	10	0	100%
shuffle_true_warning.py	9	0	100%
stratify_is_set_warning.py	10	0	100%
time_based_column_warning.py	21	0	100%
train_test_split_warning.py	3	0	100%
skore/src/skore/_utils
__init__.py	6	2	66%	8, 13
_accessor.py	90	3	96%	34, 146, 190
_cache.py	37	0	100%
_environment.py	27	0	100%
_fixes.py	8	0	100%
_index.py	5	0	100%
_logger.py	22	4	81%	15–17, 19
_measure_time.py	10	0	100%
_parallel.py	38	3	92%	23, 33, 124
_patch.py	21	12	42%	30, 35–39, 42–43, 46–47, 58, 60
_progress_bar.py	46	0	100%
_repr_html.py	8	0	100%
_show_versions.py	38	0	100%
_testing.py	56	0	100%
skore/src/skore/project
__init__.py	2	0	100%
project.py	57	0	100%
summary.py	75	1	98%	119
widget.py	187	0	100%
TOTAL	4218	81	98%

Tests	Skipped	Failures	Errors	Time
1366	5 💤	0 ❌	0 🔥	8m 35s ⏱️