scxpand.util.metrics

scxpand.util.metrics#

Functions

calculate_metrics(y_true, y_pred_prob, obs_df)

Calculate comprehensive evaluation metrics for T-cell expansion prediction.

compute_basic_metrics(y_true, y_pred_prob[, ...])

evaluate_and_save(y_true, y_pred_prob, obs_df)

Evaluate model performance, save results, optionally create plots, and report Optuna trial results if provided.

get_score_from_nested_dict(...)

Extract a metric value from a nested dictionary.

safe_hmean(values)

scxpand.util.metrics.calculate_metrics(y_true, y_pred_prob, obs_df, threshold=0.5)#

Calculate comprehensive evaluation metrics for T-cell expansion prediction.

Computes overall classification metrics plus stratified metrics by cell type and tissue type. Includes robust error handling for edge cases.

Parameters:

  • y_true (ndarray | Tensor) – True binary labels (0=not expanded, 1=expanded).

  • y_pred_prob (ndarray | Tensor) – Predicted probabilities for expansion [0-1].

  • obs_df (DataFrame) – DataFrame with ‘imputed_labels’ and ‘tissue_type’ columns.

  • threshold (float (default: 0.5)) – Classification threshold for converting probabilities to predictions.

Return type:

dict[str, dict[str, float]]

Returns:

Nested dictionary with ‘overall’ metrics and stratified metrics by tissue_type and imputed_labels combinations. Each contains: - AUROC: Area under ROC curve - F1: F1 score - error_rate: Classification error rate - false_positive_rate, false_negative_rate: FPR and FNR - RMSE: Root mean squared error - positives_rate: Fraction of positive samples

scxpand.util.metrics.compute_basic_metrics(y_true, y_pred_prob, threshold=0.5)#
Return type:

dict[str, float]

scxpand.util.metrics.evaluate_and_save(y_true, y_pred_prob, obs_df, eval_name='', save_path=None, plots_dir=None, threshold=0.5, trial=None, score_metric='harmonic_avg/AUROC', use_table_format=True)#

Evaluate model performance, save results, optionally create plots, and report Optuna trial results if provided.

Parameters:

  • y_true (ndarray | Tensor) – Ground truth labels [n_eval] (0 or 1)

  • y_pred_prob (ndarray | Tensor) – Predicted probabilities [n_eval] in range [0, 1]

  • obs_df (DataFrame) – DataFrame containing observation data

  • eval_name (str (default: '')) – Name for the evaluation (used in file names)

  • save_path (Path | None (default: None)) – Path to save results

  • plots_dir (Path | None (default: None)) – Optional path to save plots

  • threshold (float (default: 0.5)) – Classification threshold

  • trial (Trial | None (default: None)) – Optional Optuna trial for reporting

  • score_metric (str (default: 'harmonic_avg/AUROC')) – Metric key to report to Optuna

  • use_table_format (bool (default: True)) – If True, display metrics in table format

Returns:

Dictionary containing the evaluation metrics

Return type:

dict

scxpand.util.metrics.get_score_from_nested_dict(nested_metrics_dict, metric_name)#

Extract a metric value from a nested dictionary.

Parameters:

  • nested_metrics_dict (dict) – A nested dictionary of metrics (e.g. {“dev”: {“auc”: 0.9}}).

  • metric_name (str) – A string of the metric name to get (e.g. “dev/auc”).

Return type:

float

Returns:

The score value (e.g. 0.9).

scxpand.util.metrics.safe_hmean(values)#
Return type:

float

Contents