Skip to main content

Validation Summary

The Validation Summary tab provides a high-level snapshot of the dataset's composition and the model’s behavior across various patient and imaging characteristics. It enables users to identify biases, discover performance disparities, and gain insights for improving model robustness and fairness.

1. Navigation and Export Options​

  • Access via Validation Analysis → Validation Summary
  • The header includes:
    • A Report Export button for generating a PDF summary
    • A toggle between Portrait and Landscape layout options

2. Dataset Distribution – Patient Characteristics​

  • Visual breakdown of patient demographics and conditions (e.g., Abdominal Pain, Acute Hepatic Injury, Age Range, BMI >35, COVID-19 Status, Gender)
  • Displays how each subgroup correlates with outcomes (e.g., Alive vs. Dead)

3. Dataset Distribution – Imaging Characteristics​

  • Summary of image-related metadata (e.g., modality, acquisition parameters)
  • If metadata is unavailable, this section may display a placeholder message

4. Model Performance – Class Comparison​

  • Comparison of key metrics (e.g., Accuracy, F1 Score, Precision, Sensitivity, Specificity, Matthews Correlation Coefficient)
  • Performance is broken down by prediction class (e.g., Dead vs. Alive)

5. Model Performance – Overall​

  • Micro and macro aggregated performance metrics:
    • Micro F1, Macro AUC, Macro Precision, etc.

6. Model Performance – Patient Demographics​

  • Visual performance breakdown across demographic variables, including:
    • Abdominal Pain
    • Acute Hepatic Injury
    • Age Range
    • BMI >35
    • Gender (Female/Male)
  • Shows corresponding Accuracy and Macro AUC

7. Model Performance – Imaging Metadata​

  • If imaging metadata is available, displays performance comparisons by imaging-related characteristics
  • May be empty depending on dataset

8. Cohort Size Analysis​

  • Largest Cohorts: Most frequently occurring patient subgroups (e.g., by COVID-19 status or ICU admission)
  • Smallest Cohorts: Highlights underrepresented groups that may need further attention or augmentation

9. One-Variable Metadata Comparison​

  • Automatically identifies:
    • Best Performers: Subgroups with the highest accuracy or AUC
    • Worst Performers: Subgroups with significant performance drops

10. Two-Variable Metadata Comparison​

  • Analyzes model behavior based on combinations of metadata (e.g., Age + Gender)
  • Shows best and worst performer combinations across selected variables

11. Minority Sub-Cohort Groups​

  • Lists small subgroups by patient count and relative validation percentage
  • Useful for detecting and investigating groups with potential model blind spots

✅ Tip: Use the Validation Summary to detect early signs of model bias, uncover performance bottlenecks, and generate hypotheses for more targeted validations.