Blind Spot Analysis
The Blind Spot Analysis tab helps identify subcohorts within a validation where model performance deviates significantly from the overall results. These blind spots can arise from demographic bias, modality variations, or other metadata-related factors. The tool enables users to detect, explore, and validate such regions using flexible cohort generation and performance comparison.
1. Navigation and Validation Selection
- Navigate to Validation Analysis → Blind Spot Analysis
- Choose a validation run from the list to begin analysis
- The interface lists all available validations, allowing quick switching between runs
2. Generating Blind Spot Analysis
Once a validation is selected:
- Click on Generate Blind Spot Analysis
- Configure parameters:
- Min Cohort Size: Minimum number of samples required in a cohort
- Lower Bound: Threshold below which performance drop is flagged
- Max Combinations: Limits how many cohort combinations to evaluate
- Optional: Specify relevant metadata fields (e.g., Age, Gender) to control the scope
- Click Submit to start the analysis
- While the analysis runs, a loading status is shown
3. Viewing Analysis Results
Once complete, the interface displays results under a new Round tab:
- Each round corresponds to a unique configuration of the blind spot analysis
- The latest round is always shown at the top
- You can filter by class, or toggle cohort dimensions like Age, Gender, or Ethnicity
4. Filtering and Interacting with Results
- Use the Filter panel to customize which metadata combinations appear
- When selecting a filter (e.g., only Age), the list updates to show only subcohorts including that field
- The subcohort table updates dynamically:
- Rows re-order based on filter priority
- Non-selected metadata columns are hidden
5. Interpreting Performance Charts
The tool visualizes performance changes across identified subcohorts:
- Each subcohort is represented as a bar in a chart
- X-axis shows sample size
- Bar height reflects performance drop compared to base validation
- Subcohort size is proportional to sample count
- Cohorts below the Min Cohort Size threshold are listed separately
6. Subcohort Validation
Users can generate targeted validation for selected subcohorts:
- Select one or more subcohorts
- Click Generate Subcohorts
- A new validation is triggered for the selected cohorts
- Once complete, shortcut icons appear next to the subcohort entry
- Clicking these icons opens the Validation Metrics page for that cohort, enabling deeper analysis
✅ Tip: Use Blind Spot Analysis to proactively detect performance gaps across demographic or technical subgroups, and create validations to confirm model reliability under varying conditions.