Spekra
Docs

Getting Started

  • Overview
  • Playwright
  • Jest
  • Vitest

Core Concepts

  • Flaky Tests
  • Stability Metrics
  • Test Identity

Reporters

  • Playwright
  • Playwright Config
  • Jest
  • Jest Config
  • Vitest
  • Vitest Config

Platform

  • Dashboard
  • Flaky Tests View
  • Test Runs
  • API Keys
  • Rate Limits

CI/CD

  • Overview
  • GitHub Actions
  • GitLab CI

Security

  • Overview
  • Data Handling
  • Compliance

Troubleshooting

  • Overview
  • Connection Issues
  • Missing Data
DocsPlatformFlaky Tests View

Flaky Tests View

The Flaky Tests view is your command center for identifying, prioritizing, and tracking flaky tests in your test suite.

Accessing the View

Navigate to Flaky Tests in the sidebar, or click the flaky tests count on the dashboard.

Understanding the List

Each flaky test is displayed with:

ColumnDescription
Test NameFull test title with suite path
FileTest file location
ReliabilityPercentage of consistent runs
Last FlakyWhen the test was last flaky
Flake CountNumber of flaky occurrences
SeverityImpact score (Critical/High/Medium/Low)

Sorting and Filtering

Sort Options

  • Severity (default) - Most impactful tests first
  • Reliability - Least reliable tests first
  • Last Flaky - Most recently flaky first
  • Flake Count - Most frequent flakers first

Filters

  • Time Range - Show tests flaky in the last 7/30/90 days
  • Severity - Filter by impact level
  • File - Search by file path
  • Tag - Filter by test tags

Test Details

Click any test to open the detail panel:

Overview Tab

  • Reliability and stability trends over time
  • Recent results timeline
  • Error messages from failures

History Tab

  • All occurrences of flakiness
  • Comparison with previous runs
  • Links to CI jobs where flakiness occurred

Identity Tab

  • Test's unique identifier
  • Previous names/locations (if moved)
  • Related tests

Actions

Mark as Resolved

When you've fixed a flaky test, mark it as resolved. It will be removed from the flaky list but continue to be monitored. If it becomes flaky again, it will reappear.

Quarantine

For tests you can't fix immediately, quarantine them. Quarantined tests:

  • Are hidden from the main flaky list
  • Still appear in a separate "Quarantined" tab
  • Continue to be tracked for metrics

Quarantining should be temporary. Set a reminder to revisit quarantined tests.

Add Note

Add notes to track investigation progress or document known issues:

2024-01-15: Investigating - seems related to database timing
2024-01-16: Added explicit wait, monitoring

Prioritizing Fixes

Use this priority matrix:

SeverityAction
CriticalFix immediately - these are blocking
HighFix this week
MediumSchedule in upcoming sprint
LowMonitor, fix if time permits

Workflow Recommendations

Daily

  • Check for new critical/high severity tests
  • Review any tests that became flaky overnight

Weekly

  • Review medium severity tests
  • Check quarantined tests for any that can be fixed
  • Review overall reliability trends

Sprint Planning

  • Include flaky test fixes in sprint work
  • Set reliability goals for the sprint
  • Celebrate improvements

Next Steps

  • Understanding flaky tests - What causes flakiness
  • Stability metrics - How metrics are calculated
  • Test runs - Analyzing individual runs

Previous

Dashboard

Next

Test Runs