Skip to main content

Overview

This runbook covers operational procedures for monitoring SLA health, responding to breaches, and troubleshooting common issues.

Architecture

Current (Phase 1)

  • SLA definitions and instances stored in pf_sla_definitions and pf_sla_instances
  • Instance status computed client-side via hooks (useSLADashboard, useSLAInstances)
  • Events published to fw_domain_events for downstream notifications
  • Deadline calculation uses interval-based fallback (now() + interval)

Planned (Phase 2)

  • pf-sla-checker Edge Function — Periodic cron job to scan active instances, update statuses, and fire breach/warning events automatically
  • Database Webhook on fw_domain_events — Auto-instantiate SLA instances when trigger events arrive
  • PF-84 Business Calendars — Business-hours-aware deadline calculation (excludes weekends, holidays)
  • Breach detection latency target: < 2 minutes from event to status update

Health Checks

Active Instance Count

Compliance Rate (Last 30 Days)

Stale Active Instances

Instances that have been active longer than 2× their target duration may indicate missed completion events:

Manual Instance Management

Pause an Instance (UI)

  1. Navigate to SettingsSLAInstances
  2. Find the instance → menu → Pause
  3. Enter reason → Confirm

Resume a Paused Instance (UI)

  1. Find the paused instance → menu → Resume

Extend a Deadline (SQL — Admin Only)

Force-Complete a Stale Instance (SQL — Admin Only)

Troubleshooting

Instance Not Created After Trigger Event

  1. Verify the trigger event exists in fw_domain_events
  2. Check the SLA definition’s trigger_event_type matches the event’s event_type
  3. Confirm the definition’s is_active flag is true
  4. Phase 2: Check pf-sla-checker edge function logs for errors

Instance Stuck in Active (Should Be Breached)

  1. Check if deadline_at has passed — the checker may not have run yet (Phase 2)
  2. In Phase 1, breach detection is client-side; the dashboard will show it correctly on next load
  3. To manually mark as breached:

Missed Completion Events

  1. Query fw_domain_events for the expected completion event type
  2. If the event exists but the instance wasn’t completed, check event matching logic
  3. Phase 2: The checker edge function will handle this automatically

High Active Instance Count

  1. Run the active instance count query above
  2. If near the max limit, archive or complete stale instances
  3. Review whether definitions are too broad (triggering on high-frequency events)

Monitoring Alerts (Planned)

References