> ## Documentation Index > Fetch the complete documentation index at: https://docs.encoreos.io/llms.txt > Use this file to discover all available pages before exploring further. # External Webhook Triggers — Integration > Feature ID: FW-59 Status: ✅ Implemented Spec Reference: FW-59-external-webhook-triggers.md Last Updated: 2026-03-22 **Feature ID:** FW-59\ **Status:** ✅ Implemented\ **Spec Reference:** [FW-59-external-webhook-triggers.md](../../../specs/fw/specs/FW-59-external-webhook-triggers.md)\ **Last Updated:** 2026-03-22 *** ## Overview FW-59 adds a single inbound webhook surface so external systems can trigger FW workflow executions without bespoke edge functions per integration. The Edge Function performs endpoint resolution, auth, schema validation, and durable enqueue (FW-46). Tenant resolution uses `pf_organizations.slug` plus per-endpoint `endpoint_slug`. **Not the same as PF inbound webhooks:** [process-inbound-webhook](../../../supabase/functions/process-inbound-webhook/index.ts) handles **`pf_integrations`** (platform integration registry, stores raw payload on `pf_webhook_deliveries`). FW-59 targets **workflow execution** via `fw_workflow_definitions` and FW-46 — different tables, auth model, and retention/encryption rules. *** ## Integration Points (from Spec) | Dependency | Type | Purpose | | ---------------------------- | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | PF-01 (Organizations) | Data | Resolve `org_slug` → `organization_id` via `pf_organizations.slug`; all rows scoped by `organization_id` | | PF-04 (Audit Trail) | Platform | Configuration changes on `fw_webhook_endpoints` / secrets audited | | PF-10 (Notifications) | Platform | Auto-suspension, secret rotation, expiry warnings | | PF-42 (Rate Limiting) | Platform / Edge | Per-endpoint limits via `pf_check_rate_limit` + **`pf_rate_limits`** row (`scope = 'endpoint'`, key `fw_webhook:{endpoint_id}`) — **required** or PF-42 returns allow | | FW-16 (Event Triggers) | Intra-core | Conceptual alignment: external trigger produces workflow execution like domain events | | FW-46 (Durable Execution) | Intra-core | `pgmq` enqueue + `fw_workflow_executions` lifecycle after successful ingest | | FW-06 (Workflow Builder) | Intra-core | Target `fw_workflow_definitions`; optional “webhook trigger” authoring surface | | FW-53 (Workflow Rate Limits) | Intra-core | Limits may apply after enqueue (document interaction; no bypass) | *** ## API / Platform Contracts ### Public Edge Function: `fw-webhook-receiver` * **Supabase URL:** `POST https://{project_ref}.supabase.co/functions/v1/fw-webhook-receiver/{org_slug}/{endpoint_slug}` * **Path parsing:** Full pathname includes `/functions/v1/` prefix; extract `org_slug` and `endpoint_slug` as the two segments **after** `fw-webhook-receiver` (see [process-inbound-webhook](../../../supabase/functions/process-inbound-webhook/index.ts) for a similar `url.pathname.split('/')` approach). * **Optional app gateway:** Same path may be exposed as `POST https://{app_host}/api/webhooks/{org_slug}/{endpoint_slug}` via reverse proxy (implementation choice). * **JWT:** `verify_jwt: false` — callers are not Supabase users; authentication is endpoint-specific (API key, HMAC, or bearer JWT from external IdP). * **Runtime credentials:** Service role used **only** inside the function after successful endpoint auth, to call SECURITY DEFINER ingest RPC and write logs (see spec). * **PF-42 call pattern:** Call `pf_check_rate_limit` with `p_endpoint` set to `fw_webhook:` plus the endpoint UUID — align with [check-rate-limit](../../../supabase/functions/check-rate-limit/index.ts) (fail-open on DB error is PF-42 default; product may choose fail-closed for abuse-sensitive ingress in a follow-up). ### SECURITY DEFINER RPC (planned) * **`fw_webhook_ingest(...)`** (name illustrative): Validates resolved `organization_id`, endpoint active state, performs idempotency insert, creates workflow execution row, calls existing FW-46 enqueue path (`pgmq.send` / same contract as `fw_process_domain_event` worker flow). Invoked by edge with service role; **not** exposed to `anon`. * **`fw_webhook_replay_log(...)`**: Authenticated + `fw_has_org_access`; re-enqueues from stored encrypted payload metadata. Exact signatures and payload shapes are defined at implementation time in migration + this doc (update § API when landed). ### Idempotency * Header `X-Idempotency-Key` optional. Dedup key: `(endpoint_id, idempotency_key)` when header present; else behavior unchanged from spec (payload hash window) for backward compatibility. *** ## Security and Tenant Isolation * No cross-core imports: all logic stays in FW + platform dependencies. * Webhook payloads may contain PHI: store ciphertext only; restrict log body viewing to permission `fw.webhooks.view` (and replay to `fw.webhooks.replay`). * Edge must not log raw bodies to console or external APM in production. * RLS on `fw_webhook_*` tables uses `fw_has_org_access(organization_id, auth.uid())` for authenticated app users; service role bypasses RLS for ingest path only inside controlled RPC. *** ## Database: RLS policy patterns (Supabase / Postgres) **Canonical DDL** for `fw_webhook_endpoints`, `fw_webhook_logs`, and `fw_webhook_secrets` (policies, helpers, and indexes) lives in **[FW-59-external-webhook-triggers.md](../../../specs/fw/specs/FW-59-external-webhook-triggers.md)** § Data Model. This section summarizes **patterns** implementers must follow. ### Authenticated app users (`authenticated`) * **Tenant scope:** Every policy on org-scoped tables uses **`fw_has_org_access(organization_id, auth.uid())`** (or, for child rows keyed by `endpoint_id`, a **SECURITY DEFINER** helper such as `fw_webhook_endpoint_org_id(endpoint_id)` so policies do not recurse — constitution §5.7). * **WITH CHECK on UPDATE:** All `FOR UPDATE` policies must include **`WITH CHECK`** mirroring `USING` (constitution §5.2.9). **Example — org members read endpoints:** ```sql theme={null} CREATE POLICY "fw_webhook_endpoints_select" ON fw_webhook_endpoints FOR SELECT TO authenticated USING (fw_has_org_access(organization_id, auth.uid())); ``` **Example — org members read delivery logs:** ```sql theme={null} CREATE POLICY "fw_webhook_logs_select" ON fw_webhook_logs FOR SELECT TO authenticated USING (fw_has_org_access(organization_id, auth.uid())); ``` ### Ingest path (service role + RPC only) * **No** broad **`anon` INSERT** on `fw_webhook_logs` or execution tables for ingress. * The Edge Function uses the **service role** only to call **`SECURITY DEFINER` RPCs** (e.g. `fw_webhook_ingest`) that validate `organization_id`, endpoint state, idempotency, and then enqueue (FW-46). All writes from the public internet are funneled through that contract. *** ## Database: required and recommended indexes **Minimum (tenant + ingress resolution):** | Table | Index | Purpose | | ---------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------------- | | `fw_webhook_endpoints` | `(organization_id, endpoint_slug)` partial where active | Resolve `org_slug` + `endpoint_slug` → row for `fw-webhook-receiver` | | `fw_webhook_endpoints` | `(organization_id, is_active)` | Org admin UI lists / filters | | `fw_webhook_endpoints` | `(organization_id, workflow_definition_id)` (recommended) | “All endpoints for this workflow” admin views | | `fw_webhook_logs` | `(endpoint_id, received_at DESC)` | Endpoint activity timeline | | `fw_webhook_logs` | `(organization_id, processing_status)` partial for failed/pending | Ops / retry dashboards | | `fw_webhook_logs` | `(endpoint_id, idempotency_key)` **UNIQUE** where `idempotency_key IS NOT NULL` | Dedup per FR / CONTEXT | | `fw_webhook_secrets` | `(endpoint_id)` partial where `is_active` | Active secret rotation lookups | Illustrative DDL is in the spec; migrations must stay the source of truth. **Foreign keys:** `workflow_definition_id` → `fw_workflow_definitions(id)` must keep **`ON DELETE RESTRICT`** (or equivalent) so executions are not orphaned silently. *** ## Webhook execution isolation (multi-tenant / PHI-safe) 1. **Resolve tenant first:** Map path `org_slug` → `pf_organizations.slug` → **`organization_id`**. Reject if unknown slug. 2. **Resolve endpoint second:** Load `fw_webhook_endpoints` by **`(organization_id, endpoint_slug)`** (and active / not suspended). Never select an endpoint by slug alone. 3. **Bind workflow to org:** The endpoint’s **`workflow_definition_id`** MUST reference a row whose **`organization_id`** matches the endpoint’s **`organization_id`** (enforce in RPC before enqueue; add a DB constraint or validate in `fw_webhook_ingest` if the schema allows cross-row checks). 4. **Enqueue:** Create / link **`fw_workflow_executions`** only for that definition and org; FW-53 and other limits apply after enqueue — **no bypass**. 5. **Logs / PHI:** Encrypted body in `BYTEA` only; structured edge logs contain **no** raw payload, headers with secrets redacted, and correlation ids only. *** ## Monitoring, alerting, and PF-42 fail-open behavior PF-42’s **`check-rate-limit`** Edge Function **fails open** on DB or unexpected errors (returns `allowed: true`) and logs: * `DB error in rate limit check` with `{ error, orgId, endpoint }` * `Unexpected error in rate limit check` with `{ error }` **`fw-webhook-receiver`** MUST emit **structured logs** on the same failure modes when calling `pf_check_rate_limit` (or inline RPC), with fields aligned for dashboards: | Field | Description | | ----------------------------- | ------------------------------------------------------------------------------------------------ | | `function` | `'fw-webhook-receiver'` | | `event` | `'pf42_rate_limit_db_error'` \| `'pf42_rate_limit_fail_open'` (or single code with `error_type`) | | `error_type` | `'db'` \| `'unexpected'` | | `error` | Sanitized message string (no secrets / no payload body) | | `organization_id` | When known after org resolution | | `endpoint_id` | When known after endpoint resolution | | `org_slug` | Path segment | | `endpoint_slug` | Path segment | | `p_endpoint` / rate-limit key | e.g. `fw_webhook:{endpoint_id}` for correlation with `pf_rate_limits` | ### Alert thresholds (starting defaults — tune per environment) * **≥ 10** `pf42_*` / `pf42_rate_limit_*` log lines **per minute** per project (or per `org_slug` if high-volume single-tenant). * **OR ≥ 5%** of `fw-webhook-receiver` requests hitting the fail-open path over a **5-minute** rolling window. ### Channels and escalation * **Slack:** `#platform-alerts` or `#fw-webhooks` for warning-level sustained rates. * **Pager / on-call (SEV-2+):** Opsgenie/PagerDuty rotation shared by **Platform SRE** and **FW module owner** when fail-open volume indicates abuse or prolonged PF-42/DB outage. ### Incident response playbook (short) 1. **Triage:** Confirm spike in Edge logs / Supabase metrics; segment by `org_slug` and `endpoint_id`. 2. **Infra:** Check Postgres availability, `pf_check_rate_limit` errors, and `pf_rate_limits` row presence for affected endpoints (FR-2.1a). 3. **Mitigate:** Enable **`FW_WEBHOOK_PF42_FAIL_CLOSED`** (see below) if fail-open is amplifying abuse or hiding enforcement; communicate to affected orgs if ingress is degraded. 4. **Postmortem:** For SEV-2+, document timeline, blast radius, and corrective actions within **72 hours**. ### Feature flag: fail-closed toggle | | | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Name** | `FW_WEBHOOK_PF42_FAIL_CLOSED` | | **Default** | `false` (fail-open; parity with PF-42 / `check-rate-limit`) | | **Meaning** | When `true`, `fw-webhook-receiver` treats PF-42 / rate-limit check failures as **deny** (e.g. 503 or 429 per product choice) instead of allowing ingress. | | **Owners** | FW module owner + Platform (rate limiting / SRE) | | **Implementation** | Register in `pf_feature_flags` / consume via platform feature-flag pattern — see **[PF-45 Feature Flags](./feature-flags-integration.md)**. | *** ## Event Contracts FW-59 does not publish a new cross-core domain event in Phase 1. Optional future: `fw_webhook_received` for observability (defer). *** ## Related Docs * [FW-59 Spec](../../../specs/fw/specs/FW-59-external-webhook-triggers.md) * [FW-59 CONTEXT](../../../specs/fw/specs/FW-59-external-webhook-triggers-CONTEXT.md) — ingress edge decisions (PF-42 fail-open; points here for monitoring) * [FW-46 Durable Execution Worker](./durable-execution-worker-integration.md) * [FW-16 Event-Based Workflow Triggers](./event-based-workflow-triggers-integration.md) * [PF-42 Rate Limiting](./rate-limiting-throttling-integration.md) * [PF-45 Feature Flags](./feature-flags-integration.md) — `FW_WEBHOOK_PF42_FAIL_CLOSED` consumer pattern * [check-rate-limit Edge Function](../../../supabase/functions/check-rate-limit/index.ts) — reference log messages for fail-open * [CROSS\_CORE\_INTEGRATIONS.md](./CROSS_CORE_INTEGRATIONS.md)