Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.encoreos.io/llms.txt

Use this file to discover all available pages before exploring further.

Version: 1.0.0
Last Updated: 2026-03-18
Status: Active
Module: FW

Overview

The durable execution worker processes queued workflow executions via pgmq. It runs on a per-minute cron schedule and processes batches per organization with semaphore-based concurrency control.

Configuration

Worker settings are stored in fw_module_settings (one row per organization):
SettingColumnDefaultDescription
Worker Enabledfw_execution_worker_enabledfalseMaster toggle for the worker
Batch Sizefw_worker_batch_size10Messages per batch (1–50)
Visibility Timeoutfw_worker_visibility_timeout_seconds30Seconds before unacked message becomes visible again (30–600)

Enabling the Worker

  1. Navigate to FW SettingsWorker tab
  2. Toggle Enable Execution Worker on
  3. Adjust batch size and visibility timeout as needed
  4. Save settings
WorkloadBatch SizeVisibility Timeout
Low volume (<100/day)530s
Medium volume (100–1000/day)1060s
High volume (>1000/day)25120s

Queue Monitoring

Check Queue Depth

SELECT count(*) AS pending_messages
FROM pgmq.q_workflow_execution_queue
WHERE vt <= now();

Check Dead Letter Queue

SELECT count(*) AS dlq_messages
FROM pgmq.q_workflow_dlq;

-- View DLQ message details
SELECT msg_id, enqueued_at, message->>'execution_id' AS execution_id,
       message->>'error' AS error, message->>'failed_at' AS failed_at
FROM pgmq.q_workflow_dlq
ORDER BY enqueued_at DESC
LIMIT 20;

Worker Run History

SELECT organization_id, worker_running, worker_last_run_at, worker_last_batch_size
FROM fw_module_settings
WHERE fw_execution_worker_enabled = true;

DLQ Handling

Messages are routed to the DLQ after 5 failed attempts. To retry a DLQ message:
-- Move a message back to the main queue
WITH dlq_msg AS (
  SELECT msg_id, message FROM pgmq.q_workflow_dlq WHERE msg_id = :msg_id
)
SELECT pgmq.send('workflow_execution_queue', 
  jsonb_set(message, '{attempt}', '1'::jsonb)
) FROM dlq_msg;

-- Then delete from DLQ
SELECT pgmq.delete('workflow_dlq', :msg_id);

Purge Old DLQ Messages

DELETE FROM pgmq.q_workflow_dlq
WHERE enqueued_at < now() - interval '30 days';

Troubleshooting

See Runbook for operational procedures.