Encore OS Data Management Architecture

Purpose: Unified guide to the data management platform capabilities in Encore OS. This document explains how PF-15 (Picklists), PF-16 (Custom Fields), PF-17 (Field Configuration), PF-23 (Object Browser), PF-24 (Custom Objects), PF-25 (Raw Data Editor), and PF-26 (Object Permissions) work together.

​

Overview

​

Component Diagram

• Solid arrows: Direct dependencies (PF-16 uses PF-15)
• Dotted arrows: Consumer relationships (cores use platform features)

​

Component Relationships

​

Data Flow

1. Picklists (PF-15) → Custom Fields (PF-16)
   • Select/multiselect fields reference picklists via picklist_id
   • Single source of truth for dropdown values
2. Custom Fields (PF-16) → Field Configuration (PF-17)
   • Field definitions are configured for visibility, ordering, permissions
   • Field Configuration manages how Custom Fields appear in UI
3. Custom Fields (PF-16) → Custom Objects (PF-24)
   • Custom objects use PF-16 field definitions for their schema
   • Reuses validation rules and field types
4. Custom Objects (PF-24) → Object Browser (PF-23)
   • Custom objects automatically appear in Data Manager
   • Unified view of core objects + custom objects
5. Object Browser (PF-23) → Raw Data Editor (PF-25)
   • Raw Data Editor is a tab in Object Browser detail view
   • Enables bulk editing of object records
6. Object Browser (PF-23) → Object Permissions (PF-26)
   • Permissions tab in Object Browser detail view
   • Configure access control per object

​

Decision Tree: When to Use What

Need to customize dropdown values?
├─ Yes → Use PF-15 Picklists
│   └─ Create picklist → Use in forms/fields
│
Need to add fields to existing entities?
├─ Yes → Use PF-16 Custom Field Definitions
│   ├─ Define field (type, validation, picklist)
│   └─ Use CustomFieldsSection component in forms
│
Need to control field visibility/ordering?
├─ Yes → Use PF-17 Field Configuration
│   ├─ Configure layout per entity type
│   └─ Use ConfigurableForm component
│
Need to create new entity type?
├─ Yes → Use PF-24 Custom Objects
│   ├─ Create object → Add fields (PF-16)
│   └─ Object appears in Data Manager (PF-23)
│
Need to browse/discover all data?
├─ Yes → Use PF-23 Object Browser
│   ├─ View all objects (core + custom)
│   └─ Access Raw Data Editor (PF-25)
│
Need bulk data operations?
├─ Yes → Use PF-25 Raw Data Editor
│   ├─ Export to CSV
│   ├─ Import from CSV
│   └─ Inline editing
│
Need object-level permissions?
├─ Yes → Use PF-26 Object Permissions
│   ├─ Configure per role
│   └─ Field-level permissions

​

Integration Patterns

​

Pattern 1: Platform Integration Layer

// ✅ CORRECT: Use Platform Integration Layer
import { usePicklistItems } from '@/platform/picklists';
import { CustomFieldsSection } from '@/platform/custom-fields';
import { ConfigurableForm } from '@/platform/field-config';

// ❌ WRONG: Direct import from cores
import { usePicklists } from '@/cores/hr/picklists';

​

Pattern 2: Event-Based Integration

• org_data_changed events published via pg_notify
• Modules subscribe and update local data structures
• See PF-18 spec for event contracts

​

Pattern 3: API Contracts

• Object discovery API
• Field metadata API
• Record count API
• See PF-23 spec for API contracts

​

Common Workflows

​

Workflow 1: Adding Custom Data to Core Entities

1. Create Picklist (if needed) - PF-15
   // If badge number needs validation options, create picklist
   // Otherwise, skip to step 2
   
2. Define Custom Field - PF-16
   // Admin creates field definition at /settings/custom-fields
   // Entity: hr_employees
   // Field: badge_number
   // Type: text
   // Validation: pattern "^[A-Z]{3}-\\d{4}$"
   
3. Configure Field Layout - PF-17 (optional)
   // Admin configures field visibility/ordering at /settings/field-config
   // Group: "Employment Details"
   // Order: After "employee_id"
   
4. Use in Forms - Module Integration
   // HR module uses CustomFieldsSection component
   <CustomFieldsSection
     entityType="hr_employees"
     values={employee.custom_fields}
     onChange={handleCustomFieldChange}
   />
   
5. View in Data Manager - PF-23
   // Field appears in Object Browser → Employees → Fields tab
   // Shows in Raw Data Editor (PF-25)
   

​

Workflow 2: Creating a Custom Entity

1. Create Custom Object - PF-24
   // Admin creates object at Data Manager → Create Object
   // Name: "Clinical Licenses"
   // API Name: "clinical_licenses"
   // Category: "HR & Workforce"
   
2. Add Fields - PF-16 + PF-24
   // Add fields using PF-16 field definitions
   // - employee_id (lookup to hr_employees)
   // - license_type (select, uses picklist)
   // - license_number (text, required)
   // - expiration_date (date, required)
   
3. Import Existing Data - PF-25
   // Export existing data to CSV
   // Map columns to fields
   // Import with validation
   
4. Configure Permissions - PF-26
   // Set object-level permissions
   // HR managers: full access
   // Staff: view only
   
5. View in Data Manager - PF-23
   // Object appears in Object Browser
   // Can favorite, categorize, browse records
   

​

Workflow 3: Managing Organization Data Structure

1. Browse Objects - PF-23
   // Data Manager → Objects
   // View all core objects + custom objects
   
2. Edit Metadata - PF-23
   // Click object → Settings tab
   // Update description, category, icon
   
3. Manage Categories - PF-23
   // Data Manager → Categories
   // Create/edit/delete categories
   // Assign objects to categories
   
4. Set Favorites - PF-23
   // Star icon on frequently-used objects
   // Filter by favorites
   

​

Workflow 4: Bulk Data Operations

⚠️ Security Note - PHI Protection for CSV Operations: CSV exports may contain Protected Health Information (PHI) and require additional safeguards:

• Encryption: Ensure CSV files are encrypted at rest (disk encryption) and in transit (secure download over HTTPS)
• Access Control: Restrict export functionality to authorized staff with appropriate permissions; verify user has {module}.{entity}.export permission
• Audit Logging: All export and import operations are logged (who, what data, when, row counts) in pf_audit_log
• Import Validation: Imported CSVs are integrity-checked (column validation, data type verification, row count confirmation) to detect tampering
• Secure Handling: Downloaded CSV files should be stored in encrypted locations; delete files after import is confirmed successful
• Offline Security: When editing CSVs offline, use password-protected devices; avoid storing on shared/unencrypted drives
• High-Sensitivity Orgs: Organizations handling highly sensitive PHI (e.g., behavioral health clinical notes) should consider disabling CSV export entirely via organization settings; use secure API-based integrations instead

See docs/security/DATA_EXPORT_POLICY.md for complete export security guidelines.

1. Export Data - PF-25
   // Object Browser → Employees → Raw Data tab
   // Click Export → CSV download
   
2. Edit Offline
   // Open CSV in Excel/Google Sheets
   // Make bulk edits
   // Save CSV
   
3. Import Updates - PF-25
   // Raw Data tab → Import
   // Upload CSV
   // Map columns
   // Preview validation
   // Confirm import
   

​

Quick Reference Table

​

Module Adoption

• ✅ Complete
• ⚠️ Planned
• ❌ Not Started / N/A

​

Performance Considerations

​

Query Optimization

• Picklists: Cached with 5-minute staleTime (QueryClient config)
• Custom Fields: Indexed on (organization_id, entity_type, is_active)
• Object Metadata: Computed fields (field_count, record_count) cached
• Raw Data: Pagination required (25/50/100 per page)

​

Scalability Limits

​

Security Considerations

​

Multi-Tenant Isolation

• RLS policies on all tables (organization_id filter)
• Application-level defense-in-depth (explicit organization_id in queries)
• No cross-organization data access

​

Role-Based Access

​

Related Documentation

• PF-15: Picklist System
• PF-16: Custom Field Definitions
• PF-17: Entity Field Configuration
• PF-23: Data Manager - Object Browser
• PF-24: Custom Objects
• PF-25: Raw Data Editor
• PF-26: Object Permissions
• Platform Integration Layers
• Constitution (current: see docs/VERSIONS.md)