> ## Documentation Index
> Fetch the complete documentation index at: https://docs.encoreos.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Encore Health OS Data Management Architecture

> Version: 1.0 Last Updated: 2026-01-08 Constitution Reference: Section 1.3 (Integration Patterns), Section 5.2 (Database Design)

**Version:** 1.0\
**Last Updated:** 2026-01-08\
**Constitution Reference:** Section 1.3 (Integration Patterns), Section 5.2 (Database Design)

> **Purpose:** Unified guide to the data management platform capabilities in Encore Health OS. This document explains how PF-15 (Picklists), PF-16 (Custom Fields), PF-17 (Field Configuration), PF-23 (Object Browser), PF-24 (Custom Objects), PF-25 (Raw Data Editor), and PF-26 (Object Permissions) work together.

***

## Overview

The Encore Health OS data management platform provides a comprehensive set of capabilities for organizations to customize, manage, and interact with their data. These capabilities are built on Platform Foundation (PF) and provide reusable patterns for all domain cores.

***

## Component Diagram

```mermaid theme={null}
flowchart TB
    subgraph platform [Platform Data Layer]
        PF15[PF-15 Picklists<br/>Dropdown Values]
        PF16[PF-16 Custom Fields<br/>Field Definitions]
        PF17[PF-17 Field Config<br/>Layout & Visibility]
        PF23[PF-23 Object Browser<br/>Discovery & Metadata]
        PF24[PF-24 Custom Objects<br/>User-Defined Entities]
        PF25[PF-25 Raw Data Editor<br/>Bulk Operations]
        PF26[PF-26 Object Permissions<br/>Access Control]
    end
    
    subgraph cores [Domain Cores]
        HR[HR Core]
        FA[FA Core]
        RH[RH Core]
        FW[FW Core]
        GR[GR Core]
        FM[FM Core]
        IT[IT Core]
    end
    
    PF15 --> PF16
    PF16 --> PF17
    PF16 --> PF24
    PF24 --> PF23
    PF23 --> PF25
    PF23 --> PF26
    
    PF15 -.-> HR
    PF15 -.-> FA
    PF15 -.-> RH
    PF16 -.-> HR
    PF16 -.-> FA
    PF16 -.-> RH
    PF17 -.-> HR
    PF17 -.-> FA
    PF23 -.-> HR
    PF23 -.-> FA
    PF24 -.-> HR
    PF24 -.-> FA
    
    style PF15 fill:#e0f2fe
    style PF16 fill:#e0f2fe
    style PF17 fill:#e0f2fe
    style PF23 fill:#e0f2fe
    style PF24 fill:#e0f2fe
    style PF25 fill:#e0f2fe
    style PF26 fill:#e0f2fe
```

**Legend:**

* Solid arrows: Direct dependencies (PF-16 uses PF-15)
* Dotted arrows: Consumer relationships (cores use platform features)

***

## Component Relationships

### Data Flow

1. **Picklists (PF-15)** → **Custom Fields (PF-16)**
   * Select/multiselect fields reference picklists via `picklist_id`
   * Single source of truth for dropdown values

2. **Custom Fields (PF-16)** → **Field Configuration (PF-17)**
   * Field definitions are configured for visibility, ordering, permissions
   * Field Configuration manages how Custom Fields appear in UI

3. **Custom Fields (PF-16)** → **Custom Objects (PF-24)**
   * Custom objects use PF-16 field definitions for their schema
   * Reuses validation rules and field types

4. **Custom Objects (PF-24)** → **Object Browser (PF-23)**
   * Custom objects automatically appear in Data Manager
   * Unified view of core objects + custom objects

5. **Object Browser (PF-23)** → **Raw Data Editor (PF-25)**
   * Raw Data Editor is a tab in Object Browser detail view
   * Enables bulk editing of object records

6. **Object Browser (PF-23)** → **Object Permissions (PF-26)**
   * Permissions tab in Object Browser detail view
   * Configure access control per object

***

## Decision Tree: When to Use What

```
Need to customize dropdown values?
├─ Yes → Use PF-15 Picklists
│   └─ Create picklist → Use in forms/fields
│
Need to add fields to existing entities?
├─ Yes → Use PF-16 Custom Field Definitions
│   ├─ Define field (type, validation, picklist)
│   └─ Use CustomFieldsSection component in forms
│
Need to control field visibility/ordering?
├─ Yes → Use PF-17 Field Configuration
│   ├─ Configure layout per entity type
│   └─ Use ConfigurableForm component
│
Need to create new entity type?
├─ Yes → Use PF-24 Custom Objects
│   ├─ Create object → Add fields (PF-16)
│   └─ Object appears in Data Manager (PF-23)
│
Need to browse/discover all data?
├─ Yes → Use PF-23 Object Browser
│   ├─ View all objects (core + custom)
│   └─ Access Raw Data Editor (PF-25)
│
Need bulk data operations?
├─ Yes → Use PF-25 Raw Data Editor
│   ├─ Export to CSV
│   ├─ Import from CSV
│   └─ Inline editing
│
Need object-level permissions?
├─ Yes → Use PF-26 Object Permissions
│   ├─ Configure per role
│   └─ Field-level permissions
```

***

## Integration Patterns

### Pattern 1: Platform Integration Layer

All data management features are provided as Platform Integration Layers:

| Feature                  | Location                       | Public API                                                               |
| ------------------------ | ------------------------------ | ------------------------------------------------------------------------ |
| PF-15 Picklists          | `/src/platform/picklists/`     | `usePicklists`, `usePicklistItems`, `PicklistSelector`                   |
| PF-16 Custom Fields      | `/src/platform/custom-fields/` | `useCustomFieldDefinitions`, `CustomFieldInput`, `CustomFieldsSection`   |
| PF-17 Field Config       | `/src/platform/field-config/`  | `useEntityFieldConfigs`, `ConfigurableForm`, `ConfigurableDetailView`    |
| PF-23 Data Manager       | `/src/platform/data-manager/`  | `useObjectMetadata`, `DataManagerPage`, `ObjectDetailPage`               |
| PF-24 Custom Objects     | `/src/platform/data-manager/`  | `useCustomObjects`, `useCustomObjectRecords`, `CreateCustomObjectDialog` |
| PF-25 Raw Data Editor    | `/src/platform/data-manager/`  | `useRawData`, `RawDataTable`, `RawDataImportWizard`                      |
| PF-26 Object Permissions | `/src/platform/data-manager/`  | `useObjectPermissions`, `ObjectPermissionsTab`                           |

**Usage Example:**

```typescript theme={null}
// ✅ CORRECT: Use Platform Integration Layer
import { usePicklistItems } from '@/platform/picklists';
import { CustomFieldsSection } from '@/platform/custom-fields';
import { ConfigurableForm } from '@/platform/field-config';

// ❌ WRONG: Direct import from cores
import { usePicklists } from '@/cores/hr/picklists';
```

### Pattern 2: Event-Based Integration

Organizational data changes (PF-18) use event-based integration:

* `org_data_changed` events published via `pg_notify`
* Modules subscribe and update local data structures
* See PF-18 spec for event contracts

### Pattern 3: API Contracts

Data Manager provides query APIs for object metadata:

* Object discovery API
* Field metadata API
* Record count API
* See PF-23 spec for API contracts

***

## Common Workflows

### Workflow 1: Adding Custom Data to Core Entities

**Scenario:** Add "Badge Number" field to HR Employees

1. **Create Picklist (if needed)** - PF-15
   ```typescript theme={null}
   // If badge number needs validation options, create picklist
   // Otherwise, skip to step 2
   ```

2. **Define Custom Field** - PF-16
   ```typescript theme={null}
   // Admin creates field definition at /settings/custom-fields
   // Entity: hr_employees
   // Field: badge_number
   // Type: text
   // Validation: pattern "^[A-Z]{3}-\\d{4}$"
   ```

3. **Configure Field Layout** - PF-17 (optional)
   ```typescript theme={null}
   // Admin configures field visibility/ordering at /settings/field-config
   // Group: "Employment Details"
   // Order: After "employee_id"
   ```

4. **Use in Forms** - Module Integration
   ```typescript theme={null}
   // HR module uses CustomFieldsSection component
   <CustomFieldsSection
     entityType="hr_employees"
     values={employee.custom_fields}
     onChange={handleCustomFieldChange}
   />
   ```

5. **View in Data Manager** - PF-23
   ```typescript theme={null}
   // Field appears in Object Browser → Employees → Fields tab
   // Shows in Raw Data Editor (PF-25)
   ```

### Workflow 2: Creating a Custom Entity

**Scenario:** Create "Clinical Licenses" custom object

1. **Create Custom Object** - PF-24
   ```typescript theme={null}
   // Admin creates object at Data Manager → Create Object
   // Name: "Clinical Licenses"
   // API Name: "clinical_licenses"
   // Category: "HR & Workforce"
   ```

2. **Add Fields** - PF-16 + PF-24
   ```typescript theme={null}
   // Add fields using PF-16 field definitions
   // - employee_id (lookup to hr_employees)
   // - license_type (select, uses picklist)
   // - license_number (text, required)
   // - expiration_date (date, required)
   ```

3. **Import Existing Data** - PF-25
   ```typescript theme={null}
   // Export existing data to CSV
   // Map columns to fields
   // Import with validation
   ```

4. **Configure Permissions** - PF-26
   ```typescript theme={null}
   // Set object-level permissions
   // HR managers: full access
   // Staff: view only
   ```

5. **View in Data Manager** - PF-23
   ```typescript theme={null}
   // Object appears in Object Browser
   // Can favorite, categorize, browse records
   ```

### Workflow 3: Managing Organization Data Structure

**Scenario:** Organize all objects by category

1. **Browse Objects** - PF-23
   ```typescript theme={null}
   // Data Manager → Objects
   // View all core objects + custom objects
   ```

2. **Edit Metadata** - PF-23
   ```typescript theme={null}
   // Click object → Settings tab
   // Update description, category, icon
   ```

3. **Manage Categories** - PF-23
   ```typescript theme={null}
   // Data Manager → Categories
   // Create/edit/delete categories
   // Assign objects to categories
   ```

4. **Set Favorites** - PF-23
   ```typescript theme={null}
   // Star icon on frequently-used objects
   // Filter by favorites
   ```

### Workflow 4: Bulk Data Operations

**Scenario:** Export employees, edit offline, import updates

> **⚠️ Security Note - PHI Protection for CSV Operations:**
>
> CSV exports may contain Protected Health Information (PHI) and require additional safeguards:
>
> * **Encryption:** Ensure CSV files are encrypted at rest (disk encryption) and in transit (secure download over HTTPS)
> * **Access Control:** Restrict export functionality to authorized staff with appropriate permissions; verify user has `{module}.{entity}.export` permission
> * **Audit Logging:** All export and import operations are logged (who, what data, when, row counts) in `pf_audit_log`
> * **Import Validation:** Imported CSVs are integrity-checked (column validation, data type verification, row count confirmation) to detect tampering
> * **Secure Handling:** Downloaded CSV files should be stored in encrypted locations; delete files after import is confirmed successful
> * **Offline Security:** When editing CSVs offline, use password-protected devices; avoid storing on shared/unencrypted drives
> * **High-Sensitivity Orgs:** Organizations handling highly sensitive PHI (e.g., behavioral health clinical notes) should consider disabling CSV export entirely via organization settings; use secure API-based integrations instead
>
> See `docs/security/DATA_EXPORT_POLICY.md` for complete export security guidelines.

1. **Export Data** - PF-25
   ```typescript theme={null}
   // Object Browser → Employees → Raw Data tab
   // Click Export → CSV download
   ```

2. **Edit Offline**
   ```typescript theme={null}
   // Open CSV in Excel/Google Sheets
   // Make bulk edits
   // Save CSV
   ```

3. **Import Updates** - PF-25
   ```typescript theme={null}
   // Raw Data tab → Import
   // Upload CSV
   // Map columns
   // Preview validation
   // Confirm import
   ```

***

## Quick Reference Table

| Need                   | Solution                 | Key Component              | Location                        |
| ---------------------- | ------------------------ | -------------------------- | ------------------------------- |
| Custom dropdown values | PF-15 Picklists          | `PicklistSelector`         | `/settings/picklists`           |
| Add field to entity    | PF-16 Custom Fields      | `CustomFieldsSection`      | `/settings/custom-fields`       |
| Control field layout   | PF-17 Field Config       | `ConfigurableForm`         | `/settings/field-config`        |
| Browse all objects     | PF-23 Object Browser     | `DataManagerPage`          | `/data-manager`                 |
| Create new entity      | PF-24 Custom Objects     | `CreateCustomObjectDialog` | `/data-manager` → Create Object |
| Bulk edit records      | PF-25 Raw Data Editor    | `RawDataTable`             | Object detail → Raw Data tab    |
| Set permissions        | PF-26 Object Permissions | `ObjectPermissionsTab`     | Object detail → Permissions tab |

***

## Module Adoption

| Module | Picklists | Custom Fields | Field Config   | Custom Objects | Data Manager |
| ------ | --------- | ------------- | -------------- | -------------- | ------------ |
| HR     | ✅ 8       | ✅ 12+         | ✅ 2 layouts    | ⚠️ Planned     | ✅ Browse     |
| FA     | ✅ 12      | ✅ 8+          | ⚠️ Planned     | ❌ N/A          | ✅ Browse     |
| RH     | ✅ 4+      | ✅ 6+          | ⚠️ Planned     | ⚠️ Planned     | ✅ Browse     |
| FW     | ✅ 4       | ✅ Form-level  | ✅ Form builder | ❌ N/A          | ✅ Browse     |
| GR     | ✅ 2+      | ✅ 3+          | ❌ Not Started  | ❌ N/A          | ✅ Browse     |
| FM     | ✅ 2+      | ✅ 4+          | ❌ Not Started  | ❌ N/A          | ✅ Browse     |
| IT     | ✅ 2+      | ✅ 3+          | ❌ Not Started  | ❌ N/A          | ✅ Browse     |

**Legend:**

* ✅ Complete
* ⚠️ Planned
* ❌ Not Started / N/A

***

## Performance Considerations

### Query Optimization

* **Picklists:** Cached with 5-minute staleTime (QueryClient config)
* **Custom Fields:** Indexed on `(organization_id, entity_type, is_active)`
* **Object Metadata:** Computed fields (`field_count`, `record_count`) cached
* **Raw Data:** Pagination required (25/50/100 per page)

### Scalability Limits

| Feature                   | Limit     | Mitigation                              |
| ------------------------- | --------- | --------------------------------------- |
| Picklists per org         | 100       | Archive inactive picklists              |
| Custom fields per entity  | 50        | Group fields into sections              |
| Custom objects per org    | 100       | Archive inactive objects                |
| Records per custom object | 100,000   | Use pagination, indexes                 |
| Hierarchy depth           | 10 levels | Materialized views for deep hierarchies |

***

## Security Considerations

### Multi-Tenant Isolation

All data management features enforce organization-level isolation:

* RLS policies on all tables (`organization_id` filter)
* Application-level defense-in-depth (explicit `organization_id` in queries)
* No cross-organization data access

### Role-Based Access

| Feature            | View        | Create/Edit | Delete                       |
| ------------------ | ----------- | ----------- | ---------------------------- |
| Picklists          | org\_member | org\_admin  | org\_admin (non-system only) |
| Custom Fields      | org\_member | org\_admin  | org\_admin                   |
| Field Config       | org\_member | org\_admin  | org\_admin                   |
| Object Metadata    | org\_member | org\_admin  | org\_admin                   |
| Custom Objects     | org\_member | org\_admin  | org\_admin                   |
| Raw Data           | org\_member | org\_member | org\_member                  |
| Object Permissions | org\_member | org\_admin  | org\_admin                   |

***

## Related Documentation

* [PF-15: Picklist System](../../specs/pf/specs/PF-15-picklist-system.md)
* [PF-16: Custom Field Definitions](../../specs/pf/specs/PF-16-custom-field-definitions.md)
* [PF-17: Entity Field Configuration](../../specs/pf/specs/PF-17-entity-field-configuration.md)
* [PF-23: Data Manager - Object Browser](../../specs/pf/specs/PF-23-data-manager-object-browser.md)
* [PF-24: Custom Objects](../../specs/pf/specs/PF-24-custom-objects.md)
* [PF-25: Raw Data Editor](../../specs/pf/specs/PF-25-raw-data-editor.md)
* [PF-26: Object Permissions](../../specs/pf/specs/PF-26-object-permissions.md)
* [Platform Integration Layers](./integrations/PLATFORM_INTEGRATION_LAYERS.md)
* [Constitution](../../constitution.md) (current: see docs/VERSIONS.md)

***

**Last Updated:** 2026-01-08\
**Next Review:** Quarterly or when new data management features are added