> ## Documentation Index > Fetch the complete documentation index at: https://docs.encoreos.io/llms.txt > Use this file to discover all available pages before exploring further. # Bulk Import Review: PF & Module Inventory vs Rippling-Style Flow > Purpose: Compare Encore Health OS's bulk import setup across PF and modules to a Rippling-style "HR history import" flow, and recommend improvements and enhanc… **Purpose:** Compare Encore Health OS's bulk import setup across PF and modules to a Rippling-style "HR history import" flow, and recommend improvements and enhancements. **References:** [bulk-import-migration.md](./bulk-import-migration.md), BULK\_IMPORT\_MIGRATION\_RECOMMENDATIONS.md, [.cursor/rules/bulk-import-patterns.md](../../../.cursor/rules/bulk-import-patterns.md). **Last Updated:** 2026-02-16 *** ## 1. Rippling-Style Reference (What We Like) From the Rippling "Import HR history" flow: | Aspect | Rippling | Encore Health OS alignment | | -------------------- | -------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Entry point** | Company Settings → Data import tab → Import HR history | We use context-specific entry (e.g. Employee Directory, Contacts, Credentialing). No central "Data import" hub. | | **Disclaimer** | "Importing HR history will never update active employee information or trigger downstream effects." | HR employee import creates net-new employees/sites/depts; we don't have a separate "history only" import. | | **Attributes** | Up to 32 attributes; categories (compensation, position, contract). | HR employee template has fixed columns (name, title, department, compensation, status, dates, etc.). | | **Step 1** | Start → Review process timeline → Continue | We have Upload → Preview (stats) → Role mapping → Review → Import → Complete. No explicit "process timeline" step. | | **Step 2** | Choose: CSV upload **or** manual entry; select attributes; choose unique identifier (email, profile ID, national ID, etc.) | We have CSV only; no manual entry; no "select attributes" (fixed template); identifier is effectively work email. | | **CSV** | Download template; max 10,000 rows; column header match + field value match (e.g. map "year" to compensation time period) | We have Download template; max rows enforced via `BULK_IMPORT_MAX_ROWS` (10,000) from `@/platform/csv` in UI; HR has fixed headers (no column mapping); CE/PF-24 have column mapping. No explicit "match field values" step. | | **Preview** | Review in Rippling spreadsheet; errors highlighted; fix then Continue | We have preview + row-level errors + "rows with errors will be skipped." | | **Confirm** | "Last opportunity to edit"; confirm → optional refresh change history → Done | We have Review step and "Import N Employees" then Done. No "change history" or batch list. | | **Mistake handling** | Delete entire import batch **or** delete individual history records from profile; batches listed under Data import | We have **no import batch tracking**. Cannot delete "last import" or list past imports. Skipped rows are reported in Complete step only. | | **FAQ** | e.g. Time/Timezone required | We don't have in-app FAQ for imports; could add QuickTip or help panel. | **Rippling strengths we could adopt:** 1. **Central Data import area** – One place (e.g. Settings → Data import) that lists all import types and **past batches** (with delete). 2. **Max rows per file** – e.g. 10,000 with clear message so users know limits. 3. **Optional manual entry** – For small data sets, type rows instead of CSV. 4. **Choose unique identifier** – For history/update flows: email vs profile ID vs national ID (we're mostly create-only today). 5. **Batch list + delete batch** – Store import batches and allow "delete this import" to undo. 6. **Clear disclaimer** – "This import only adds history / never updates active data" (when we add history-style imports). *** ## 2. Current Encore Health OS Bulk Import Inventory ### 2.1 Summary Table | Module | Feature | Entry point | Steps | Parser | Insert | Progress | Template | Max rows | | ------------ | ------------------------ | --------------------------- | --------------------------------------------------------------- | ---------------------------------------------- | ----------------------------- | ------------- | --------------- | ----------------------------------------- | | **HR** | Employee roster | Employee Directory (icon) | upload → preview → role-mapping → review → importing → complete | `employeeCsvParser.ts` (uses `@/platform/csv`) | Edge `import-employee-roster` | Indeterminate | Yes (in parser) | No | | **HR** | Credentials | Credentialing area | upload → mapping → validation → import → complete | `credentialCsvTemplate.ts` | Client chunked 50 | Determinate | Yes | No | | **HR** | Oversight sessions | Oversight/compliance | 5-step wizard | Session parser | Client chunked | Yes | Yes | No | | **CE** | Contacts | Contacts page | upload → mapping → validation → importing → complete | `ce/utils/csvImport.ts` | Client batches 50 | Determinate | No | No | | **Platform** | Custom objects (PF-24) | Custom object records table | upload → mapping → validation → importing | `data-manager/csvUtils.ts` | Client CHUNK\_SIZE 50 | Determinate | Via wizard | No | | **Platform** | Raw data (custom fields) | Raw data toolbar | 5-step (preview only) | `rawDataImportUtils.ts` | N/A | N/A | N/A | No | | **Platform** | Bulk doc generation | Templates | CSV upload for generation | Inline parse | N/A (generation) | N/A | sampleCsvUrl | 100 (BulkGenerationDialog; doc-gen limit) | | **FA** | Bank statement | Reconciliation wizard | Single step (file + form) | Inline | Single create | N/A | N/A | N/A | | **FA** | Budget template lines | Budget flow | `templateImport.ts` | useImportTemplate | N/A | N/A | N/A | N/A | ### 2.2 Shared CSV (Already in Place) * **`@/platform/csv`** provides `parseCsvLine`, `parseCsv`, `formatCsvValue`, `BULK_IMPORT_CHUNK_SIZE` (50), and `BULK_IMPORT_MAX_ROWS` (10,000). All bulk import wizards must enforce max rows in the UI after parse. * **HR employee** parser uses `parseCsvLine` from `@/platform/csv/parse`. * Other modules (CE, credential, oversight, data-manager, FA template, BulkGenerationDialog) should use `@/platform/csv` per Constitution §5.10 and BULK\_IMPORT\_MIGRATION\_RECOMMENDATIONS.md; some still have local or duplicate parsing. ### 2.3 Standard Steps (Aligned) We already follow: 1. **Upload** – Dropzone + "Download template" where applicable. 2. **Mapping** – CE and PF-24; HR employee has fixed schema (no mapping). 3. **Preview / Validation** – Row-level errors, stats, "rows with errors skipped." 4. **Import** – Edge (HR employee) or client-side chunked (CE, HR credential, HR oversight, PF-24). 5. **Complete** – Summary (created/skipped/failed); optional "Done" / close. ### 2.4 Gaps vs Rippling / Best Practice | Gap | Priority | Recommendation | | ---------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | No central "Data import" hub | Medium | Add Settings → Data import (or Company Settings → Data import) that lists import types and, when we have batches, past batches. | | No import batch tracking | High | For HR employee (and optionally others), persist an "import batch" record (e.g. `hr_import_batches` or `pf_import_batches`) with summary + link to created IDs; allow "View history" and "Delete batch." | | No max rows per file | Medium | Enforce a limit (e.g. 10,000) in UI/parser and show clear error: "Maximum 10,000 rows per file. Split your file or use multiple imports." | | No "manual entry" option | Low | For small loads, optional step: add rows in a table/form instead of CSV (nice-to-have). | | No "choose unique identifier" | Low | Relevant when we add update/history imports (e.g. map by work email vs profile ID vs national ID). | | HR employee progress indeterminate | Low | Keep as-is unless we refactor edge to report progress or process in chunks. | | CE contact import: no template download | Medium | Add "Download template" for CE contact import (e.g. in `csvImport.ts` or ContactImportDialog). | | Delete batch / delete individual records | High | Depends on batch tracking; then "Delete this import" and, where applicable, "Remove history record" from profile. | *** ## 3. Recommendations and Enhancement Plan ### 3.1 Quick Wins (No Schema Change) * **Document max rows** – In bulk-import-patterns and each wizard: "Recommended max 10,000 rows per file; larger files may be slow or fail." * **Enforce max rows in UI** – After parse, if `rows.length > 10000`, show error and block "Next" until user reduces or splits. * **CE contact template** – Add a "Download template" button and a small CSV template (e.g. first\_name, last\_name, email, phone, contact\_type) using `@/platform/csv` and the same download pattern as HR. * **In-app tips** – Add QuickTip or help snippet on first step of each import: "Each file can have up to 10,000 rows. Use the template for correct column headers." ### 3.2 Medium-Term (Schema + UI) * **Import batch table (e.g. `pf_import_batches` or per-core)** * Columns: `id`, `organization_id`, `import_type` (e.g. `hr_employee`, `ce_contact`), `filename`, `row_count`, `created_count`, `skipped_count`, `failed_count`, `created_at`, `created_by`. * Optional: `metadata` JSONB for skipped/failed details or list of created IDs. * Enables "list past imports" and "delete this batch" (with care: delete batch might mean "mark as rolled back" and then run a rollback flow, or only hide from list and keep data—product decision). * **Settings → Data import** * New section under Platform (or Company) Settings: "Data import" with links to: * Import employee roster (→ HR Employee Directory or open BulkEmployeeImportDialog) * Import contacts (→ CE Contacts or open ContactImportDialog) * Import credentials (→ HR Credentialing) * Import custom object records (→ PF-24, from custom object detail) * Once batches exist: "Recent imports" list with date, type, filename, counts, and "Delete" where supported. ### 3.3 Longer-Term (Rippling-Style Parity) * **History-only import** – If we add "HR history" (promotions, compensation history) that does **not** create employees or update active records, add disclaimer and separate flow; consider "choose unique identifier" in that flow. * **Manual entry** – Optional table/form to add rows without CSV for small datasets. * **Field value mapping** – For flexible CSVs, "match CSV values to system values" (e.g. "Full-Time" → "full\_time") in addition to column mapping; CE and PF-24 are the best candidates. *** ## 4. Implementation Status (Bulk Import Recommendations) | Phase | Description | Status | | ----- | --------------------------------------------------- | ----------- | | 1.1 | Max rows constant (10,000) and documentation | Implemented | | 1.2 | Enforce max rows in every CSV import wizard | Implemented | | 1.3 | CE contact import: Download template | Implemented | | 1.4 | QuickTip on upload step of each import | Implemented | | 2 | CSV parser audit; all use @/platform/csv | Implemented | | 3 | pf\_import\_batches table + record batch on success | Implemented | | 4 | Data import hub (/settings/data-import) | Implemented | | 5 | Module audit (table below) | Implemented | | 6 | Documentation and checklist updates | Implemented | *** ## 5. Module Audit (All Cores) | Module | Existing bulk import | Potential future bulk import | Notes | | ------------ | ------------------------------------------------------------- | ------------------------------------------------------- | ---------------------------------------------------------------- | | **HR** | Employee roster, Credentials, Oversight sessions | — | All use @/platform/csv; max rows enforced; batch recorded. | | **CE** | Contacts | — | Template + max rows + batch recorded. | | **Platform** | Custom objects (PF-24), Raw data preview, Bulk doc generation | — | Custom object import records batch; doc gen keeps 100-row limit. | | **FA** | Bank statement (single), Budget template lines | — | No CSV bulk for general entities. | | **FW** | TemplateImportDialog (JSON/API), TestDatasetImportDialog | — | Not CSV bulk record imports. | | **RH** | None | Resident list, bed/unit import, census | No bulk import today. | | **FM** | None (export-only: asset reports, depreciation) | Asset register, work order list | No bulk import today. | | **GR** | None (export-only: QI reports, etc.) | Policy acknowledgments, training enrollments | No bulk import today. | | **LO** | None | Goals, meetings, scorecards | No bulk import today. | | **IT** | None | Asset list, license list, ticket list | No bulk import today. | | **CL** | None | Patient list, historical data (PHI/compliance critical) | Document in spec when added. | | **PM** | None | Patient list, payer list, claim/scheduling data | Document in spec when added. | *** ## 6. Consistency Checklist (Per Module) When adding or refactoring a bulk import, use this and the existing [bulk-import-migration](./bulk-import-migration.md) checklist: * [ ] Use `@/platform/csv` for all parsing (no local `parseCsvLine`/`parseCsv`). * [ ] Choose edge vs client per [bulk-import-migration](./bulk-import-migration.md) decision tree. * [ ] Steps: Upload → \[Mapping] → Preview/Validation → Import → Complete. * [ ] "Download template" (or sample CSV) when format is fixed or suggested. * [ ] Progress: determinate for client chunked, indeterminate (or future progress) for edge. * [ ] Enforce or document max rows (e.g. 10,000) and show clear error if exceeded. * [ ] Row-level validation errors; `sanitizeErrorMessage` for user-facing errors. * [ ] Invalidate all affected query keys on success. * [ ] Use `BULK_IMPORT_CHUNK_SIZE` (50) for client-side chunked inserts. * [ ] Record import batch in `pf_import_batches` on success (for Data import hub "Recent imports" and "Delete batch" ). *** ## 7. References * [Bulk import migration guide](./bulk-import-migration.md) – Module inventory, decision tree, checklist * BULK\_IMPORT\_MIGRATION\_RECOMMENDATIONS.md – Shared CSV, edge vs client, progress * [.cursor/rules/bulk-import-patterns.md](../../../.cursor/rules/bulk-import-patterns.md) – Shared CSV, batch size, steps, validation * Constitution §5.10 – Bulk Import & Data Migration Standards * AGENTS.md – Bulk Import Pattern, Pre-Flight Checklist