> ## Documentation Index
> Fetch the complete documentation index at: https://docs.encoreos.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Bulk Import & Migration Use Case Guide

> Purpose: Module-by-module summary of CSV/bulk import flows, when to use which approach (edge vs client), and a checklist for adding new bulk imports. This guid…

**Purpose:** Module-by-module summary of CSV/bulk import flows, when to use which approach (edge vs client), and a checklist for adding new bulk imports. This guide complements the Bulk Import Recommendations and is referenced by Constitution §5.10 and AGENTS.md.

**Last Updated:** 2026-02-16

***

## Overview

Bulk import in Encore Health OS means parsing CSV (or similar) and writing records to the database. All such flows must:

* Use shared CSV parsing from `@/platform/csv`
* Enforce max rows per file: use `BULK_IMPORT_MAX_ROWS` (10,000) from `@/platform/csv`; after parse, if row count exceeds it, show error and block Next
* Choose edge function vs client-side chunked insert per the decision tree below
* Follow standard wizard steps (Upload → \[Mapping] → Preview/Validation → Import → Complete) where applicable
* Show progress for long-running imports and invalidate affected query keys on success

**References:** [Constitution §5.10](../../../constitution.md) (Bulk Import & Data Migration Standards), [AGENTS.md](../../../AGENTS.md) (Bulk Import Pattern), [.cursor/rules/bulk-import-patterns.md](../../../.cursor/rules/bulk-import-patterns.md), BULK\_IMPORT\_MIGRATION\_RECOMMENDATIONS.md. **Data import hub:** Settings → Data import (route: `/settings/data-import`) lists import types and **Recent imports** from `pf_import_batches`; record each successful import via `recordImportBatch` from `@/platform/import-batches`. **Full module audit:** See [bulk-import-review.md](./bulk-import-review.md) §Module Audit for a table of existing and potential bulk import by core.

***

## Module Inventory

| Module       | Feature                  | Parser                           | Insert                        | Progress      | Template     |
| ------------ | ------------------------ | -------------------------------- | ----------------------------- | ------------- | ------------ |
| **HR**       | Employee roster          | `employeeCsvParser.ts`           | Edge `import-employee-roster` | Indeterminate | In parser    |
| **HR**       | Credentials              | `credentialCsvTemplate.ts`       | Client chunked (50)           | Yes           | In util      |
| **HR**       | Oversight sessions       | `oversightSessionCsvTemplate.ts` | Client chunked                | Yes           | In util      |
| **CE**       | Contacts                 | `csvImport.ts`                   | Client batches 50             | Yes           | No           |
| **Platform** | Custom objects (PF-24)   | `csvUtils.ts`                    | Client CHUNK\_SIZE 50         | Yes           | Via wizard   |
| **Platform** | Raw data import          | `rawDataImportUtils.ts`          | N/A (preview)                 | N/A           | N/A          |
| **Platform** | Bulk doc generation      | Inline in BulkGenerationDialog   | N/A (generation)              | N/A           | sampleCsvUrl |
| **FA**       | Bank statement           | Inline in ImportStatementStep    | Single create                 | N/A           | N/A          |
| **FA**       | Budget template lines    | `templateImport.ts`              | useImportTemplate             | N/A           | N/A          |
| **FW**       | Workflow/template import | TemplateImportDialog             | JSON/import API               | N/A           | N/A          |
| **FW**       | Test dataset import      | TestDatasetImportDialog          | Test data API                 | N/A           | N/A          |

All CSV parsing in these modules should use `@/platform/csv` (see BULK\_IMPORT\_MIGRATION\_RECOMMENDATIONS.md for migration order).

***

## Decision Tree: Edge Function vs Client-Side Chunked Insert

| Question                                                                                     | Answer | Use                                                           |
| -------------------------------------------------------------------------------------------- | ------ | ------------------------------------------------------------- |
| Does the import create multiple entity types (e.g. sites, departments, profiles, employees)? | Yes    | **Edge function**                                             |
| Does it create auth accounts or send invites?                                                | Yes    | **Edge function**                                             |
| Is transaction ordering or server-side sequencing critical?                                  | Yes    | **Edge function**                                             |
| Is it simple insert-only to one or a few tables?                                             | Yes    | **Client-side chunked** (batch size 50, determinate progress) |

**Examples:**

* HR employee roster (sites, departments, profiles, employees, optional accounts/invites) → Edge function
* CE contacts, HR credentials, HR oversight sessions, custom object records → Client-side chunked

***

## Standard Steps and Progress

**Wizard steps (where applicable):**

1. **Upload** — Dropzone; optional "Download template"
2. **Mapping** — Only if column mapping is needed (e.g. CE, PF-24)
3. **Preview / Validation** — Row-level errors, duplicates, stats
4. **Import** — Call edge function or run client-side chunked insert
5. **Complete** — Summary (created / skipped / failed); optional "View list" / "Import again"

**Progress:**

* **Edge function:** Indeterminate progress is acceptable (e.g. "Importing employees...").
* **Client-side chunked:** Show determinate progress (e.g. percentage per batch). Use batch size 50.

***

## Adding a New Bulk Import (Checklist)

* [ ] Use shared CSV parsing from `@/platform/csv` (no local `parseCsvLine`/`parseCsv`).
* [ ] Choose edge function vs client-side per the decision tree above.
* [ ] Follow standard steps: Upload → \[Mapping] → Preview/Validation → Import → Complete.
* [ ] Provide "Download template" or sample CSV when the format is fixed or suggested.
* [ ] Show progress for imports that can take more than a few seconds (determinate for chunked).
* [ ] Return row-level validation errors; use `sanitizeErrorMessage` for user-facing errors.
* [ ] Invalidate all affected query keys on success.
* [ ] Use batch size 50 for client-side chunked inserts (or document override).
* [ ] Enforce or document max rows: use `BULK_IMPORT_MAX_ROWS` from `@/platform/csv` (10,000); show clear error if exceeded and block Next.
* [ ] Record import batch in `pf_import_batches` on success (via `recordImportBatch` from `@/platform/import-batches`) so it appears in Settings → Data import → Recent imports.
* [ ] If the feature touches DB or multiple entities, create a spec per SPEC\_TEMPLATE and reference this guide.

***

## References

* **Constitution:** [§5.10 Bulk Import & Data Migration Standards](../../../constitution.md)
* **AGENTS.md:** [Bulk Import Pattern](../../../AGENTS.md)
* **Pattern file:** [.cursor/rules/bulk-import-patterns.md](../../../.cursor/rules/bulk-import-patterns.md)
* **Recommendations:** BULK\_IMPORT\_MIGRATION\_RECOMMENDATIONS.md
* **Review (Rippling-style comparison):** [bulk-import-review.md](./bulk-import-review.md) – PF & module inventory vs Rippling-style flow, gaps, and enhancement plan