Purpose: Module-by-module summary of CSV/bulk import flows, when to use which approach (edge vs client), and a checklist for adding new bulk imports. This guide complements the Bulk Import Recommendations and is referenced by Constitution §5.10 and AGENTS.md. Last Updated: 2026-02-16Documentation Index
Fetch the complete documentation index at: https://docs.encoreos.io/llms.txt
Use this file to discover all available pages before exploring further.
Overview
Bulk import in Encore Health OS means parsing CSV (or similar) and writing records to the database. All such flows must:- Use shared CSV parsing from
@/platform/csv - Enforce max rows per file: use
BULK_IMPORT_MAX_ROWS(10,000) from@/platform/csv; after parse, if row count exceeds it, show error and block Next - Choose edge function vs client-side chunked insert per the decision tree below
- Follow standard wizard steps (Upload → [Mapping] → Preview/Validation → Import → Complete) where applicable
- Show progress for long-running imports and invalidate affected query keys on success
/settings/data-import) lists import types and Recent imports from pf_import_batches; record each successful import via recordImportBatch from @/platform/import-batches. Full module audit: See bulk-import-review.md §Module Audit for a table of existing and potential bulk import by core.
Module Inventory
| Module | Feature | Parser | Insert | Progress | Template |
|---|---|---|---|---|---|
| HR | Employee roster | employeeCsvParser.ts | Edge import-employee-roster | Indeterminate | In parser |
| HR | Credentials | credentialCsvTemplate.ts | Client chunked (50) | Yes | In util |
| HR | Oversight sessions | oversightSessionCsvTemplate.ts | Client chunked | Yes | In util |
| CE | Contacts | csvImport.ts | Client batches 50 | Yes | No |
| Platform | Custom objects (PF-24) | csvUtils.ts | Client CHUNK_SIZE 50 | Yes | Via wizard |
| Platform | Raw data import | rawDataImportUtils.ts | N/A (preview) | N/A | N/A |
| Platform | Bulk doc generation | Inline in BulkGenerationDialog | N/A (generation) | N/A | sampleCsvUrl |
| FA | Bank statement | Inline in ImportStatementStep | Single create | N/A | N/A |
| FA | Budget template lines | templateImport.ts | useImportTemplate | N/A | N/A |
| FW | Workflow/template import | TemplateImportDialog | JSON/import API | N/A | N/A |
| FW | Test dataset import | TestDatasetImportDialog | Test data API | N/A | N/A |
@/platform/csv (see BULK_IMPORT_MIGRATION_RECOMMENDATIONS.md for migration order).
Decision Tree: Edge Function vs Client-Side Chunked Insert
| Question | Answer | Use |
|---|---|---|
| Does the import create multiple entity types (e.g. sites, departments, profiles, employees)? | Yes | Edge function |
| Does it create auth accounts or send invites? | Yes | Edge function |
| Is transaction ordering or server-side sequencing critical? | Yes | Edge function |
| Is it simple insert-only to one or a few tables? | Yes | Client-side chunked (batch size 50, determinate progress) |
- HR employee roster (sites, departments, profiles, employees, optional accounts/invites) → Edge function
- CE contacts, HR credentials, HR oversight sessions, custom object records → Client-side chunked
Standard Steps and Progress
Wizard steps (where applicable):- Upload — Dropzone; optional “Download template”
- Mapping — Only if column mapping is needed (e.g. CE, PF-24)
- Preview / Validation — Row-level errors, duplicates, stats
- Import — Call edge function or run client-side chunked insert
- Complete — Summary (created / skipped / failed); optional “View list” / “Import again”
- Edge function: Indeterminate progress is acceptable (e.g. “Importing employees…”).
- Client-side chunked: Show determinate progress (e.g. percentage per batch). Use batch size 50.
Adding a New Bulk Import (Checklist)
- Use shared CSV parsing from
@/platform/csv(no localparseCsvLine/parseCsv). - Choose edge function vs client-side per the decision tree above.
- Follow standard steps: Upload → [Mapping] → Preview/Validation → Import → Complete.
- Provide “Download template” or sample CSV when the format is fixed or suggested.
- Show progress for imports that can take more than a few seconds (determinate for chunked).
- Return row-level validation errors; use
sanitizeErrorMessagefor user-facing errors. - Invalidate all affected query keys on success.
- Use batch size 50 for client-side chunked inserts (or document override).
- Enforce or document max rows: use
BULK_IMPORT_MAX_ROWSfrom@/platform/csv(10,000); show clear error if exceeded and block Next. - Record import batch in
pf_import_batcheson success (viarecordImportBatchfrom@/platform/import-batches) so it appears in Settings → Data import → Recent imports. - If the feature touches DB or multiple entities, create a spec per SPEC_TEMPLATE and reference this guide.
References
- Constitution: §5.10 Bulk Import & Data Migration Standards
- AGENTS.md: Bulk Import Pattern
- Pattern file: .cursor/rules/bulk-import-patterns.md
- Recommendations: BULK_IMPORT_MIGRATION_RECOMMENDATIONS.md
- Review (Rippling-style comparison): bulk-import-review.md – PF & module inventory vs Rippling-style flow, gaps, and enhancement plan