Bulk Import Review: PF & Module Inventory vs Rippling-Style Flow

Documentation Index

Fetch the complete documentation index at: https://docs.encoreos.io/llms.txt

Use this file to discover all available pages before exploring further.

​

1. Rippling-Style Reference (What We Like)

1. Central Data import area – One place (e.g. Settings → Data import) that lists all import types and past batches (with delete).
2. Max rows per file – e.g. 10,000 with clear message so users know limits.
3. Optional manual entry – For small data sets, type rows instead of CSV.
4. Choose unique identifier – For history/update flows: email vs profile ID vs national ID (we’re mostly create-only today).
5. Batch list + delete batch – Store import batches and allow “delete this import” to undo.
6. Clear disclaimer – “This import only adds history / never updates active data” (when we add history-style imports).

​

2. Current Encore Health OS Bulk Import Inventory

​

2.1 Summary Table

​

2.2 Shared CSV (Already in Place)

• @/platform/csv provides parseCsvLine, parseCsv, formatCsvValue, BULK_IMPORT_CHUNK_SIZE (50), and BULK_IMPORT_MAX_ROWS (10,000). All bulk import wizards must enforce max rows in the UI after parse.
• HR employee parser uses parseCsvLine from @/platform/csv/parse.
• Other modules (CE, credential, oversight, data-manager, FA template, BulkGenerationDialog) should use @/platform/csv per Constitution §5.10 and BULK_IMPORT_MIGRATION_RECOMMENDATIONS.md; some still have local or duplicate parsing.

​

2.3 Standard Steps (Aligned)

1. Upload – Dropzone + “Download template” where applicable.
2. Mapping – CE and PF-24; HR employee has fixed schema (no mapping).
3. Preview / Validation – Row-level errors, stats, “rows with errors skipped.”
4. Import – Edge (HR employee) or client-side chunked (CE, HR credential, HR oversight, PF-24).
5. Complete – Summary (created/skipped/failed); optional “Done” / close.

​

2.4 Gaps vs Rippling / Best Practice

​

3. Recommendations and Enhancement Plan

​

3.1 Quick Wins (No Schema Change)

• Document max rows – In bulk-import-patterns and each wizard: “Recommended max 10,000 rows per file; larger files may be slow or fail.”
• Enforce max rows in UI – After parse, if rows.length > 10000, show error and block “Next” until user reduces or splits.
• CE contact template – Add a “Download template” button and a small CSV template (e.g. first_name, last_name, email, phone, contact_type) using @/platform/csv and the same download pattern as HR.
• In-app tips – Add QuickTip or help snippet on first step of each import: “Each file can have up to 10,000 rows. Use the template for correct column headers.”

​

3.2 Medium-Term (Schema + UI)

• Import batch table (e.g. pf_import_batches or per-core)
  • Columns: id, organization_id, import_type (e.g. hr_employee, ce_contact), filename, row_count, created_count, skipped_count, failed_count, created_at, created_by.
  • Optional: metadata JSONB for skipped/failed details or list of created IDs.
  • Enables “list past imports” and “delete this batch” (with care: delete batch might mean “mark as rolled back” and then run a rollback flow, or only hide from list and keep data—product decision).
• Settings → Data import
  • New section under Platform (or Company) Settings: “Data import” with links to:
    • Import employee roster (→ HR Employee Directory or open BulkEmployeeImportDialog)
    • Import contacts (→ CE Contacts or open ContactImportDialog)
    • Import credentials (→ HR Credentialing)
    • Import custom object records (→ PF-24, from custom object detail)
  • Once batches exist: “Recent imports” list with date, type, filename, counts, and “Delete” where supported.

​

3.3 Longer-Term (Rippling-Style Parity)

• History-only import – If we add “HR history” (promotions, compensation history) that does not create employees or update active records, add disclaimer and separate flow; consider “choose unique identifier” in that flow.
• Manual entry – Optional table/form to add rows without CSV for small datasets.
• Field value mapping – For flexible CSVs, “match CSV values to system values” (e.g. “Full-Time” → “full_time”) in addition to column mapping; CE and PF-24 are the best candidates.

​

4. Implementation Status (Bulk Import Recommendations)

​

5. Module Audit (All Cores)

​

6. Consistency Checklist (Per Module)

• Use @/platform/csv for all parsing (no local parseCsvLine/parseCsv).
• Choose edge vs client per bulk-import-migration decision tree.
• Steps: Upload → [Mapping] → Preview/Validation → Import → Complete.
• “Download template” (or sample CSV) when format is fixed or suggested.
• Progress: determinate for client chunked, indeterminate (or future progress) for edge.
• Enforce or document max rows (e.g. 10,000) and show clear error if exceeded.
• Row-level validation errors; sanitizeErrorMessage for user-facing errors.
• Invalidate all affected query keys on success.
• Use BULK_IMPORT_CHUNK_SIZE (50) for client-side chunked inserts.
• Record import batch in pf_import_batches on success (for Data import hub “Recent imports” and “Delete batch” ).

​

7. References

• Bulk import migration guide – Module inventory, decision tree, checklist
• BULK_IMPORT_MIGRATION_RECOMMENDATIONS.md – Shared CSV, edge vs client, progress
• .cursor/rules/bulk-import-patterns.md – Shared CSV, batch size, steps, validation
• Constitution §5.10 – Bulk Import & Data Migration Standards
• AGENTS.md – Bulk Import Pattern, Pre-Flight Checklist