Skip to main content
Purpose: Compare Encore OS’s bulk import setup across PF and modules to a Rippling-style “HR history import” flow, and recommend improvements and enhancements. References: bulk-import-migration.md, BULK_IMPORT_MIGRATION_RECOMMENDATIONS.md, .cursor/rules/bulk-import-patterns.md. Last Updated: 2026-02-16

1. Rippling-Style Reference (What We Like)

From the Rippling “Import HR history” flow: Rippling strengths we could adopt:
  1. Central Data import area – One place (e.g. Settings → Data import) that lists all import types and past batches (with delete).
  2. Max rows per file – e.g. 10,000 with clear message so users know limits.
  3. Optional manual entry – For small data sets, type rows instead of CSV.
  4. Choose unique identifier – For history/update flows: email vs profile ID vs national ID (we’re mostly create-only today).
  5. Batch list + delete batch – Store import batches and allow “delete this import” to undo.
  6. Clear disclaimer – “This import only adds history / never updates active data” (when we add history-style imports).

2. Current Encore OS Bulk Import Inventory

2.1 Summary Table

2.2 Shared CSV (Already in Place)

  • @/platform/csv provides parseCsvLine, parseCsv, formatCsvValue, BULK_IMPORT_CHUNK_SIZE (50), and BULK_IMPORT_MAX_ROWS (10,000). All bulk import wizards must enforce max rows in the UI after parse.
  • HR employee parser uses parseCsvLine from @/platform/csv/parse.
  • Other modules (CE, credential, oversight, data-manager, FA template, BulkGenerationDialog) should use @/platform/csv per Constitution §5.10 and BULK_IMPORT_MIGRATION_RECOMMENDATIONS.md; some still have local or duplicate parsing.

2.3 Standard Steps (Aligned)

We already follow:
  1. Upload – Dropzone + “Download template” where applicable.
  2. Mapping – CE and PF-24; HR employee has fixed schema (no mapping).
  3. Preview / Validation – Row-level errors, stats, “rows with errors skipped.”
  4. Import – Edge (HR employee) or client-side chunked (CE, HR credential, HR oversight, PF-24).
  5. Complete – Summary (created/skipped/failed); optional “Done” / close.

2.4 Gaps vs Rippling / Best Practice


3. Recommendations and Enhancement Plan

3.1 Quick Wins (No Schema Change)

  • Document max rows – In bulk-import-patterns and each wizard: “Recommended max 10,000 rows per file; larger files may be slow or fail.”
  • Enforce max rows in UI – After parse, if rows.length > 10000, show error and block “Next” until user reduces or splits.
  • CE contact template – Add a “Download template” button and a small CSV template (e.g. first_name, last_name, email, phone, contact_type) using @/platform/csv and the same download pattern as HR.
  • In-app tips – Add QuickTip or help snippet on first step of each import: “Each file can have up to 10,000 rows. Use the template for correct column headers.”

3.2 Medium-Term (Schema + UI)

  • Import batch table (e.g. pf_import_batches or per-core)
    • Columns: id, organization_id, import_type (e.g. hr_employee, ce_contact), filename, row_count, created_count, skipped_count, failed_count, created_at, created_by.
    • Optional: metadata JSONB for skipped/failed details or list of created IDs.
    • Enables “list past imports” and “delete this batch” (with care: delete batch might mean “mark as rolled back” and then run a rollback flow, or only hide from list and keep data—product decision).
  • Settings → Data import
    • New section under Platform (or Company) Settings: “Data import” with links to:
      • Import employee roster (→ HR Employee Directory or open BulkEmployeeImportDialog)
      • Import contacts (→ CE Contacts or open ContactImportDialog)
      • Import credentials (→ HR Credentialing)
      • Import custom object records (→ PF-24, from custom object detail)
    • Once batches exist: “Recent imports” list with date, type, filename, counts, and “Delete” where supported.

3.3 Longer-Term (Rippling-Style Parity)

  • History-only import – If we add “HR history” (promotions, compensation history) that does not create employees or update active records, add disclaimer and separate flow; consider “choose unique identifier” in that flow.
  • Manual entry – Optional table/form to add rows without CSV for small datasets.
  • Field value mapping – For flexible CSVs, “match CSV values to system values” (e.g. “Full-Time” → “full_time”) in addition to column mapping; CE and PF-24 are the best candidates.

4. Implementation Status (Bulk Import Recommendations)


5. Module Audit (All Cores)


6. Consistency Checklist (Per Module)

When adding or refactoring a bulk import, use this and the existing bulk-import-migration checklist:
  • Use @/platform/csv for all parsing (no local parseCsvLine/parseCsv).
  • Choose edge vs client per bulk-import-migration decision tree.
  • Steps: Upload → [Mapping] → Preview/Validation → Import → Complete.
  • “Download template” (or sample CSV) when format is fixed or suggested.
  • Progress: determinate for client chunked, indeterminate (or future progress) for edge.
  • Enforce or document max rows (e.g. 10,000) and show clear error if exceeded.
  • Row-level validation errors; sanitizeErrorMessage for user-facing errors.
  • Invalidate all affected query keys on success.
  • Use BULK_IMPORT_CHUNK_SIZE (50) for client-side chunked inserts.
  • Record import batch in pf_import_batches on success (for Data import hub “Recent imports” and “Delete batch” ).

7. References

  • Bulk import migration guide – Module inventory, decision tree, checklist
  • BULK_IMPORT_MIGRATION_RECOMMENDATIONS.md – Shared CSV, edge vs client, progress
  • .cursor/rules/bulk-import-patterns.md – Shared CSV, batch size, steps, validation
  • Constitution §5.10 – Bulk Import & Data Migration Standards
  • AGENTS.md – Bulk Import Pattern, Pre-Flight Checklist