Data Migration Planning in Australia: Mapping, Cleansing, Testing and Cutover

Plan a data migration as its own project: profile the source data, map every field, clean and deduplicate, rehearse the migration at least twice on recent copies, reconcile counts, totals and samples, plan the cutover with a rehearsed rollback, and validate with business users before switching off the old system. In Australia, also check record retention, destruction or de-identification of personal information no longer needed, and overseas access by migration teams. Nobody should promise zero downtime or zero data loss.

This guide is for businesses moving data between business systems: CRM, ERP, accounting, practice management, job management or custom applications. For online stores, see ecommerce data migration; for the wider system decision, legacy software modernisation.

It includes a practical checklist at the end. Nothing here is legal or tax advice.

List every source: the main system, spreadsheets, shared drives, other tools that hold related records. For each, profile the data: volumes, field fill rates, unique values, odd formats, duplicates, orphaned records and free-text fields used as codes. Profiling turns assumptions into facts and usually changes the plan.

Fix what you can in the source before migration, so the new system starts clean. Standardise formats (phone numbers, addresses, ABNs, dates), merge duplicates with business review, close or archive inactive records, and fill required fields. Automated rules handle the obvious cases; people should decide the ambiguous ones. Track cleansing progress by measure, such as the duplicate rate, rather than by effort. See customer data integration for matching techniques.

Script the migration so it is repeatable: extract, transform, load and verify. Run it end to end on a recent copy of production data in a test environment, timed, following the runbook. Reconcile, fix problems, and run it again. Each rehearsal should be more predictable than the last; the final rehearsal tells you how long cutover will take.

Agree acceptable downtime with the business and choose an approach: a freeze-and-migrate cutover during a quiet period, or a phased cutover with change replication so the final switch only moves recent changes. Communicate timings to staff and customers.

Write the rollback plan before cutover: what triggers it, who decides, how long rollback remains possible, and what happens to data entered in the new system in the meantime. Back up the source fully before starting and keep it read-only and available until the migration is accepted. A rollback plan that has not been rehearsed is an assumption. For high-risk systems, consider a parallel run; see parallel runs.

Migrations create extra copies of data in extracts, staging databases and spreadsheets, often with broader access than production. Limit who can access migration data, use masked or synthetic data where real data is not needed, encrypt extracts, log access, and delete temporary copies when the migration is accepted.

Retention. Business guidance generally says most business records must be kept for five years, and electronic records are acceptable when they are true and complete reproductions (business.gov.au). Other laws may require longer retention for some records. Check before deciding what not to migrate or archive.

Destruction or de-identification. If you are covered by the Privacy Act, APP 11 requires reasonable steps to destroy or de-identify personal information you no longer need, unless a law requires you to keep it (OAIC). A migration is a natural point to apply this, rather than moving everything forward forever.

Overseas access. If migration staff or tools outside Australia access personal information, APP 8 applies (OAIC). Use masked data where possible and put protections in contracts.

Breaches. Lost extracts and misconfigured staging environments are common breach causes. If an eligible data breach occurs, the Notifiable Data Breaches scheme applies. See website security for Australian small businesses for the incident sequence.

Watch closely for the first weeks: data errors reported by users, integration failures and performance. Fix issues at source and record lessons. When the business has signed off, decommission the old system according to plan, keeping archives accessible for the retention period and deleting temporary migration copies.

This example is hypothetical. An accounting practice is moving from a self-hosted practice system used for fifteen years to a cloud product. Profiling finds thousands of duplicate contacts, job codes stored in free text, and documents linked by file paths. The practice migrates seven years of jobs and invoices, archives older records in a searchable read-only format after its accountant confirms retention needs, de-identifies data for clients who left long ago where it is not required to keep it, and runs three rehearsals before a weekend cutover. Reconciliation compares work-in-progress and debtor balances to the cent.

For cloud moves, see cloud migration for Australian businesses. For CRM changes, see custom vs off-the-shelf CRM. For stores, see ecommerce migration testing.

Moving between database engines, for example from SQL Server or Oracle to PostgreSQL, or from a self-managed database to a managed cloud service, is more than copying tables. Data types, functions, stored procedures, collations, time zones and performance behaviour differ, and applications often rely on engine-specific features.

business.gov.au, record keeping; OAIC, APP 11; OAIC, APP 8.

The checklist, tables and example are ours. Nothing here is legal or tax advice or ZSpace client data.

Data migrations succeed through preparation and proof: profile, map, clean, rehearse, reconcile, and only then cut over with a way back. In Australia, use the migration to apply retention and privacy rules properly, and control access to the extra copies migrations create. Promise careful verification, not perfection.