Zero-Downtime Migration of 50 Million Customer Records

The Client

A major UK financial services provider with over 25 years of customer data held across multiple legacy mainframe and relational database systems. The organisation serves hundreds of thousands of retail and business customers across the UK. Client identity withheld under NDA.

The Challenge

The client's legacy data infrastructure had accumulated significant technical debt over two and a half decades. Their systems comprised multiple database technologies, inconsistent schemas, and data quality issues that had never been systematically resolved. The board had approved a cloud migration programme, but the data layer presented the highest risk.

• 50 million customer records spread across seven legacy systems with different schemas
• Zero tolerance for data loss or service interruption under FCA operational resilience requirements
• Strict PCI DSS and UK GDPR compliance requirements governing how data could be handled during migration
• Complex relational dependencies between customer, account, transaction, and compliance records
• Significant data quality issues: duplicate records, inconsistent date formats, and legacy character encoding
• A fixed regulatory deadline that could not be moved

Our Solution

UK Data Services designed a phased, parallel-run migration strategy that allowed the new cloud platform to operate alongside legacy systems during the transition, with automated reconciliation to ensure data integrity at every stage.

• Data audit and profiling: Comprehensive analysis of all seven source systems to map relationships, identify anomalies, and quantify data quality issues before a single record was moved
• Cleanse and standardise pipeline: Automated transformation layer to resolve duplicates, standardise formats, and apply consistent business rules before loading into the target system
• Parallel run architecture: Both legacy and new systems operated in parallel for 8 weeks, with automated reconciliation jobs running every 30 minutes to detect any discrepancy
• Incremental cutover: Customer segments migrated in tranches by risk level, with rollback capability maintained throughout
• Audit trail and compliance documentation: Full lineage tracking for every record, supporting FCA reporting requirements and GDPR Article 30 records of processing

Implementation Timeline

• Months 1–2: Data audit, schema mapping, and cleansing rules definition
• Months 3–4: Pipeline development, test environment validation, and reconciliation framework build
• Month 5: Parallel run initiation and first customer segment cutover
• Months 6–7: Phased cutover of remaining segments with continuous reconciliation
• Month 8: Legacy system decommission, final audit sign-off

The project completed six weeks ahead of the original schedule, which the client attributed primarily to the quality of data profiling completed in months one and two reducing the volume of issues discovered mid-migration.

Results

The migration was completed with zero customer-facing disruption. The automated reconciliation framework caught and resolved 847 data discrepancies before they reached the production system — none required manual intervention from the client's team.

• 50 million records migrated with 99.99% verified accuracy
• Zero minutes of unplanned service downtime throughout the 8-week parallel run
• Project completed 6 weeks ahead of schedule
• £2M under the original budget estimate, primarily through efficient automation of cleansing tasks originally scoped for manual review
• Full FCA operational resilience and GDPR Article 30 documentation delivered as part of the project