Google Professional Data Engineer Question 60
Select 3Google Cloud PlatformYour company is migrating its on-premises data warehouse to BigQuery. As a data engineer, you need to design the migration validation strategy to ensure data accuracy and consistency. Which actions should you include in your validation process?
- A
Perform row count and checksum comparisons between the source and target systems.
- B
Use BigQuery’s built-in machine learning capabilities to predict data discrepancies during migration.
- C
Validate schema compatibility and ensure that data types are mapped correctly between source and target.
- D
Run queries on both the source and target systems to compare aggregated metrics, such as sum or average of numerical columns.
- E
Rely exclusively on the success status of the migration job as an indicator of successful validation.
Show answer and explanation
Correct answers: A, C, D
Explanation
Designing a migration validation strategy involves multiple steps to ensure data accuracy, consistency, and integrity. Key actions include comparing row counts and checksums, validating schema compatibility, and running aggregated metric comparisons. These steps collectively provide a comprehensive approach to identifying discrepancies and ensuring data fidelity during the migration process. Relying solely on the migration job status or using unrelated tools like machine learning for validation is not sufficient or appropriate.
- A. Correct.
Performing row count and checksum comparisons is a standard approach to ensure that the number of records and their contents match between the source and the target systems. This is a critical step in data migration validation.
- B. Incorrect.
BigQuery’s machine learning capabilities are not intended for data validation during migration. This is not a relevant or appropriate action for the validation process.
- C. Correct.
Validating schema compatibility and ensuring correct data type mapping is essential to avoid data loss, truncation, or type mismatches during migration.
- D. Correct.
Running queries to compare aggregated metrics helps ensure that key data characteristics are preserved in the migration and can highlight discrepancies that might not be evident through row counts alone.
- E. Incorrect.
Relying solely on the success status of the migration job is insufficient, as it does not ensure data accuracy or completeness. Additional validation steps are necessary to verify the integrity of the migrated data.