Google Professional Data Engineer Question 58
Select 3Google Cloud PlatformYour company is migrating a large-scale data warehouse from an on-premises system to BigQuery. You are tasked with designing a migration validation strategy to ensure the data's accuracy, consistency, and completeness after migration. Which of the following are key components you should include in your validation strategy?
- A
Perform row-by-row data comparison between the source and destination systems.
- B
Validate schema consistency between the source and destination systems.
- C
Conduct sample-based data validation using a subset of the migrated data.
- D
Implement automated query result comparison for business-critical reports.
- E
Ignore validation of metadata as it does not impact the data migration process.
Show answer and explanation
Correct answers: B, C, D
Explanation
An effective migration validation strategy for large-scale migrations like BigQuery should focus on scalable and practical validation methods. Schema consistency ensures structural alignment, sample-based validation offers a resource-efficient way to check data accuracy, and automated query result comparisons validate real-world usability. Row-by-row comparison is too resource-intensive, and metadata validation cannot be ignored as it impacts the manageability of the data.
- A. Incorrect.
Performing row-by-row data comparison is highly resource-intensive and impractical for large-scale migrations like this. While it might work for small datasets, it is not a scalable approach for BigQuery migrations.
- B. Correct.
Validating schema consistency is essential to ensure that the structure of the data (tables, columns, data types) in the destination system aligns with the source system, which is critical for accurate data processing post-migration.
- C. Correct.
Conducting sample-based data validation is a practical approach to verifying data accuracy and consistency without incurring the high overhead of validating every single row in a large dataset.
- D. Correct.
Automated query result comparison ensures that key business-critical reports yield the same results after migration, validating the usability of the data in real-world scenarios.
- E. Incorrect.
Ignoring metadata validation is incorrect because metadata (such as table properties, column descriptions, and partitioning details) plays a critical role in ensuring the usability and manageability of the migrated data.