Google Professional Data Engineer Question 86
Select 3Google Cloud PlatformYou are designing a data pipeline in Google Cloud to process customer transaction data. The source data contains missing values, duplicate records, and inconsistent formatting for date fields. Which of the following steps should you include in your pipeline to effectively cleanse the data before analysis?
- A
Perform deduplication to remove duplicate records.
- B
Use Cloud Dataflow to fill missing values with default or calculated values.
- C
Normalize date formats using a consistent standard, such as ISO 8601.
- D
Store the raw data in BigQuery without any transformations to preserve its original state.
- E
Apply a machine learning model to predict and correct missing values.
Show answer and explanation
Correct answers: A, B, C
Explanation
Data cleansing is a critical step in preparing data for analysis. This includes removing duplicate records, handling missing values, and ensuring consistent formatting, as these steps improve data quality and reliability. While advanced techniques like machine learning could be applied in specific use cases, fundamental cleansing steps such as deduplication, filling missing values, and normalizing formats are essential for most pipelines.
- A. Correct.
Deduplication is a key step in data cleansing to ensure data quality and remove redundant records.
- B. Correct.
Cloud Dataflow provides a scalable and efficient way to preprocess data, including handling missing values by filling them with default or calculated values.
- C. Correct.
Normalizing date formats ensures consistency and reduces errors in downstream analysis.
- D. Incorrect.
Storing raw data in BigQuery without transformations does not address the cleansing requirements outlined in the scenario, such as handling missing values or inconsistent formats.
- E. Incorrect.
While machine learning can be used for advanced data cleansing, it is not explicitly required in this scenario nor is it a standard initial step for handling missing values.