Databricks Data Engineer Associate Question 19
Select 4A data engineering team is building a data pipeline in Databricks to process customer transactions. The pipeline ingests raw transactional data into a bronze table, processes it into a silver table for filtering and cleaning, and finally produces a gold table for business reporting and analytics. Which of the following statements correctly explains which workloads use bronze and gold tables as data sources?
- A
Data cleansing and enrichment workloads typically use bronze tables as a source.
- B
Machine learning model training typically uses gold tables as a source.
- C
Data aggregation and reporting workloads typically use gold tables as a source.
- D
Real-time streaming ingestion processes typically use gold tables as a source.
- E
Exploratory data analysis (EDA) typically uses silver or bronze tables as a source.
Show answer and explanation
Correct answers: A, B, C, E
Explanation
Bronze tables store raw ingested data and are used as a source for data cleansing, enrichment, and exploratory analysis workloads. Gold tables store highly refined, aggregated, and curated data, which makes them suitable sources for machine learning, reporting, and analytics workloads. Real-time streaming processes typically write to bronze tables rather than using gold tables as a source.
- A. Correct.
Correct. Data cleansing and enrichment workloads usually start with raw data from bronze tables to apply transformations, remove duplicates, and prepare the data for downstream use.
- B. Correct.
Correct. Machine learning model training often requires high-quality, aggregated, or curated data, which is typically stored in gold tables.
- C. Correct.
Correct. Gold tables are optimized for business intelligence, reporting, and analytics, making them the primary source for such workloads.
- D. Incorrect.
Incorrect. Real-time streaming ingestion processes write data into bronze tables but do not use gold tables as a source. Gold tables are typically used for downstream analytics and reporting, not for ingestion processes.
- E. Correct.
Correct. Exploratory data analysis (EDA) can be performed on silver or bronze tables to understand the data structure and patterns before it is further processed or refined.