Databricks Data Engineer Associate Question 18
Select 3You are designing a Delta Lake-based architecture for a retail company. The company uses the following table layers: bronze tables for raw ingestion, silver tables for cleaned and transformed data, and gold tables for aggregated data used for business intelligence (BI) reporting. Which of the following workloads would most likely use a bronze table as a source, and which would most likely use a gold table as a source?
- A
A machine learning feature engineering pipeline that requires raw transactional data.
- B
An ETL pipeline that enriches and cleanses raw data for further processing.
- C
A real-time dashboard that displays aggregated sales metrics for executive reporting.
- D
A data science model training pipeline that relies on cleaned and pre-processed data.
- E
A financial reporting workload that uses pre-aggregated data for monthly summaries.
Show answer and explanation
Correct answers: A, C, E
Explanation
Workloads that require minimal preprocessing, such as raw data for machine learning feature engineering, typically source data from bronze tables. In contrast, workloads that rely on aggregated or pre-processed data for BI or reporting, like real-time dashboards and financial reporting, are better suited to use gold tables. Understanding the intended use case is critical for determining the appropriate table layer.
- A. Correct.
Correct: Machine learning feature engineering often requires raw data with minimal preprocessing, making the bronze table the appropriate source.
- B. Incorrect.
Incorrect: This workload typically works on raw data to create cleaned and transformed datasets, which aligns with silver tables as the source, not bronze or gold.
- C. Correct.
Correct: Real-time dashboards rely on pre-aggregated metrics from gold tables since they are optimized for BI and reporting purposes.
- D. Incorrect.
Incorrect: Model training pipelines usually leverage cleaned and pre-processed data, making silver tables the more suitable source, not bronze or gold.
- E. Correct.
Correct: Financial reporting workloads, such as monthly summaries, are well-suited for gold tables since they use aggregated and curated data.