Databricks Data Engineer Associate Question 248
Single answerA data engineering team is tasked with analyzing a large volume of log files stored in an Azure Data Lake Storage (ADLS) account. The logs are updated daily, and the team needs to run ad-hoc analytical queries without moving the data into the Databricks workspace. Which of the following scenarios best justifies using an external table?
- A
The team wants to create a Delta table with ACID transactions for the logs.
- B
The team wants to query the logs directly without duplicating the data into Databricks.
- C
The team wants to enforce schema evolution and manage the data lifecycle.
- D
The team wants to ensure the logs are versioned and support time travel queries.
Show answer and explanation
Correct answer: B
Explanation
External tables are suitable when you need to query data stored in external storage without moving or duplicating it into Databricks. In this scenario, the team needs to analyze logs stored in ADLS while minimizing data duplication, making an external table the best choice.
- A. Incorrect.
This is incorrect because Delta tables are typically used when you need to enable ACID transactions, which requires moving the data into the Databricks workspace.
- B. Correct.
This is correct because an external table allows querying data stored in external storage (e.g., ADLS) directly without duplicating or moving it into Databricks.
- C. Incorrect.
This is incorrect because schema evolution and lifecycle management are features commonly associated with Delta Lake, not external tables.
- D. Incorrect.
This is incorrect because versioning and time travel are features of Delta Lake, not features of external tables.