Databricks Data Engineer Associate Question 252
Single answerYou are working on a Databricks project where a team of data scientists needs frequent access to a shared dataset stored in an external cloud storage system (e.g., AWS S3 or Azure Data Lake). The dataset is updated daily by another team, and the data scientists need to query the latest data without moving it to the Databricks workspace. Which Databricks table type should you use in this scenario?
- A
External table
- B
Managed table
- C
Temporary view
- D
Delta table
Show answer and explanation
Correct answer: A
Explanation
An external table is the best choice when you need to query data directly from an external storage location, ensuring that the data remains up-to-date without duplicating it into the Databricks workspace. This fits the described scenario where external updates occur frequently, and the team requires seamless access to the latest data.
- A. Correct.
Correct. An external table allows you to query data directly from an external storage location without importing or copying it into Databricks. This is ideal for scenarios where the data is updated externally and needs to be accessed in its latest state.
- B. Incorrect.
Incorrect. A managed table stores data internally within Databricks, which is unnecessary in this scenario since the dataset is stored externally and updated by another team.
- C. Incorrect.
Incorrect. A temporary view does not persist beyond the session and cannot be used for external data that needs consistent, long-term access.
- D. Incorrect.
Incorrect. A Delta table is a format for managing data with ACID properties but does not specifically address the need to directly query external data without copying it into Databricks.