Databricks Data Engineer Associate Question 367
Select 3You are tasked with creating a new Databricks Delta Live Tables (DLT) pipeline for your organization's data engineering project. Which of the following components are required to successfully create the pipeline?
- A
A JSON or YAML configuration file defining the pipeline settings
- B
A notebook or script containing the DLT-specific transformation logic
- C
A cluster pre-configured with the desired libraries for the pipeline
- D
A storage location for the pipeline's output data
- E
A Databricks SQL endpoint for pipeline execution
Show answer and explanation
Correct answers: A, B, D
Explanation
To create a new DLT pipeline, you need a JSON or YAML configuration file to define pipeline settings, a notebook or script with transformation logic, and a storage location for the output data. Clusters and SQL endpoints are managed or used separately and are not prerequisites for DLT pipeline creation.
- A. Correct.
Correct. The pipeline settings, such as input/output locations, are defined in a JSON or YAML configuration file during pipeline creation.
- B. Correct.
Correct. A notebook or script containing the transformation logic is necessary to define the data transformations in the DLT pipeline.
- C. Incorrect.
Incorrect. Databricks DLT pipelines automatically handle cluster creation and management, so a pre-configured cluster is not required.
- D. Correct.
Correct. A storage location must be provided for the pipeline to store its output data, such as processed tables.
- E. Incorrect.
Incorrect. While Databricks SQL endpoints are useful for querying data, they are not required to create or execute a DLT pipeline.