Databricks Machine Learning Professional Question 100
Select 3A data engineering team is running nightly ETL pipelines in Databricks to process large datasets and generate reports. They are considering switching from all-purpose clusters to job clusters for their workloads. What are the advantages of using job clusters in this scenario?
- A
Job clusters automatically terminate upon job completion, reducing costs.
- B
Job clusters allow for easy sharing of resources across multiple users.
- C
Each job cluster is created with an isolated environment, ensuring no resource contention with other workloads.
- D
Job clusters support custom configurations tailored for specific jobs.
- E
Job clusters provide direct access to interactive notebooks for experimentation.
Show answer and explanation
Correct answers: A, C, D
Explanation
Job clusters are optimized for ephemeral workloads like scheduled ETL jobs. They are cost-efficient due to automatic termination, ensure resource isolation, and allow job-specific configurations. However, they lack features like interactive notebook access or multi-user resource sharing, which are typical of all-purpose clusters.
- A. Correct.
Job clusters are designed to automatically terminate once the job is complete, making them cost-efficient for ephemeral workloads like ETL pipelines.
- B. Incorrect.
Resource sharing across multiple users is not a feature of job clusters. It is a feature more associated with all-purpose clusters.
- C. Correct.
Job clusters are created per job, ensuring isolation and eliminating resource contention, which is beneficial for dedicated workloads.
- D. Correct.
Job clusters allow specifying configurations (e.g., cluster size, libraries) tailored to the needs of a specific job, enhancing performance and efficiency.
- E. Incorrect.
Job clusters do not provide direct access to interactive notebooks, as they are designed for non-interactive, scheduled processing tasks.