Databricks Data Engineer Professional Question 229
Select 2You are troubleshooting the performance of a Spark job running on a Databricks cluster. The job seems to be taking longer than expected. To inspect the event timelines and metrics for stages and jobs to identify bottlenecks, which steps should you take?
- A
Navigate to the Spark UI from the Databricks cluster details page and review the 'Stages' tab.
- B
Use the Databricks Jobs UI to directly view the timeline of stages and tasks for the job.
- C
Inspect the 'SQL' tab in the Spark UI to analyze query execution details for the job.
- D
Download the event logs from the cluster and use a third-party tool to analyze the job's execution timeline.
- E
Enable 'Ganglia Metrics' for the cluster to visualize stage execution timelines.
Show answer and explanation
Correct answers: A, C
Explanation
To inspect event timelines and metrics for stages and jobs, the Spark UI's 'Stages' tab and 'SQL' tab are the most appropriate tools. These provide granular details about stage execution and query performance, making them essential for identifying bottlenecks. While the Jobs UI and Ganglia Metrics offer useful information, they do not provide the required level of detail for stage-level troubleshooting.
- A. Correct.
Correct: The 'Stages' tab in the Spark UI provides detailed metrics and timelines for individual stages, allowing you to identify which stages are causing delays.
- B. Incorrect.
Incorrect: The Databricks Jobs UI summarizes job statuses but does not provide detailed timelines or metrics for individual stages and tasks.
- C. Correct.
Correct: The 'SQL' tab in the Spark UI provides insights into query execution plans and metrics, which are useful for analyzing performance bottlenecks in SQL-based workloads.
- D. Incorrect.
Incorrect: While downloading event logs and using third-party tools is an option, it is not the standard or recommended way to inspect event timelines in Databricks.
- E. Incorrect.
Incorrect: 'Ganglia Metrics' is used for cluster-level monitoring but does not provide detailed insights into Spark stage execution timelines.