Databricks Data Engineer Professional Question 280
Select 3You are tasked with creating a Databricks Job to process daily transactional data from a Delta table and store aggregated results in another Delta table for reporting purposes. The job should be scheduled to run daily at a specific time. Which of the following steps are required to configure this Job correctly?
- A
Define a new Job and specify a notebook or JAR task that performs the data processing logic.
- B
Configure a JDBC connection to the Delta table in the Job settings.
- C
Set a daily schedule with the desired time for the Job to run.
- D
Enable cluster autoscaling in the Job configuration to optimize resource usage.
- E
Specify the cluster or cluster configuration to execute the Job.
Show answer and explanation
Correct answers: A, C, E
Explanation
To create a Job for processing Delta tables, you must define a task (notebook or JAR) that contains the data processing logic, set a schedule to automate the Job, and specify the cluster configuration to execute the Job. These are mandatory components of a Databricks Job. Optional features, such as enabling cluster autoscaling, can enhance performance but are not required for the Job to function.
- A. Correct.
Defining a task (notebook or JAR) for the Job is a required step to implement the data processing logic.
- B. Incorrect.
JDBC connections are not necessary for accessing Delta tables in Databricks. Delta tables can be read and written directly using Spark APIs.
- C. Correct.
Setting a daily schedule is essential to automate the Job execution at the required time.
- D. Incorrect.
While cluster autoscaling can be beneficial, it is not a required step for configuring a Job. It's an optional optimization feature.
- E. Correct.
Specifying the cluster or cluster configuration is necessary to ensure the Job has the required resources for execution.