Databricks Data Engineer Associate Question 412
Select 3You are working as a data engineer, and your team needs to track the usage of a specific notebook to ensure compliance with internal audit policies. How can you query the Databricks event logs to retrieve information about notebook usage and ensure that these logs can be used for auditing purposes?
- A
Query the
event_logtable in the Databricks workspace to retrieve events related to the notebook. - B
Filter the event logs for
notebookevents and analyze theactionNamefield to identify operations such asrunoredit. - C
Use the Databricks REST API to extract event logs in real-time and store them externally for compliance.
- D
Access the event logs directly from the Databricks file system (DBFS) under the
/logsdirectory. - E
Query the Delta table containing event logs and join it with the workspace metadata to track user-specific notebook activity.
Show answer and explanation
Correct answers: A, B, E
Explanation
To retrieve and analyze notebook usage for auditing purposes, you can query the event_log table, filter for relevant notebook events, and enrich the data by joining it with workspace metadata. This approach ensures detailed tracking of notebook operations while aligning with compliance requirements. The Databricks REST API and raw DBFS logs are not appropriate methods for this use case.
- A. Correct.
Correct: The
event_logtable in Databricks contains structured logs that can be queried to track various activities, including notebook usage. - B. Correct.
Correct: Filtering for
notebookevents and analyzing fields likeactionNameallows you to identify specific notebook-related operations such as runs or edits. - C. Incorrect.
Incorrect: While the Databricks REST API can be used to manage workspaces, it is not the correct approach for querying event logs directly.
- D. Incorrect.
Incorrect: Event logs are not stored directly in the
/logsdirectory on DBFS. This location is not relevant for querying event metrics. - E. Correct.
Correct: Joining the event log data with workspace metadata provides additional context, such as user-specific activity, making it useful for auditing.