Databricks Data Engineer Associate Question 410
Select 2You are tasked with auditing the usage of a Databricks workspace by querying the event logs. Specifically, you need to identify which users have executed DELETE statements on tables in the past 7 days. Which of the following queries or methods can you use to achieve this?
- A
Query the
event_logtable in the Databricks metastore and filter for events with theeventTypecolumn set to 'delete' and atimestampwithin the past 7 days. - B
Use the
DESCRIBE HISTORYcommand on each table to review past operations and filter for DELETE operations within the past 7 days. - C
Query the
event_logtable'sdetailscolumn for JSON keys indicating DELETE operations, and filter by thetimestampcolumn for the past 7 days. - D
Enable cluster logging to write all SQL queries to an external storage location, then search for DELETE statements manually.
- E
Directly use the Databricks REST API to fetch DELETE operations logs from the event logs endpoint, filtering by event type and timestamp.
Show answer and explanation
Correct answers: A, C
Explanation
The event_log table is the primary source of information for auditing operations in a Databricks workspace. By filtering for DELETE operations using either the eventType column or parsing the details column, you can efficiently identify relevant events. Other methods, like manual searching or using the REST API, are either impractical or not supported for this specific use case.
- A. Correct.
Correct: The
event_logtable contains information about events in the workspace, including SQL operations like DELETE. Filtering byeventTypeand a time range is a valid approach to identify such actions. - B. Incorrect.
Incorrect: The
DESCRIBE HISTORYcommand provides information about Delta table version history but does not directly allow querying DELETE statements by users or their timestamps. - C. Correct.
Correct: The
detailscolumn in theevent_logtable contains JSON data, which can be parsed to identify operation types like DELETE. Filtering by timestamps is also valid. - D. Incorrect.
Incorrect: While enabling cluster logging can capture SQL queries, this option would require manual searching and does not leverage the structured nature of the event logs, making it inefficient and impractical.
- E. Incorrect.
Incorrect: The Databricks REST API does not provide a direct endpoint to fetch specific SQL operation logs like DELETE. Event logs must be queried using SQL or similar tools.