Databricks Data Engineer Associate Question 303
Single answerWhen using Delta Lake's OPTIMIZE command, which type of files does it compact?
- A
Small files created by frequent writes or updates
- B
Large files that exceed a predefined size threshold
- C
Corrupted files in the Delta Lake table
- D
Files that are not part of the Delta log
Show answer and explanation
Correct answer: A
Explanation
Delta Lake's OPTIMIZE command is used to improve query performance by compacting small files into fewer, larger files. This process addresses the problem of small files generated during frequent writes or updates, which can otherwise degrade performance due to higher metadata overhead.
- A. Correct.
Correct. The OPTIMIZE command in Delta Lake is specifically designed to compact small files into larger ones to improve query performance and reduce metadata overhead.
- B. Incorrect.
Incorrect. Large files exceeding the size threshold are not compacted by OPTIMIZE. The focus is on merging small files, not splitting large ones.
- C. Incorrect.
Incorrect. OPTIMIZE does not handle corrupted files. Handling corrupted files usually involves manual intervention or specific tools for data recovery.
- D. Incorrect.
Incorrect. OPTIMIZE only works on files that are part of the Delta log. Files outside the Delta log are ignored.