Databricks Data Engineer Associate exam dumps

Databricks Data Engineer Associate practice question 358 of 532

Databricks Certified Data Engineer Associate. Associate level, Databricks. Free question with the correct answer and a full explanation.

Databricks Data Engineer Associate Question 358

Select 2

You are working on a Databricks notebook and need to load data from a cloud storage location into a Delta table using the COPY INTO command. The data in the source files is in CSV format, and you want to ensure that the command only processes files that have not been previously loaded. Which of the following configurations should you include in the COPY INTO command to achieve this?

  1. A

    Specify the FILES parameter to explicitly list files to be processed.

  2. B

    Use the FORMAT option to specify the file format as 'CSV'.

  3. C

    Set FILEFORMAT_OPTIONS to include 'mergeSchema' for handling schema evolution.

  4. D

    Enable the AUTOLOADER option in the COPY INTO command.

  5. E

    Use the COPY_OPTIONS parameter to include 'mergeSchema' for schema evolution.

  6. F

    Ensure you include the 'overwrite' option to overwrite the Delta table with new data.

Show answer and explanation

Correct answers: A, B

Explanation

The COPY INTO command in Databricks is designed for loading data incrementally into Delta tables. To achieve this, specifying the correct file format using the FORMAT option is essential to parse the data correctly. Additionally, explicitly listing files using the FILES parameter ensures that only new or specific files are loaded, preventing duplication. Other options, such as 'mergeSchema' or 'overwrite', are unrelated to the COPY INTO command, while AUTOLOADER is a separate functionality.

  • A. Correct.

    Specifying the FILES parameter allows you to explicitly list the files to be processed, ensuring only specific files are loaded. This can help avoid re-processing previously ingested files.

  • B. Correct.

    The FORMAT option is required to specify the format of the source files. In this case, it should be set to 'CSV' to correctly parse the data.

  • C. Incorrect.

    The FILEFORMAT_OPTIONS parameter does not exist for the COPY INTO command. Instead, schema evolution is not directly handled within COPY INTO itself.

  • D. Incorrect.

    AUTOLOADER is a separate mechanism in Databricks for ingesting data incrementally, and it is not used in the COPY INTO command.

  • E. Incorrect.

    The COPY_OPTIONS parameter does not include 'mergeSchema' since schema evolution is not a feature of COPY INTO.

  • F. Incorrect.

    The 'overwrite' option is not used in COPY INTO. COPY INTO loads new data incrementally rather than overwriting the Delta table.

Timed practice exam

Take a Databricks Data Engineer Associate practice test under exam conditions

45 questions in 90 minutes, drawn from this bank, with a score report and a per-question review when you finish.

Start timed exam

More Databricks Data Engineer Associate practice questions

  1. 359.You are tasked with loading new JSON data from a cloud storage location into an existing Delta...
  2. 360.A data engineer is tasked with loading data from a cloud storage location into a Delta table using...
  3. 361.You are working on a Databricks notebook and need to incrementally load new data from an external...
  4. 362.You are working on a Databricks pipeline to process and load CSV files into a Delta table named...
  5. 363.You are tasked with creating a new Delta Live Tables (DLT) pipeline in Databricks. Which of the...
Back to the Databricks Data Engineer Associate exam dumps