Databricks Data Engineer Associate Question 362
Select 3You are working on a Databricks pipeline to process and load CSV files into a Delta table named sales_data. The files are stored in an S3 bucket at s3://company-data/sales/. You decide to use the COPY INTO command to ingest the data. Which of the following considerations are mandatory to ensure the COPY INTO command executes successfully?
- A
The Delta table
sales_datamust already exist before executing theCOPY INTOcommand. - B
The source files in the
s3://company-data/sales/location must have the same schema as the Delta tablesales_data. - C
The
COPY INTOcommand automatically creates the Delta table if it does not exist. - D
You must specify the file format (e.g., CSV, JSON, etc.) in the
COPY INTOcommand. - E
The files in the source location can be in any format, as the
COPY INTOcommand automatically detects the format.
Show answer and explanation
Correct answers: A, B, D
Explanation
The COPY INTO command is used to efficiently load data into a Delta table. To execute successfully, the Delta table must exist, the schema of the source files must match the table schema, and the file format of the source files must be explicitly provided. Misunderstanding these requirements can lead to errors or failures during execution.
- A. Correct.
Correct: The
COPY INTOcommand requires the target Delta table to already exist. It does not create the Delta table automatically. - B. Correct.
Correct: The schema of the source files must match the schema of the target Delta table; otherwise, the command will fail.
- C. Incorrect.
Incorrect: The
COPY INTOcommand does not create tables. The table must be created before running the command. - D. Correct.
Correct: The file format must be explicitly specified in the
COPY INTOcommand, such as using theFORMAToption. - E. Incorrect.
Incorrect: The
COPY INTOcommand does not automatically detect the file format. You must define the format explicitly.