Databricks Machine Learning Associate Question 279
Single answerYou are working on a machine learning project in Databricks using Spark ML. You have a DataFrame named data containing features and labels for a supervised learning problem. You need to split the data into training and test sets with 80% of the data used for training and 20% for testing. Which of the following code snippets correctly performs this data split?
- A
trainingData, testData = data.randomSplit([0.8, 0.2])
- B
trainingData, testData = data.split([0.8, 0.2])
- C
trainingData, testData = data.randomSplit([0.8, 0.2], seed=42)
- D
trainingData, testData = data.random_split([0.8, 0.2])
Show answer and explanation
Correct answer: C
Explanation
In Spark ML, the randomSplit method is used to split a DataFrame into multiple subsets based on specified proportions. Adding a seed ensures that the split is reproducible, which is important for debugging and consistency in experiments. The correct syntax is data.randomSplit([proportion1, proportion2], seed=optional_seed).
- A. Incorrect.
This option is almost correct, but it does not include a seed for reproducibility, which is a recommended practice when splitting data.
- B. Incorrect.
This option is incorrect because Spark ML does not have a
splitmethod. The method to use israndomSplit. - C. Correct.
This option is correct because it uses the
randomSplitmethod with the specified proportions (80% and 20%) and includes a seed for reproducibility. - D. Incorrect.
This option is incorrect because
random_splitis not a valid method in Spark ML. The method should berandomSplit.