Databricks Machine Learning Associate Question 282
Single answerYou are working on a machine learning project in Databricks and need to split a dataset into training and testing subsets using Spark ML. Which of the following code snippets correctly splits the data into 80% training and 20% testing subsets?
- A
data.randomSplit([0.8, 0.2], seed=42)
- B
data.split([0.8, 0.2], seed=42)
- C
data.randomSplit([0.2, 0.8], seed=42)
- D
data.randomSplit([0.8, 0.2])
Show answer and explanation
Correct answer: A
Explanation
The randomSplit method in Spark is the standard way to split a dataset into multiple subsets. It takes an array of weights (e.g., [0.8, 0.2]) to specify the proportions of each subset and optionally accepts a random seed to ensure consistent splits across different runs. The correct answer includes the proper proportions (80% for training, 20% for testing) and uses a random seed for reproducibility.
- A. Correct.
This is the correct syntax for splitting a dataset in Spark ML into training and testing subsets.
randomSplitis the correct method, and the proportions (0.8, 0.2) represent 80% for training and 20% for testing. A random seed is included to ensure reproducibility. - B. Incorrect.
This is incorrect because
splitis not a valid method in Spark ML for splitting data. The correct method israndomSplit. - C. Incorrect.
This is incorrect because the proportions are reversed; 0.2 would assign 20% to training and 80% to testing, which does not match the requirements of the question.
- D. Incorrect.
This is incorrect because while the proportions are correct, the absence of a random seed makes the split non-deterministic, which can lead to inconsistent results when running the code multiple times.