Databricks Machine Learning Professional Question 257
Select 4You are monitoring a deployed machine learning model and suspect that numerical data drift might be affecting its performance. You are considering using either Jensen-Shannon divergence (JSD) or the Kolmogorov-Smirnov (KS) test to detect drift. Which of the following are TRUE about the characteristics of these two methods?
- A
Jensen-Shannon divergence measures the similarity between two probability distributions and is symmetric.
- B
Kolmogorov-Smirnov test is a non-parametric statistical test that compares the empirical cumulative distribution functions of two samples.
- C
Jensen-Shannon divergence requires binning the data to approximate probability distributions, while Kolmogorov-Smirnov works directly on raw numerical data.
- D
Kolmogorov-Smirnov test can detect differences in both the central tendency and shape of two distributions.
- E
Jensen-Shannon divergence is particularly suited for detecting drift in categorical data rather than numerical data.
Show answer and explanation
Correct answers: A, B, C, D
Explanation
Jensen-Shannon divergence and Kolmogorov-Smirnov test are both effective tools for numerical drift detection, but they operate differently. JSD measures the similarity between probability distributions and requires binning for numerical data, offering a symmetric metric. The KS test, on the other hand, is a non-parametric test that works directly on raw numerical data and identifies differences in the central tendency and shape of distributions. Both are valuable, and understanding their strengths and limitations helps in selecting the appropriate method for a given scenario.
- A. Correct.
Correct: Jensen-Shannon divergence measures the similarity between two probability distributions and is symmetric, making it useful for numerical drift detection in numerical data when distributions are approximated.
- B. Correct.
Correct: The Kolmogorov-Smirnov test is a non-parametric test that compares the cumulative distribution functions of two samples and is widely used for numerical data drift detection.
- C. Correct.
Correct: Jensen-Shannon divergence requires data to be binned into discrete intervals to estimate probability distributions, while the Kolmogorov-Smirnov test works directly on raw numerical data without binning.
- D. Correct.
Correct: The Kolmogorov-Smirnov test can capture differences in both the central tendency and distribution shape, making it versatile for numerical drift detection.
- E. Incorrect.
Incorrect: While Jensen-Shannon divergence can theoretically be applied to categorical data, it is also effective for numerical data when approximate probability distributions are used. Thus, it is not limited to categorical data.