AI-102 Question 272
Select 3You are tasked with creating a language understanding model to analyze customer service queries for categorization into predefined intents. After training the model using Azure Language Understanding (LUIS), you want to ensure the model performs well before deploying it to production. Which steps should you take to evaluate the performance of your model and ensure its readiness for deployment?
- A
Use labeled test data to calculate the precision, recall, and F1-score of the model.
- B
Directly deploy the model and observe user feedback to assess performance.
- C
Test the model using utterances it has not seen during training to check its generalization.
- D
Manually review the model's predictions on training data to confirm high accuracy.
- E
Evaluate the model's robustness by testing with slightly modified or misspelled input utterances.
Show answer and explanation
Correct answers: A, C, E
Explanation
To ensure a language understanding model is ready for deployment, it must be rigorously tested using labeled test data to calculate metrics like precision, recall, and F1-score. Testing on unseen utterances evaluates the model's generalization, while testing with modified or misspelled inputs determines its robustness. These steps help identify weaknesses and optimize the model before deployment. Avoid relying solely on training data evaluation or skipping pre-deployment evaluations to prevent subpar performance in real-world scenarios.
- A. Correct.
Using labeled test data to calculate precision, recall, and F1-score is a standard practice for evaluating machine learning models, including LUIS. This ensures you understand the model's ability to correctly classify intents.
- B. Incorrect.
Directly deploying the model without prior evaluation is not recommended as it may result in poor user experience and incorrect intent predictions. Proper evaluation should be conducted before deployment.
- C. Correct.
Testing the model with unseen utterances (validation or test data) is critical to assess generalization and avoid overfitting to training data.
- D. Incorrect.
Manually reviewing predictions on training data only evaluates how well the model memorized the training data, which is not a good indicator of performance on unseen data.
- E. Correct.
Evaluating with slightly modified or misspelled utterances ensures the model is robust and flexible enough to handle real-world input variations.