AIF-C01 Question 106
Single answerYou are deploying a natural language processing (NLP) model on Amazon SageMaker for generating text responses. During testing, you notice that the responses are highly random and lack coherence. Which inference parameter should you adjust to make the model's responses more focused and deterministic?
- A
Increase the temperature parameter
- B
Decrease the temperature parameter
- C
Increase the maximum output length parameter
- D
Decrease the input length parameter
Show answer and explanation
Correct answer: B
Explanation
The temperature parameter controls the randomness in a model's predictions. A higher temperature allows for more diverse outputs, while a lower temperature makes the model's responses more deterministic and focused. In this scenario, reducing the temperature will mitigate the issue of incoherent and overly random responses, making the output more predictable and aligned with the expected behavior.
- A. Incorrect.
Increasing the temperature parameter introduces more randomness into the model's predictions, making the output less deterministic and more varied. This would worsen the issue of incoherence.
- B. Correct.
Decreasing the temperature parameter reduces randomness in the model's predictions, making the output more deterministic and focused. This is the correct solution to address the issue of incoherent responses.
- C. Incorrect.
Increasing the maximum output length parameter allows the model to generate longer responses, but it does not directly address the issue of randomness or coherence in the text generation.
- D. Incorrect.
Decreasing the input length parameter limits the amount of input provided to the model, which may lead to less context for generating responses but does not directly control randomness or coherence.