NCA-GENL Question 191
Select 3You are tasked with contributing to a generative AI project focused on fine-tuning a large language model (LLM) for a customer support chatbot. Under the guidance of a senior developer, you are asked to write a script to preprocess customer conversation data by performing the following steps: tokenizing text, removing stop words, and lowercasing all text. Which of the following practices should you follow when writing the script?
- A
Use a well-documented library like NLTK or SpaCy for tokenization and stop word removal.
- B
Manually code the tokenization process from scratch to demonstrate your understanding.
- C
Add comments in the script to explain each preprocessing step in detail.
- D
Ensure that the script is modular and reusable for other datasets.
- E
Directly save the preprocessed data into a proprietary format without consulting the senior developer.
Show answer and explanation
Correct answers: A, C, D
Explanation
When contributing to a generative AI project under supervision, it is important to rely on reliable tools and libraries, write clear and maintainable code, and follow best practices such as modularity and commenting. Consulting the senior developer before making key decisions ensures alignment with the project’s goals and standards.
- A. Correct.
Using a well-documented library ensures reliability, efficiency, and maintainability, particularly when working with standard NLP tasks.
- B. Incorrect.
Manually coding tokenization is unnecessary and error-prone, especially when robust libraries already exist for this purpose.
- C. Correct.
Adding comments improves code readability and makes it easier for team members, including the senior developer, to review and understand your work.
- D. Correct.
Writing modular and reusable code is a good practice, as it allows for scalability and adaptability when working with other datasets.
- E. Incorrect.
Saving data in a proprietary format without consulting the senior developer can lead to compatibility or workflow issues. Always align with team guidelines before making such decisions.