NCA-GENL Question 163
Select 2You are building a generative AI application that requires named entity recognition (NER), semantic similarity calculations, and efficient storage/retrieval of text embeddings. Which combination of Python packages is most suitable for achieving these tasks?
- A
spaCy for NER, NumPy for embedding calculations, and a vector database for storage and retrieval
- B
NLTK for NER, pandas for embedding calculations, and SQLite for storage and retrieval
- C
spaCy for NER, scikit-learn for embedding calculations, and a vector database for storage and retrieval
- D
spaCy for NER, NumPy for embedding calculations, and Elasticsearch for storage and retrieval
Show answer and explanation
Correct answers: A, D
Explanation
When working with generative AI applications requiring NER, semantic similarity, and embedding storage/retrieval, spaCy is a strong choice for NER due to its pre-trained models and efficient pipeline. NumPy is widely used for numerical computations, including embedding manipulation. Finally, vector databases or systems like Elasticsearch (with dense vector support) are specifically designed for high-dimensional vector search, making options 1 and 4 the most suitable combinations.
- A. Correct.
spaCy is a robust library for NER, NumPy is excellent for handling numerical data like embeddings, and vector databases are optimized for storing and retrieving high-dimensional vectors efficiently. This is a valid and efficient combination for the tasks.
- B. Incorrect.
NLTK is not as advanced as spaCy for NER, pandas is primarily for data manipulation and not for numerical computation of embeddings, and SQLite is not optimized for storing and searching embeddings, making this combination suboptimal.
- C. Incorrect.
While spaCy can handle NER well, scikit-learn is primarily used for machine learning and lacks the low-level numerical computation efficiency of NumPy for embeddings. Furthermore, a vector database would typically outperform general-purpose databases for embedding retrieval tasks.
- D. Correct.
This combination is valid as spaCy effectively handles NER, NumPy is highly efficient for embedding calculations, and Elasticsearch (configured with dense vector support) is a capable solution for embedding storage and retrieval. This makes it another suitable option.