NCA-GENL Question 62
Select 3You are building a Named Entity Recognition (NER) system using Python. To preprocess the dataset and extract token features, you decide to use spaCy for tokenization and NumPy for numerical operations. Which of the following steps correctly implement this pipeline?
- A
Use spaCy to load a language model and tokenize the text into words and entities.
- B
Use NumPy to compute the cosine similarity of token embeddings for clustering.
- C
Extract part-of-speech (POS) tags and entity labels using spaCy and convert them into numerical arrays using NumPy.
- D
Train the NER model using Keras by feeding in raw text directly without any preprocessing.
- E
Use spaCy's pre-trained language model to generate word vectors and convert them into a NumPy matrix for feature representation.
Show answer and explanation
Correct answers: A, C, E
Explanation
Named Entity Recognition (NER) systems require preprocessing steps such as tokenization, feature extraction, and numerical representation of text data. spaCy is well-suited for tokenization and linguistic feature extraction, while NumPy can handle numerical transformations. Using pre-trained models in spaCy for vectorization and converting the data into a NumPy matrix are standard practices. Training a model directly on raw text, however, is not recommended, as preprocessing is critical for successful model performance.
- A. Correct.
Correct: spaCy is commonly used to load language models and tokenize text, making it suitable for breaking text into words and entities.
- B. Incorrect.
Incorrect: While NumPy can perform numerical operations, cosine similarity is not typically a step in preprocessing for NER tasks. It is more relevant for tasks like clustering or similarity analysis.
- C. Correct.
Correct: spaCy can extract linguistic features like part-of-speech tags and entity labels, which can then be converted into numerical data using NumPy for further processing.
- D. Incorrect.
Incorrect: Raw text input without preprocessing is not suitable for training an NER model. Preprocessing steps like tokenization and feature extraction are essential.
- E. Correct.
Correct: SpaCy's pre-trained language models can generate word vectors, which can be converted into a NumPy matrix for structured numerical feature representation.