NCA-GENM Question 24
Single answerYou are designing a multimodal generative AI model that integrates text and image inputs. The model often encounters vanishing gradient issues during training, particularly in deep layers. To resolve this, the team decides to implement residual connections. How do residual connections address this issue?
- A
Residual connections allow the gradient to flow directly to earlier layers, reducing the risk of vanishing gradients.
- B
Residual connections add additional hidden layers to the network, increasing model complexity to handle vanishing gradients.
- C
Residual connections completely replace non-linear activation functions to avoid gradient loss.
- D
Residual connections skip certain layers during forward propagation, ensuring information is not degraded.
Show answer and explanation
Correct answer: A
Explanation
Residual connections are a key feature of nonsequential neural networks, often used in architectures like ResNet. They address the vanishing gradient problem by introducing shortcut paths that allow gradients and information to flow directly to earlier layers without degradation. This improves training stability and performance, especially in deep networks.
- A. Correct.
Residual connections allow the gradient to flow directly to earlier layers by introducing shortcut paths. These paths help mitigate the vanishing gradient problem, especially in deep neural networks, by ensuring information and gradients are preserved.
- B. Incorrect.
Adding additional hidden layers increases model depth, which can exacerbate the vanishing gradient problem rather than solve it. This is not how residual connections work.
- C. Incorrect.
Residual connections do not replace non-linear activation functions. They work alongside these activations to stabilize the training process by introducing shortcut paths.
- D. Incorrect.
Residual connections do not skip layers during forward propagation. Instead, they add the output of earlier layers to later layers, ensuring that critical information is retained and gradients can flow more effectively.