712-50 Question 35
Select 3A global financial services company plans to deploy a generative AI assistant to help analysts summarize customer complaints, draft responses, and identify potential conduct-risk trends. During a pilot, the CISO learns that the model sometimes produces confident but incorrect summaries, the business cannot clearly explain why certain complaints are flagged as higher risk, and employees have been pasting full customer records into prompts to improve output quality. The board wants the fastest path to production, but also expects the CISO to address the most significant AI-specific risks before approval. Which THREE actions should the CISO prioritize to reduce the most material risks in this scenario?
- A
Implement prompt and data-handling controls that minimize sensitive data exposure, including input filtering, redaction/tokenization of customer information, and contractual restrictions on model-provider data retention and secondary use
- B
Establish human review and outcome validation for high-impact use cases, and require testing for hallucinations, bias, and performance against representative complaint datasets before production release
- C
Accept limited model explainability because all machine learning systems are inherently opaque, and focus only on perimeter security controls around the application
- D
Adopt model governance requirements such as use-case approval, transparency documentation, and monitoring for drift, anomalous outputs, and emerging vulnerabilities after deployment
- E
Reduce cyber risk by allowing unrestricted employee prompting so the model has more context, which will improve accuracy enough to offset privacy and security concerns
Show answer and explanation
Correct answers: A, B, D
Explanation
The best answer is 1, 2, and 4 because they collectively address the primary AI risks presented: data security, inaccuracies, limited transparency, and model vulnerabilities/governance gaps. In this scenario, the CISO should not treat the system as just another application protected by traditional network controls. AI introduces distinct risks, including hallucinated or inaccurate outputs, biased or unvalidated outcomes, opaque decision-making, data leakage through prompts, and model-specific operational weaknesses.
From a CCISO perspective, the decision should be risk-based and business-aligned. High-priority actions include: (1) protecting sensitive data through minimization, redaction, access control, and vendor restrictions; (2) validating outputs with representative testing and human review for higher-impact use cases; and (3) implementing governance across the AI lifecycle, including approval criteria, transparency artifacts, monitoring, and incident response integration.
These actions align with widely recognized best practices such as the NIST AI Risk Management Framework (govern, map, measure, manage), which emphasizes managing validity and reliability, safety, security and resilience, privacy enhancement, transparency, explainability, fairness, and accountability. They are also consistent with ISO/IEC 23894 guidance on AI risk management and general security principles from ISO/IEC 27001 and privacy-by-design approaches. In regulated environments like financial services, these controls are particularly important because inaccurate or biased outputs can create customer harm, conduct risk, and regulatory exposure, while improper prompt handling can lead to confidentiality and data protection failures.
- A. Correct.
Correct. This directly addresses one of the clearest risks in the scenario: employees entering full customer records into prompts. For a financial services firm, this raises data security, privacy, confidentiality, and third-party risk concerns. Practical controls include data minimization, redaction or tokenization of personally identifiable information (PII), prompt filtering, access control, logging, and clear provider terms prohibiting retention or reuse of enterprise prompts and data for model training unless explicitly approved. This is aligned with security and privacy-by-design principles and emerging AI governance guidance.
- B. Correct.
Correct. The scenario highlights inaccurate outputs presented with confidence, which is a classic generative AI risk often described as hallucination or unreliable output generation. In regulated or customer-impacting workflows, human oversight is important before actions are taken based on model output. Testing against representative datasets is also necessary to assess accuracy, bias, false positives/negatives, and consistency across complaint categories and customer segments. This addresses both inaccuracies and fairness risk in a measurable way.
- C. Incorrect.
Incorrect. While some AI models are less interpretable than traditional systems, it is not acceptable for a CISO or governance function to ignore transparency and focus only on perimeter controls. The scenario includes unexplained risk flagging, which creates model risk, compliance risk, and accountability concerns. Explainability may be achieved through documentation, model cards, feature/decision rationale where feasible, use-case limitations, and compensating controls. Perimeter security alone does not address bias, inaccurate outputs, or lack of governance.
- D. Correct.
Correct. This option addresses governance and lifecycle risk. The scenario is not only about current pilot issues but also about production readiness and ongoing oversight. Use-case approval ensures the AI system is appropriate for the business purpose and risk level. Transparency documentation helps stakeholders understand intended use, limitations, data sources, and controls. Monitoring after deployment is essential because model behavior can degrade over time, new attack techniques can emerge, and output patterns may shift. This is especially important for model vulnerabilities, drift, and operational resilience.
- E. Incorrect.
Incorrect. More context can sometimes improve model output, but unrestricted prompting with sensitive customer information materially increases privacy, confidentiality, and regulatory risk. It also expands the attack surface for data leakage and misuse. Accuracy improvements do not justify bypassing core security and privacy controls. This option reflects a common misconception that productivity gains should override data protection requirements.