AI Risk Management
Free ISACA Advanced in AI Security Management practice — 6 questions on AI Risk Management, with explanations. No sign-up. Full 12-question mixed test →
A security manager is assessing risk for a computer-vision model licensed from a third-party vendor. The vendor discloses that the training dataset was aggregated from multiple public web scrapes with no documented chain of custody, and the vendor cannot confirm whether copyrighted or PII-containing images were filtered out. The model is scheduled for deployment in a public-facing product within 30 days. Using AI risk assessment principles, which action should the security manager prioritize BEFORE deployment?
AI risk assessment requires treating provenance gaps in training data as a distinct risk category from output-level controls; when provenance cannot be verified and legal/privacy exposure exists, the correct treatment is to obtain attestation and independently verify before accepting residual risk, especially given a hard deployment deadline.
A bank's AI-based transaction fraud detection model has a documented false-negative rate that fraud rings have begun exploiting by structuring transactions just below the model's decision threshold — a classic evasion technique. The security team's risk register currently rates this threat as 'Low' because the model's overall accuracy remains above 98%. From an AI risk management perspective, what is the MOST significant flaw in this rating?
AI risk assessment for adversarial threats must evaluate targeted, exploitable weaknesses (evasion at the decision boundary) rather than relying on aggregate performance metrics; a high overall accuracy figure can mask a systematically exploitable blind spot with significant financial impact — this is a core AAISM distinction between traditional model performance metrics and security-relevant robustness metrics.
An attacker sends carefully crafted queries to a publicly exposed model-inference API, systematically varying inputs and recording outputs to reconstruct a functionally equivalent copy of the proprietary model without ever accessing the underlying weights or infrastructure. Which MITRE ATLAS-aligned threat category does this technique represent?
Systematic input/output querying to reverse-engineer or clone a model's functionality is the defining pattern of model extraction (model theft), a technique explicitly cataloged in the MITRE ATLAS matrix under exfiltration/theft of ML artifacts via API abuse.
During a quarterly AI risk review, a residual risk score of 18 (on a 1–25 scale, where likelihood and impact are each rated 1–5) is calculated for an AI supply-chain compromise scenario involving a compromised open-source model repository used in the organization's ML pipeline. The organization's documented AI risk appetite threshold is 12. According to AI risk treatment principles, what is the CORRECT next step?
When a residual risk score exceeds the defined risk appetite threshold, governance requires formal treatment selection from the standard options (mitigate, transfer, avoid, or accept), with acceptance requiring explicit documented approval at an appropriate authority level — simply logging or ignoring the exceedance violates the risk management lifecycle.
A security manager is negotiating a contract with a hosted foundation-model provider that will process the organization's confidential customer data through its API for a customer-service chatbot. The vendor's standard terms are silent on whether customer inputs may be used to further train or fine-tune the provider's models. From an AI vendor risk management standpoint, which contractual requirement is MOST critical to add before signing?
For hosted/foundation-model vendors, the highest-priority security and privacy risk is unauthorized use of confidential inputs for further model training or retention beyond need, which can lead to data leakage into future model outputs served to other customers; an explicit data-usage and retention clause directly addresses this AI-specific supply-chain risk and is a core AAISM vendor-management concern.
During threat modeling for a deployed recommendation model, an analyst distinguishes between two attack techniques: one where an adversary queries the model repeatedly to determine whether a specific individual's record was part of the training dataset, and another where an adversary manipulates input features at inference time to force a misclassification without altering the model itself. Which pairing correctly labels these two techniques?
Membership inference determines whether a specific data record was used in training by analyzing model confidence/output patterns on queries — a privacy attack. Evasion involves crafting inputs at inference time to cause misclassification without modifying the model — an integrity attack against the deployed model's decision boundary. This distinction is a core part of the AI threat taxonomy tested in AAISM.
Ready for the real thing?
The full course: two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed explanations.
undefined $34.99 with code FREETEST33 — valid through Oct 6.
Get my $34.99 deal →