Governing AI Downstream Harms

A model can pass every pre-launch test and still do damage in the field. The AIGP Body of Knowledge, the blueprint that maps what the exam tests, treats this as its own discipline: forecasting and reducing the risks of secondary or unintended uses and downstream harms once a system is live. Downstream harms are the harms that reach people and groups after a system ships, often people who never touched it. Domain IV of the curriculum, governing AI deployment and use, is where the work sits, and it carries real weight in the question count.

The design-time risk assessment is not the finish line. It is the opening position. A system meets the world through users you did not pick, in contexts you did not model, and the harm that follows is yours to anticipate.

Secondary use versus unintended use

The exam keeps two terms apart, and so should you. Secondary use is the system, or its output, put to a purpose it was not built, validated or approved for. A model trained to flag payment fraud gets pointed at job applicants; the purpose changed and the validation did not follow. Unintended use is narrower. Users stay inside the stated purpose but apply the system in ways the provider never foresaw, including deliberate misuse. Secondary use is a purpose problem. Unintended use is a behaviour problem. Read the stem for which one it describes before you commit to an answer.

The forces that make AI harm travel

Four traits named in the Body of Knowledge explain why this matters more for AI than for ordinary software: scale, speed, autonomy and probabilistic output. A flawed spreadsheet harms one desk. A flawed model behind an API harms every request, in seconds, with logic no reviewer can fully inspect. Downstream harm is the same error multiplied by reach.

When you map who a system can hurt, the exam wants third parties in the frame, not only the direct user. A credit model's downstream harm lands on the rejected applicant. A triage tool's downstream harm lands on the patient behind the clinician. If your probability-and-severity harms matrix stops at the user, it is measuring the wrong thing.

The Body of Knowledge frames the response as a mitigation hierarchy: remove the risk where you can, reduce it where you cannot and disclose what is left. Applied downstream, that means designing out the tempting misuse before launch, not bolting a warning label on after the harm has landed.

The deployer's duties and the trap inside them

Under the EU AI Act, the deployer of a high-risk system carries live duties. Use the system in line with the provider's instructions, keep human oversight genuine rather than nominal and tell the provider when a risk or serious incident appears (Article 26). Providers, for their part, run post-market monitoring and report serious incidents under Articles 72 and 73. None of it is a one-time sign-off; it is a standing loop.

Here is the trap. Change the intended purpose of a high-risk system, or put your own name on it, and Article 25 can turn a deployer into a provider, with the heavier obligations that follow. Secondary use is not only a risk to the public. It can move legal responsibility onto you. Candidates who assume the provider is the sole accountable party walk into this one on a regular basis.

Controls that reduce downstream harms

The governance job is to make secondary and unintended use hard and to catch downstream harm early. The controls that carry weight:

• Define and document the intended purpose, then give deployers clear instructions for use, so off-label use shows up as off-label.
• Set an acceptable-use policy with contractual limits on downstream use, not a buried wiki note.
• Run continuous and post-market monitoring for misuse and drift, not performance figures alone.
• Position human oversight to intervene, with a real route to escalate.
• Keep a backstop: the ability to deactivate or localise the system when harm outruns the controls.

These are not five separate projects. They are one loop: state the purpose, restrict the use, watch the live system, intervene and keep the power to pull it back. A downstream harm caught in monitoring is a footnote. The same harm caught by a regulator is a case.

What the exam rewards

The mark goes to the candidate who treats deployment as a live obligation, separates secondary use from unintended use and remembers that downstream harms reach past the user to the people a system acts upon. An answer that stops at "we tested it before launch" has missed the point the Body of Knowledge is making. Governing downstream harms is continuous, contractual and evidenced, or it is not governance at all.

If you want to drill Domain IV until deployment questions feel routine, work through the practice sets at 22academy.com/study.