Course:ELEC300E/CaseStudies/ARandomAIIncident
Context: "Pacing the Frontier"
| Navigating Good Food |
|---|
 |
| Content / Topics |
| Axiological Design |
| AI |
| Sustainable Computing |
| Indigenous AI |
| Relevant Analysis Strategies or Tools |
| Decision Impact Assessment |
| Iceberg Model |
As AI capabilities have advanced, the list of worrying incidents (cases of AI causing harm or being used to cause harm) has grown.
But which risks are most dangerous and severe?
MIT's AI Risk Initiative interviewed 272 international AI experts about which risks they found most concerning. You can read their full research report here. The three main categories of dangerous risk that experts pointed out were:
- A race towards inherently dangerous capabilities: Companies are racing to develop increasingly powerful - and dangerous - AI systems.
- Misuse of AI for mass harm: misinformation campaigns, developing bioweapons, cyberattacks on critical infrastructure, fraud
- Concentration of power: The people who have and control AI have a great deal of political, economic and military power - which is risky for the general public.
Who is responsible for addressing AI risks? And who is most at risk?
The researchers identify stakeholders, and outline their risks and responsibilities to mitigate those risks:
- Affected Stakeholders: "Entities indirectly affected by AI decisions or outputs. Examples: Communities impacted by automated decisions, advocacy groups"
- AI Infrastructure Providers: "Entities that provide compute, cloud infrastructure, and/or data to train and run AI. Examples: Nvidia, AMD (compute); AWS, Google Cloud, Microsoft Azure (cloud)"
- AI Governance Actors: "Entities that create or enforce laws, regulations, standards or guidelines for AI. Examples: Governments, regulators, standards bodies, policy makers"
- AI User: "Entities that use or rely on AI systems without significant modification. Examples: Businesses using AI transcription services; software engineers using GitHub Copilot"
- AI Deployer: "Entity that implements AI systems in products/services used within an organization or delivered to customers. Examples: JPMorgan (fraud detection), Walmart (inventory management), Netflix, Meta"
- AI Developer (Specialized AI): "Entities that create specialized AI systems for specific applications/industries. Examples: Aidoc (radiology AI), Zest AI (credit underwriting), Uptake (predictive maintenance)"
- AI Developer (General-Purpose AI): Entity that creates general-purpose foundation models. Examples: OpenAI, Anthropic, Google DeepMind
The experts placed responsibility primarily with AI developers, and governments. They found average users and stakeholders are most vulnerable to safety risks.
Case: A Random AI Incident
The AI incident database lists thousands of 'AI incidents', from cases of 'AI psychosis' and misinformation using AI deepfakes to coordinated AI-assisted hacking. This database was also used by the MIT AI Risk initiative to develop their framework of AI risks.
Use the database to find a random AI incident. Read the incident and identify which of the stakeholders identified above were involved. Which stakeholders would be responsible for (or would be able to) mitigate the risk of similar incidents?
Discussion Questions
Here are a few questions to help you begin analyzing the issues discussed in this case study. The purpose of these questions is to prompt reflection and further consideration: there are no right or wrong answers!
- Personal response: Take stock of your personal response to the researchers' warnings, recent news coverage about AI threats, and the random incident you chose: anger, frustration, fear, apprehension, hope, defensiveness, boredom? How can you leverage your own response to help build or advocate for safer and more ethical systems?
- Contradictory viewpoints and outcomes: Consider the random AI incident that you chose through the perspective of two stakeholders. What are their needs and priorities? How are they impacted? Which risks are they vulnerable to, and which mitigations are they responsible for? What decisions or actions might they advocate for?
- Social and Historical Context: AI isn't the first threat to public safety - consider pharmaceutical safety, nuclear facility safety, environmental contamination, etc. Can you draw any parallels between your random incident and historical incidents or industries that have threatened the general public in a similar way?
- Interruptions and invitations: As the MIT researchers point out, there is an asymmetry between responsibility and vulnerability when it comes to AI risk: "Affected stakeholders lack both the agency and systemic leverage to mitigate risk, and assigning responsibility to them risks reinforcing harm by misplacing accountability." Is this the case for your random case study? How can more vulnerable or more impacted stakeholders be given more leverage?
- Physical connections: Does your random case study relate to your own personal well being or health, or the health of the land you live on? How so? Can you draw any connections between this incident and your own lived experience, or the issues facing your local environment and community?
- Positive Practices: Based on your random case study, what is one practice that you could adopt or a practice that you would advocate for another stakeholder to adopt to mitigate risk. (For example, if your case study was about fraud using AI voice cloning, you might help educate your family members about the existence of this type of fraud or advocate for legislation that puts more responsibility on banks to recognize suspicious transactions).
- Positive action: More broadly, can you imagine a positive future, where risks associated with AI are minimized. What does this imagined future look like to you? What would it take for us to get there?