The Co-Creation Futures Framework
Before we wrote a single line of code or trained an algorithm, we had to answer a fundamental question: What does the AI actually need to learn?
Mapping the Reality (Discovery)
Before we wrote a single line of code or trained an algorithm, we had to answer a fundamental question: What does the AI actually need to learn?
In traditional tech development, engineers might simply scrape standard medical dictionaries and feed them into a machine learning model. But deploying Western-centric, lab-based data in a noisy, resource-constrained Ugandan clinic is a recipe for failure.
During Day 1 (The Discovery Phase) of our research, we shifted the focus from building a tool to mapping the reality of the "speech-locked" clinic. We used participatory inquiry—including contextual interviews and journey mapping—to convert the raw, lived experiences of Deaf patients into structured insights and design constraints. We learned that our AI didn't just need to translate words; it needed to translate distress, navigate cultural stigma, and bypass ableist medical standards.
To achieve this, our Diverse Disability User Committee exercised their epistemic authority to audit and build the dataset from the ground up.
Interactive Experience: The "Veto" Matrix
How the Deaf Community Rewrote the Medical Dictionary
To ensure our AI was culturally and ecologically valid, we conducted a "Lexicon Audit". We presented the cohort with standard, Western clinical terminology (from the WHO and CDC) often used in infectious disease screening. The committee was granted absolute "veto power" to reject or modify any terms that were culturally inappropriate, stigmatizing, or visually inaccurate in Ugandan Sign Language (USL).
Interactive Experience: Visual Triage & Pain Maps
Replacing the Ableist 1-10 Scale
One of the most dangerous moments in a Ugandan outpatient department is triage, where health workers fire rapid verbal questions and ask patients to rate their pain on a 1-10 verbal scale. This auditory monopoly completely excludes the Deaf community, leading clinicians to "guess" the severity of a patient's condition.
To fix this, the cohort co-designed a low-tech, high-impact clinical toolkit to bypass language and literacy barriers entirely: Visual Triage and Pain Maps.
The Clickable Human Body Map
Instead of a verbal 1-10 scale, patients point directly to a laminated body diagram to indicate the exact site and radiation of distress. Our AI is trained to track these specific pointing gestures in real-time, instantly converting them into structured triage data.
Danger Sign Icons
We replaced complex, text-heavy English medical posters with universal, culturally grounded graphics for "Danger Signs". The AI operates an "offline mode" specifically programmed to recognize these high-stakes signs instantly, even if the clinic's internet connection fails.
By the end of Phase 1, we didn't just have a list of words; we had a culturally validated, community-approved blueprint for the AI. We knew exactly what the machine needed to learn to keep patients safe.
But building it on paper wasn't enough. We had to prove it could survive the chaos of a real clinic.
Stress-Testing the Future (Speculative Prototyping)
In standard technology development, engineers build an AI model in a pristine laboratory, test it on curated datasets, and then deploy it into the real world, hoping it works. If it fails or causes harm, they try to patch it retroactively.
We refused to take that risk. In vulnerable healthcare settings, harm is easier to trigger, harder to detect, and much more expensive to recover from.
During Day 2 (The Co-Conception Phase), we didn't just build an AI; we actively tried to break the concept of it before a single line of expensive code was written. Using a methodology called Speculative Prototyping, we engaged our cohort in social simulations and role-play scripts to make abstract future risks tangible. Armed with physical "Scenario Cards" and low-fidelity mockups (like cardboard cameras), the Deaf students and interpreters tested the system against the messy, resource-constrained reality of a Ugandan public hospital.
By simulating these "edge cases," the community turned their lived fears into non-negotiable engineering rules.
Engaging the Edge Cases: How We Broke the AI
The Internet Drops (Infrastructure Collapse)
"You have just arrived at the clinic. You are bleeding heavily and need immediate triage. Suddenly, the hospital's Wi-Fi fails, and the network drops to a weak 2G signal. The human interpreter is delayed."
We asked a Deaf participant to simulate frantically signing "I am bleeding" to a mock cloud-based AI camera. The researchers intentionally enforced a 15-second "buffering" delay, mimicking a poor connection.
The delay was deemed clinically unacceptable and dangerous. The community established strict "latency budgets"—the maximum tolerable delay in seconds before an emergency translation becomes unsafe.
The AI cannot rely solely on the cloud. The developers were mandated to build a lightweight "Offline Mode". The system's edge device (the tablet or phone) must locally store and instantly recognize the visual "Danger Signs" (bleeding, fever, convulsions) without needing any internet connection, ensuring the most critical triage moments are never delayed by infrastructure collapse.
The Crowded Room (Hostile Gatekeeping & Visibility)
"The waiting room is packed with 50 other patients. You need to report a highly sensitive Sexual and Reproductive Health (SRH) issue, like an STI symptom. The nurse points to a large AI camera mounted on a tripod in the corner of the room."
Participants crowded into the room to simulate the lack of privacy. A Deaf student was asked to approach the mock tripod and sign their symptoms while the crowd simulated staring.
The participants immediately rejected the hardware setup. A large, visible camera creates a profound "stigma of visibility". It turns the Deaf patient into a public spectacle, causing surveillance anxiety and forcing them to "out" their disability and their sensitive medical needs to the entire room. We learned that if a tool stigmatizes a user, they will simply abandon it to avoid the mental stress and humiliation.
The committee exercised a complete UI/UX veto over the hardware form factor. They banned tripod-mounted cameras in public triage spaces. Instead, the engineering requirement shifted to private, handheld devices (like tablets equipped with physical privacy screens) or partitioned booths, ensuring that the patient's medical confidentiality and dignity are protected by design.
By actively stress-testing these adversarial futures, we closed the gap between "works in a lab" and "works safely in reality". We proved that technological safety is not just about translation accuracy; it is about anticipating infrastructure failures, power dynamics, and the psychological safety of the user.
But testing the AI is only half the battle. To truly protect the patients, we had to change the rules of the hospital itself.
The Technology (Building the AI Assistive Tool)
Powered by Lived Experience, Driven by Advanced AI
It is not enough to simply build a piece of technology; the technology must actively understand and adapt to the people it serves.
Many medical AI tools fail in the Global South because they are built using sterile, lab-based datasets that cannot handle the noisy, high-pressure reality of an actual public clinic. Our technology is different. By utilizing the community-validated data curated during our Discovery and Co-Conception phases, we built a highly specialized, bidirectional AI system designed explicitly for Ugandan healthcare.
To process the complexity of Ugandan Sign Language (USL) and cross-reference it with infectious disease protocols, our technical architecture relies on a Dual-Engine Approach.
The Dual-Engine Approach: How It Works
1. The Large Vision Model (LVM): Translating the Body
Reading more than just hands.
The first half of our engine is the Large Vision Model (LVM), which serves as the "eyes" of the AI. When a Deaf patient signs their symptoms into the clinic's tablet, the LVM captures the movement using advanced 3D pose estimation, tracking the spatial relationship between the hands, arms, and torso from multiple angles.
But translating sign language requires more than just hand gestures—it requires reading emotion. Crucially, our LVM is programmed to track facial expression markers and non-manual signals to recognize affect and pain expressions.
Because the community explicitly mandated this during co-design, the LVM understands when a patient is signing frantically due to severe physical agony (e.g., an ectopic pregnancy). Instead of outputting a flat text translation that a doctor might misread as "psychiatric agitation," the LVM correctly tags and translates the high-stress emotion, ensuring the doctor understands the acute physical emergency.
2. The Large Language Model (LLM): The Clinical Brain
Secure, localized medical prompting.
Once the LVM translates the physical signs into structured text, it feeds that data directly into the second half of our engine: the Large Language Model (LLM).
This is not a generic chatbot. Our conversational AI engine has been meticulously fine-tuned on domain-specific clinical datasets, including Uganda Ministry of Health guidelines, WHO protocols, and CDC screening workflows specifically for infectious diseases like malaria, tuberculosis (TB), and HIV/AIDS.
The LLM takes the translated symptoms (e.g., "burning urination for three days") and aligns them with these official medical protocols. It then securely prompts the attending clinician with structured diagnostic advice, suggested lab tests, or follow-up triage questions. The LLM can also generate a medically sound response or question, which is translated back to the patient, enabling a seamless, two-way diagnostic conversation without needing a human interpreter.
The Dataset Engine: Built for the Real World
Diversity by design, not by accident.
An AI is only as safe as the data it learns from. If an AI only learns from young, able-bodied actors signing slowly in a brightly lit studio, it will instantly crash when a frightened, elderly patient signs rapidly in a dimly lit, crowded clinic.
To prove this tool works in messy, real-world environments, our Dataset Engine was engineered for extreme diversity.
By training the dual-engine system on the actual lived reality of Ugandan patients, we built an AI that doesn't just work in theory—it works in the triage room.
But even the safest technology can be weaponized if the wrong people control the data.
Anticipatory Governance (Protecting the Data)
Technology is Dangerous Without Rules.
Building an advanced AI is only half the battle. In vulnerable healthcare settings, deploying technology without strict, community-led rules does not solve inequality—it merely digitizes it.
Historically, marginalized communities have been treated as passive data sources for extractive innovation, where privacy is an afterthought and "ethics" means fixing harms only after they occur. We reject this model. During Day 3 (The Adoption & Governance Phase) of our research, our Diverse Disability User Committee shifted from testing the AI to actively regulating it.
They instituted Anticipatory Governance—the practice of building robust safeguards, legal firewalls, and institutional readiness before a single piece of technology goes live in the clinic. The community established non-negotiable mandates that dictate exactly how the AI operates, who owns the data, and when the machine must be turned off completely.
The "Rules of the AI": Co-Defined by the Community
The "Kill Switch" (The Human Override)
Technology must know its limits.
There are specific clinical scenarios where relying on an algorithmic translation—no matter how accurate—violates fundamental bioethics, dignity, and patient safety. The committee established hard boundaries known as "Human Override" thresholds, identifying situations where the AI is legally forbidden from being the primary interface. In these scenarios, the AI automatically locks out, and a professional human interpreter is legally mandated:
- Terminal Diagnoses & High-Stakes Consent: Delivering news of a terminal illness or securing consent for major surgery via a tablet screen is a violation of compassionate care. Human "teach-back" validation is required.
- Gender-Based Violence (GBV) & Trauma: In cases of Intimate Partner Violence (IPV) or acute trauma, victims may use coded language or be accompanied by their abuser. The AI must never be the sole tool for these disclosures, ensuring nuanced human intervention and safety protocols are prioritized.
Data Sovereignty & Anti-Surveillance
Your body. Your data. Your rules.
For disabled and marginalized individuals, the introduction of cameras in public spaces brings a legitimate fear of surveillance, stigma, and forced "outing". The cohort audited the AI's data lifecycle to engineer a strict privacy-by-design framework.
- Edge Processing & Immediate Destruction: To prevent data commodification or cloud-based hacking, the AI utilizes a minimal-collection architecture. Biometric video data is processed locally on the clinic's "edge" device (the tablet itself) and is programmed to be immediately destroyed after the consultation.
- The Anti-Function Creep Mandate: The committee forced the administration to institute binding legal blocks against "function creep". This legally guarantees that the university, the state, or third-party vendors can never repurpose the AI camera feeds or biometric data for policing, behavioral profiling, or disciplinary actions.
The "Two-Sense Rule" (Fixing the Environment)
An accessible app is useless in an ableist hospital.
The cohort recognized that giving a Deaf patient an AI tool does not fix the underlying ableism of the hospital's infrastructure. Using their veto power, the committee forced Makerere University to adopt sweeping new procurement rules. Chief among these is the "Two-Sense Rule".
This institutional mandate ensures that the burden of communication shifts from the impaired body of the patient to the robust design of the building. Going forward, the hospital cannot rely on a single sensory modality for critical information:
- Queue Calls: The dangerous practice of a nurse shouting a name over a noisy waiting room is banned. Queue calls must now be delivered via an electronic display (visual), an audio announcement (auditory), and a physical numbered token (tactile).
- Emergency Infrastructure: Fire alarms and emergency sirens can no longer be exclusively auditory; procurement policies now mandate flashing strobes alongside sirens so Deaf patients are never left behind during a crisis.
By establishing these rules, our project proves that marginalized communities do not just need "access" to technology—they need the power to govern it.
Through Anticipatory Governance, we did more than deploy a software application; we re-engineered the institution itself, creating an environment where disabled futures are vibrant, autonomous, and undeniably secure.