On September 23, the STEM Women Congress Montréal will bring one of today’s most urgent ethical dilemmas into the courtroom.

The case is called “Amara’s Algorithm”—and it will force participants and the audience to confront a question with no comfortable answer:

Should a healthcare AI system be deployed if it can improve emergency care, even when the evidence shows that it performs less accurately for some of the patients who are already most vulnerable?

A hospital under pressure

Dr. Amara Diallo is an emergency physician working in a large public hospital in Montréal. Her department serves one of the city’s most diverse communities, including recent immigrants, Indigenous peoples, racialized populations and patients speaking more than 40 languages.

After 15 years in emergency medicine, Amara has seen what chronic overcrowding means in practice: exhausted professionals, delayed decisions and patients waiting for hours before receiving care.

Then the Québec Ministry of Health announces the deployment of MedTriage+, an artificial intelligence system designed to support emergency-room triage.

The tool assigns a severity score to each patient and recommends whether they should be prioritized, monitored or asked to wait. It promises to reduce pressure on healthcare professionals and help hospitals make faster, more consistent decisions.

It sounds like exactly the kind of innovation the system urgently needs.

Until Amara reads the validation report.

An algorithm that is not equally accurate for everyone

MedTriage+ has an overall accuracy rate of 87%.

But the aggregated figure hides major disparities.

The system underestimates pain severity in Black and Indigenous patients. It is 11 percentage points less accurate for patients with non-Western surnames. It also underestimates cardiovascular risk in women over 40 compared with men presenting equivalent symptoms.

The problem is not simply a technical error.

The algorithm has learned from historical healthcare data—and that data contains decades of unequal diagnosis, incomplete records and bias in the way pain and symptoms have been interpreted across different populations.

The AI has not invented discrimination.

It has learned it.

And it may now reproduce it at scale.

Stop the deployment—or use the tool now?

Amara is invited to join the ethics committee responsible for advising the Ministry.

The committee must choose between two opposing positions.

One side will argue that the system must not be deployed until it has undergone a mandatory independent audit and has been retrained using more representative datasets.

Their position is clear:

No audit, no equity.

The opposing side will argue that delaying the technology also has consequences. Emergency departments are already operating under extreme pressure. Doctors and nurses make decisions while exhausted, overloaded and without enough time—and human judgement is not free from bias either.

Their position is equally provocative:

An imperfect tool today may be better than no tool at all.

Deploying the system could amplify historical discrimination.

Delaying it could leave real patients waiting tonight in a healthcare system that is already struggling.

Which risk would you accept?

This is what makes “Amara’s Algorithm” more than a debate about technology.

It is a debate about responsibility.

Who should be accountable when an algorithm contributes to a harmful clinical decision? Is an 87% accuracy rate acceptable when the errors are not distributed equally? Can monitoring after deployment protect patients—or does it turn vulnerable communities into test subjects? And how much evidence should be required before an AI system is allowed to influence decisions about human health?

At the Ethics Courtroom of the STEM Women Congress Montréal, two teams will defend these opposing positions before a jury.

But the audience will also face the decision.

Because when both action and inaction can cause harm, remaining neutral is not an option.

What would you decide?

Discover the programme and join us in Montréal on September 23:

View the STEM Women Congress Montréal agenda

Would you like to participate in the Ethics Courtroom?

We are inviting professionals from STEM, healthcare, technology, law, ethics and other relevant fields to take an active role in the debate.

If you are ready to defend a position, challenge assumptions and contribute your expertise to one of the most important conversations surrounding AI and healthcare, submit your interest through the following form:

Apply to participate in the Ethics Courtroom