AI-assisted, not AI-powered: what we will never do

AI in healthcare raises one important question: who decides? At Medigize the answer is built into the foundation. The model suggests, the system checks, the clinician decides.

In short:

• Four rules come before any feature: no diagnosis; when in doubt, urgency goes up; a validation gate before any output; a review trail at every stage.
• Over-caution has a cost, but the two errors are not equal: arriving earlier than needed is far better than arriving late.
• These are design principles, not results. We quote no accuracy figures; any claim about impact will follow measurement.

Four rules, set before any feature.

What does AI-assisted mean?

AI-assisted means the AI helps the people who make the decision; it does not make the decision itself. At Medigize, the model organises the complaint and suggests, a separate system checks every output before it reaches anyone, and a clinician makes every medical decision. “AI-powered” suggests the opposite, that the AI runs the decision, which is why we do not use it. The difference matters most on a bad day: when something goes wrong, “who decided?” should have a person as its answer.

Three words are easy to mix up, and the rest of this article depends on keeping them apart:

• Diagnosis is naming what a person has. It needs a clinician who can examine them, and it is never the tool’s to give.
• Urgency is how soon someone needs care. A tool can assess it from what a person says, and when it is in doubt, it leans towards more urgency, not less.
• Routing is pointing to the right level of care, from an emergency department to a clinic visit. It says where to go, not what is wrong.

Four rules we do not negotiate

1. No diagnosis. The tool understands the complaint, assesses urgency and routes to the right level of care. Diagnosis belongs to the clinician.
2. When in doubt, urgency goes up and never down. That can make the system over-cautious at times, but the two errors are not equal: a patient who went earlier than needed is far better off than one who arrived late.
3. A mandatory validation gate before any output. Nothing reaches the patient without passing a checking layer.
4. A review trail at every stage. Each stage leaves a trail, so what happened can be looked at again.

Each rule leaves room for something useful, and shuts out something specific:

Rule	What it allows	What it rules out
No diagnosis	Understanding the complaint, assessing urgency, pointing to a level of care, preparing the case	Any screen that tells a patient which condition they have
When in doubt, urgency goes up	Pointing to more care when the answers leave room for doubt	Lowering a case’s urgency because it “seems minor”
A validation gate	Outputs that have passed a check that is not the model	Anything reaching a patient unchecked
A review trail at every stage	Looking back at what happened, so it can be corrected	Steps that nobody can look at again

We first set them out as limits in three things we won’t build. They come before features because a rule added after a feature tends to bend around it.

Why the two errors are not equal

Why, when in doubt, urgency goes up.

Any tool that sorts urgency will sometimes be wrong. The design question is which way it should lean when it is unsure. A patient sent to care earlier than needed loses some time, and leaves with a clinician’s answer. A patient told to wait when they should not have may lose far more. So when the tool is in doubt, it leans towards care.

Emergency care has names for the two errors. Sending someone to more care than they turned out to need is over-triage; sending them to less care than they needed is under-triage. The US national guideline for the field triage of injured patients, which helps ambulance crews choose a hospital, describes the two as inversely related: push one down and the other tends to rise. It also says trauma systems have chosen to minimise under-triage and accept more over-triage, because under-triage is associated with higher mortality.

A navigation tool is not an ambulance crew, and we did not borrow its rules. The reasoning is the same, though: a system that cannot be sure has to choose which mistake it would rather make, and it should make that choice on purpose, in advance and in public. Doubt here is not only a borderline answer. It also covers a message the tool is not sure it understood.

What that means for a patient using a navigation tool is in can an app tell you if it’s an emergency?

Why this matters

Preventable errors in care often happen at handovers between parties: department to department, clinician to clinician. Patient safety organizations focus on the same point. So we build tools that structure the handover, not tools that replace clinical judgment.

The World Health Organization counts breakdowns in communication between providers among the causes of avoidable harm, and the Patient Safety Movement Foundation collects the stories of the people that harm reached. We followed one journey full of handovers in the hidden cost of a fragmented patient journey.

Medicines show how much a handover can drop. WHO’s 2019 report on medication safety in transitions of care says medication discrepancies affect almost every patient who moves across a transition of care, such as admission to or discharge from hospital. The same pattern runs through the seven stops of one composite journey.

For a clinician, “structure the handover” has a practical meaning: the case arrives organised, with what was asked and answered, so the visit starts from information rather than a blank page. What it means stays the clinician’s judgment. A tool that tried to take that judgment would only add a new handover, from the machine to the doctor, carrying a conclusion the doctor did not reach.

For a patient or a family, every handover is a moment to ask two plain questions: who has my information now, and what is the next step?

Where global health guidance draws the same line

We did not invent the idea that a person decides. The World Health Organization’s first principle for AI in health is that humans should remain in control of health-care systems and medical decisions. Its 2024 guidance on generative AI warns about “automation bias”: errors overlooked that would otherwise have been caught, and difficult choices handed to the machine.

We cite this as context for the line we draw, not as a claim about our own compliance with any guidance.

Would a non-diagnostic tool say this?

Six things an AI health product might say about itself. Decide which a non-diagnostic tool could honestly say:

How to read an AI health claim

Whether you are a patient choosing an app or a clinic choosing a tool, the wording on the box tells you a lot. A few common phrases, and the question each one should prompt:

• “Diagnoses”, “detects your condition”, “tells you what you have”. Who is responsible for that conclusion? If the answer is the software, a clinician is missing from the decision.
• A percentage with no method. Measured on what, against what, by whom, and for which people? A number without how it was measured is not a claim to rely on.
• “No doctor needed”. Treat it as a warning. A tool that understands a complaint can prepare a visit; it cannot replace the examination.
• “Instant answers”. What checks the answer before you see it? Speed is not a check.
• “AI-powered”. Who decides? If the product cannot clearly say “a clinician”, ask again.
• “Clinically validated”. By whom, for which task, and where? WHO warns that systems trained mainly on data from high-income countries may not perform well in low- and middle-income settings.

And whatever an app says, severe chest pain, trouble breathing, fainting or signs of a stroke need an ambulance or the nearest emergency department. Do not wait for an app.

An honest limit

These are design principles we commit to and build around. We do not claim they have prevented any error, and we do not quote accuracy figures. Any claim about impact will follow measurement.

There is a reason beyond modesty. An accuracy figure from a test describes a model on a test, not a service with real patients, real dialects and real workflows.

So the order is fixed. First, a starting point, measured before anything changes. Then the same measures, taken the same way, once the tool is in use. Only then a claim, with its method beside it. We set out that order in why we build in public.

The measures themselves are ones any provider can take today, with no new system: how complete a case is when a clinician first opens it, how many manual handoffs it goes through, how long a patient waits for the next step, and how often follow-up disappears. They are described in what care operations means.

Why “AI-assisted” and not “AI-powered”

“Powered” suggests AI runs the decision. We say AI-assisted: it assists, and the clinician always leads.

Who does what, in one line.

A word looks like a small thing, but words shape how people use a tool. A label that says the AI runs the decision invites the automation bias WHO describes: a patient who stops asking questions, or a busy clinician who stops checking. “Assisted” says where the responsibility sits before anyone has to ask.

It also works as a test for us. Every feature has to fit one sentence: the model suggests, the system checks, the clinician decides. The part the model plays, and everything built around it, is in the model is not the product.

Frequently asked questions

What does AI-assisted mean?

The AI organises and suggests, a system checks its output, and a clinician makes the medical decision.

What is the difference between AI-assisted and AI-powered?

AI-assisted means the AI helps a person who decides. AI-powered suggests the AI runs the decision itself, which is why Medigize does not use the term.

Does Medigize’s AI diagnose?

No. The tool understands the complaint, assesses urgency and routes to the right level of care. Diagnosis belongs to the clinician.

Can AI replace a doctor?

Not in anything we build. The model suggests and the clinician decides; WHO’s first principle for AI in health is that humans remain in control of medical decisions.

Why would a tool be deliberately over-cautious?

Because the two possible errors are not equal. Going to care earlier than needed costs far less than arriving late.

Does Medigize publish accuracy figures?

No. We do not quote accuracy figures. Any claim about impact will follow measurement.

What is a review trail?

A record of what happened at each stage, so it can be looked at again.

Can an AI health app replace an emergency call?

No. For severe chest pain, trouble breathing, fainting or signs of a stroke, call an ambulance or go to the nearest emergency department.

Medigize is a non-diagnostic navigation tool. This article is general information, not medical advice. In an emergency, call an ambulance or go to the nearest emergency department.

References

1. World Health Organization: Patient safety (fact sheet)
2. Patient Safety Movement Foundation: Patient stories
3. World Health Organization: Six guiding principles for the design and use of AI in health (2021)
4. World Health Organization: WHO releases AI ethics and governance guidance for large multi-modal models (2024)
5. World Health Organization: Medication safety in transitions of care (2019)
6. Newgard CD and others: National guideline for the field triage of injured patients, recommendations of the National Expert Panel on Field Triage, 2021 (Journal of Trauma and Acute Care Surgery, 2022)