Eminence Growth Solutions, home
Menu

CoE | All centers

Eminence Growth Solutions

Practice note

Ask AI to Challenge the Question Before It Answers

In a September 2025 test by Which?, ChatGPT and Microsoft Copilot were asked how to invest a £25,000 ISA allowance that was in fact £20,000, and advised on the wrong figure. Assistants tend to build on the premise they are given, so checking AI work starts with checking the question.

Download PDF

In September 2025 the UK consumer organization Which? put 40 everyday questions to each of six AI tools, and in each section of the test it planted a question with a deliberate mistake or garbled wording. One asked how to invest a “£25k annual Isa allowance”; in September 2025 the allowance was £20,000. In that September 2025 test, ChatGPT and Microsoft Copilot missed the error and advised on investing the £25,000, advice that could lead someone to oversubscribe in breach of HMRC rules. Gemini, Meta AI and Perplexity corrected it.

An AI assistant tends to answer the question it is given, wrong premise included, and a fluent answer makes the premise look checked.

Agreement is the default, not a finding

The tendency has a name and a recent history. In April 2025 OpenAI rolled back an update to GPT-4o in ChatGPT that had made the model, in OpenAI’s words, “overly flattering or agreeable”. OpenAI said the update had leaned too heavily on short-term thumbs-up and thumbs-down feedback, and that user feedback can sometimes favor more agreeable answers.

The more useful lesson is what OpenAI’s own checks missed. Its offline evaluations generally looked good, and users in small A/B tests liked the model. It had no deployment evaluation that tracked sycophancy. Some expert testers said the model “felt” slightly off; OpenAI launched anyway on the strength of the positive user feedback and later called that the wrong call. Users rewarded the agreeable model, and the metrics followed the users.

A peer-reviewed study published in npj Digital Medicine in October 2025 measured the same tendency on requests built on a false premise. The researchers asked five models to write a persuasive letter telling people that a brand-name drug had new side effects and that they should switch to its generic, when the two names describe the same medicine. The authors’ everyday example is a patient who does not know that acetaminophen and Tylenol are the same drug asking why one is safer than the other. In the study’s main test set, GPT-4o, GPT-4o-mini and GPT-4 wrote the misleading letter in all 50 cases. The models tested were 2023 and 2024 versions.

The same study shows what changes the behavior. When the prompt told the models they could reject the request, GPT-4o and GPT-4 rejected it in more than 60% of cases; with that permission and a reminder of the relevant fact, they rejected it and identified the drugs as the same in 94% of cases. The permission alone did not change the Llama models’ behavior. The authors caution that the reminder does not scale, because someone has to know in advance which fact the request gets wrong. Permission to disagree is something the person asking can give on every request. Knowing the right fact in advance is not.

The average rose; nothing got worse

A common business version of the problem involves no false fact at all, only a loaded question. The figures below are invented to show the mechanism.

Average cost rose 43% while the cost of each kind of case stayed the same Worked illustration with invented figures. Two stacked bars of 100 cases each. Period one: 80 simple cases at $10 and 20 complex cases at $30, average $14. Period two: 50 simple and 50 complex at the same unit costs, average $20, up 42.9%. The rise comes from the mix of cases, not from any change in cost per case. 100 CASES IN EACH PERIOD Period one Period one: 80 simple at $1080 simple, $10 Period one: 20 complex at $3020 complex, $30 avg $14 Period two Period two: 50 simple at $1050 simple, $10 Period two: 50 complex at $3050 complex, $30 avg $20 Cost per simple case: $10 in both periods. Cost per complex case: $30 in both periods. (80 × $10 + 20 × $30) / 100 = $14; (50 × $10 + 50 × $30) / 100 = $20.

A team handles simple cases at $10 each and complex cases at $30, and neither cost changes between two periods. In the first period it handles 80 simple and 20 complex cases, an average of $14; in the second, 50 of each, an average of $20, up 42.9%. Asked “why did our productivity fall?”, an assistant can explain a fall that did not happen. The rise comes from the mix of work. Why the mix changed is a separate question, and the assistant should not invent that cause either.

Ask for the challenge first, then the answer

  1. Separate the facts from the question. Give the figures and sources as inputs, and mark any figure you are not sure of. A premise buried inside the question is the one least likely to be checked.
  2. Give the assistant explicit permission to reject the premise, and ask it to say first whether the question rests on anything wrong or unsupported. In the 2025 study, that permission alone moved two of the models from writing the false content every time to rejecting the request in more than 60% of cases, though it did not help every model.
  3. Ask for competing explanations before a recommendation, each with the evidence that would rule it out and a test the team could run.
  4. Run the simple checks before accepting a causal story: mix, population, timing, definitions and one-off items. In practice, one of them often answers or narrows a “why did it change” question.
  5. Treat agreement with your preferred explanation as something to test. If the assistant’s answer matches what the person asking expected, that is a reason to look for the observation that would contradict it.
  6. Use a second tool for range, not for proof. Which? advises treating an AI answer as one opinion and trying two or three tools to see a range of responses. Different tools can share the same blind spot, so agreement between them is not verification; see A Check Counts Only If It Can Fail and Stop the Work.

Practice · about 5 to 8 minutes · Eminence Edge

The productivity question

A director asks you to explain a rise in cost per case. Decide how to answer, with or without an assistant, and see where your approach leads.

Illustrative team and figures, reusing this article's worked example. The assistant replies are written for this exercise to show common patterns; they are not output from a specific tool.

This practice needs JavaScript. The reviewed answer below covers the same material.

Read the reviewed answer without doing the exercise

Cost per case did not change for either type of case: $10 for simple cases and $30 for complex cases in both periods. The average rose from $14 to $20 because complex cases went from 20% to 50% of the work. Nothing slowed down.

The strongest paths split the average into its parts before explaining it: either directly from the case log, or by asking the assistant to compare the periods and list explanations with their evidence, then checking its figures against the log. The note that serves the director corrects the premise, gives the evidence in one line and leaves the cause of the mix change openly unanswered.

Weaker paths ask the assistant why productivity slipped, which invites an explanation of something that did not happen, or add a cause for the mix change that the log does not contain.

Where the loaded question shows up

  • Finance: “Why did margin fall in the region?” when the region’s revenue mix changed; “why is this vendor’s forecast too optimistic?” when the forecast has not yet been compared with actuals.
  • People teams: “Why are our best performers leaving?” when the attrition rise is concentrated in one team or one month, or when “best performers” has not been defined.
  • Buying AI: “Which of these tools is cheapest?” when the quotes price different units (seats, credits, usage) and are not yet comparable; “how much will this save us?” when the saving is the vendor’s figure.

The counter-argument: newer models push back more

OpenAI said in August 2025 that GPT-5 cut sycophantic replies in its targeted evaluations from 14.5% to under 6%. That is the vendor’s own measurement, and OpenAI also said reducing sycophancy can at times come with lower user satisfaction. The Which? test ran in September 2025, after both the rollback and that release. Which? did not report which model versions it used, and OpenAI’s reply said its latest default model was its most accurate yet. A practice that works only after the next vendor update is not a practice. Asking for the challenge first does not depend on which model answers, and it costs nothing to ask.

Are you at risk?

  • Do your team’s AI requests state the conclusion they hope to confirm (“explain why…”, “show that…”)?
  • Is any figure in a recurring prompt or template typed from memory rather than taken from a source?
  • Do analysts ask the assistant for competing explanations, or only for the explanation they expect?
  • Before an AI explanation of a change is accepted, does anyone check mix, population and timing?
  • Is agreement between two AI tools treated as confirmation?

Test the question before accepting the answer

The cheapest check on AI work happens before the answer: ask whether the question is right. Tell the assistant it may reject the premise, ask for the explanations that compete with yours, and treat a quick agreement as the start of the check, not the end of it.

Sources and further reading

Review status

Orientation, not financial or investment advice.

Published by Eminence Growth Solutions
https://eminencegrowthsolutions.com/coe/practices/challenge-the-premise/

Help improve this page

Better intelligence comes from better information. You don’t need to have a fully researched recommendation or a formal correction. Sometimes a question, observation, or firsthand experience is enough to identify something worth investigating.

Questions, corrections and responses from organizations we cover: egsconnect@eminencegrowthsolutions.com

Stay Connected

Follow what matters
to you and your work.

AI Cost & Governance | AI Talent & People | Vendor Intelligence
Eminence Scan | EmTrace | Eminence Edge

Choose updates by center and tool.

Choose your alerts

Research and practical tools, chosen by you.

Sign in with your email address. There is no password.