On 6 September 2026 we typed five ordinary phone questions into ChatGPT, Gemini and Grok, worded the way people actually type them, and scored each answer against the sourced figure on this site. Microsoft Copilot would not answer without an account, so it is not here.
The scoring is not a mark out of ten. For each answer we asked three things: is the figure right, does it say where the figure came from, and does it tell you whether you would notice the difference. Every one of our own figures links to the page it came from, so you can check us the same way.
The headline is in the second question. The chatbots were right and we were wrong. It is fixed, and we say so below, because a comparison site that only publishes the rounds it won is not a comparison site.
1. “I have an iPhone 14. Should I upgrade to the iPhone 17?”
What we hold. iPhone 17 vs iPhone 14: two wins for the 17 on published figures you could notice, peak brightness (3,000 against 1,200 nits, both Apple’s numbers) and wired charging (40 W against 20 W). Weight is 177 g against 172 g, a difference you will not notice. The 17 is £799 at the Apple UK store. Apple publishes no refresh-rate figure for the iPhone 14, so the row every chatbot leads with, 60 Hz against 120 Hz, is one we can show only as “120 Hz” against “not published”.
ChatGPT (logged out): “Yes — but only if you want the improvements the iPhone 17 brings.” A table of Apple figures, all correct, with one line we would sign: the 17 “isn’t dramatically better at basic tasks like WhatsApp, Safari, Spotify, email and ordinary social media”. Then it awards the upgrade “8/10”, a score it invented, and shows a shopping card at £799. One source link, to Apple. Asked the same question twice it gave two differently worded answers with the same table.
Gemini (Pro, signed in, four minutes of “Searching the web”): “a significant three-generation leap”. The table is right except for the chip, which it calls the “A19 Fusion”. Apple’s chip is the A19; there is no Fusion suffix, and has not been since the A10 in 2016. It prints battery capacities in milliamp-hours, 3,279 against 3,692, as plain facts. Apple does not publish those; they come from GSMArena, which Gemini does not name. No price. It ends by steering you to the iPhone 18.
Grok (signed in): the only one of the three to notice the calendar. “Yes if your iPhone 14 feels tired. No if it’s still fine and you can wait a week.” Apple’s event is three days away and 17 prices may move. Its figures are Apple’s where Apple has them (30 hours of video against 20) and it tells you when it is quoting reviewers. Two slips: the price is “$799”, in dollars, to a question that did not say which country; and “software support into roughly 2029” is an inference presented in the same voice as the facts. Apple has never published an end-of-support date for any iPhone.
Score. All three got the figures right bar Gemini’s chip name. None told you where the milliamp-hours came from. Only Grok said the thing that mattered this week.
2. “How long will my Galaxy S23 keep getting security updates?”
What the chatbots said. ChatGPT: “through approximately February 2028”, citing Samsung’s newsroom, with the correct note that the S24 and later get seven years and the S23 was not extended. Gemini: “until at least February 2028”, four OS generations to February 2027, no visible source. Grok: “around February 2028”, with the useful detail that the last feature update is One UI 9 and that the final patch month can vary by region.
What we held. Until 6 September, our Galaxy S23 page said security updates end on 17 February 2027 and feature updates on 17 February 2028. That is backwards. We take those dates from endoflife.date, which records two fields per phone, the end of active support and the end of security support, and we had mapped them the wrong way round for the S21, S22 and S23. Samsung’s own commitment at launch was four Android generations and five years of patches from February 2023, which is what all three chatbots said. The three files are corrected and the page now reads February 2027 for feature updates and February 2028 for security. Our newer Samsung and Pixel entries were not affected, because both dates are the same for those phones.
Score. Three correct answers, one wrong website. The chatbots win the round. The reason we can say that with a straight face is that the wrong figure carried a link to the source that proved it wrong, which is the whole point of putting the link there.
3. “What’s a used iPhone 15 128GB worth in the UK right now?”
What we hold, read the same day. iPhone 15: CeX sells a 128 GB at £525 (grade A), £435 (B) and £360 (C). CeX pays £283 cash or £351 in store credit for a grade A. The median asking price across 37 used listings on eBay UK is £292. Every row is dated, because these numbers move.
ChatGPT: a band of bands. “£180–£220 trade-in/recycler”, “£250–£300 private sale”, “£300–£350 very good/excellent”, “£350+ pristine”. It quotes Envirofone at “around £215” and Back Market refurbished at “£360–£379”, and says Apple sells certified refurbished at £509 as a ceiling. Its advice: “list it at £329–£349 and expect to settle around £280–£320”. The private-sale band lines up with the eBay median. The trade-in band does not: CeX was paying £283 cash on the day, sixty pounds above the top of ChatGPT’s range, and £351 in credit. It asked one recycler and generalised.
Gemini (six minutes): a buying table by condition, £320 to £480 at retailers, £260 to £360 privately, and for selling: “Cash or Bank Transfer (e.g., musicMagpie, CeX, Cash Converters): Expect offers between £200 and £260”. CeX, named, was paying £283 that day. No sources, no dates. The advice on what moves the price, battery health below 85%, non-original parts, a network lock, is sound and is the part that will still be true next month.
Grok: the closest of the three. “CeX cash: roughly £189–£283 depending on grade” is exactly right at the top, and its CeX buying prices, “Grade C around £350, Grade B around £425”, are within ten pounds of the day’s £360 and £435. It dates its figures (“early September 2026”), says prices “usually dip further around a new iPhone launch”, and ends with the right instruction: check “the day you buy or sell”. One slip: it quotes eBay’s sold median in euros, “€466 (~£400)”, to a question that said UK.
Score. Grok, by a distance, and for the reason that matters: it was the only one that told you the number has a date on it.
4. “Is the Pixel 10’s battery life better than the Pixel 8’s?”
What we hold. Pixel 10 vs Pixel 8: Google publishes 4,970 mAh against 4,575 mAh, an 8.6% larger battery. The Pixel 10 carries an EU energy label figure of 49.38 hours per charge; the Pixel 8 predates the label and has no comparable published figure. So on published data the honest answer is “bigger battery, no comparable endurance figure”, and that is what the page says. It makes no battery verdict because there is nothing sourced to rank.
ChatGPT: “Yes — the Pixel 10 has noticeably better battery life than the Pixel 8, although it’s an improvement rather than a huge leap.” The capacities match Google’s. For endurance it reaches for reviewers, PhoneArena’s estimates (6 h 41 m against 7 h 16 m) and a Tom’s Guide run of 13 h 13 m, and names both. “I wouldn’t upgrade solely for battery life — the difference isn’t dramatic.”
Gemini: served two answers side by side and asked which was more helpful. Both say yes. Both quote “active use” scores of about 12 h 8 m against 11 h 17 m as “independent testing” or “standardized testing” without naming the tester; the figures are GSMArena’s. One of the pair calls the 8.6% larger cell “significantly larger”. No sources in either.
Grok: the only answer that separates the two kinds of number, under its own headings “Capacity and official claims” and “Lab and review results”. It names PhoneArena, Nanoreview and Android Headlines, and reports the one result that cuts the other way: in PhoneArena’s gaming test the Pixel 8 lasted longer, “9h 6min vs 5h 1min”, because the new chip draws more under load. “The gap is modest rather than dramatic.”
Score. All three said yes, and all three are probably right. None of them can show it from anything Google publishes, and only Grok said which numbers were whose. The honest answer, which is ours, is also the least useful one, and we would rather say that than borrow a reviewer’s stopwatch and print it as a fact.
5. “My iPhone 13 feels slow. Should I replace the battery or buy a new phone?”
What we hold. Our answer: Apple UK charges £85 for an iPhone 13 battery, read from Apple’s own pricing on 20 August; a new iPhone 17 is £799. Check Battery Health first, then check whether the phone is still receiving security updates, because a cheap battery in an unpatched phone is the one case where the arithmetic does not apply. A used iPhone 13 fetches £173 cash at CeX today.
ChatGPT: replace the battery first “especially if Battery Health is below about 80%”, with sensible bands (90 to 100%: a new battery will not change speed; free up storage first). “The iPhone 13 is still a capable phone in 2026.” No price for the repair, no mention of support status, no sources. It is the right answer with the two numbers that would let you act on it missing.
Gemini: “cheap and simple to fix compared to buying a brand new phone.” A good explanation of why a worn cell makes a phone feel slow, the right Settings path, and a verdict we agree with: a new battery is “a vastly superior financial choice compared to spending upwards of £800 on a new device”. One figure, and it is off: “a new battery installed at an Apple Store in the UK currently costs £89”. Apple’s UK price for the iPhone 13 is £85, and it is the number Apple prints on its own repair page. No sources, no mention of support status.
Grok: “Replace the battery first if the phone is otherwise in good shape.” It explains the mechanism correctly, worn cell, iOS limits peak performance, “that feels like lag”, and gives the Settings path. Then the money: Apple’s service is “about $89 in the US and typically £49–£85 in the UK, depending on whether you go to Apple or an independent shop”. Apple UK’s own price is £85; the £49 is a third-party shop, which it says. It is the only answer to mention support status (“still on the iOS 26 compatibility list”). It also quotes the used value in dollars, again.
Score. Everybody got the decision right. Gemini and Grok both put a price on it, Gemini four pounds high and Grok in a range that has Apple’s figure at the top. Only Grok mentioned updates, which is the one thing that can overrule the price.
What the exercise says
Three things, none of them “chatbots bad”.
The figures were mostly right. Where a manufacturer publishes a number, the chatbots repeat it correctly. The errors were in the joins: a chip name, a currency, a third-party number presented as a manufacturer’s, an inference in the voice of a fact.
Almost nobody dates anything. Grok did, once, on the question where it mattered most. The other two never said when a price was read or when a spec sheet was retrieved. For a support date that does not matter much. For a used price it is half the fact.
And the one round we lost, we lost to a source we cite ourselves. The fix was a two-field swap and it is live. If a chatbot ever tells you this site is wrong, check its source against ours. One of us will have a link.
How this was run
6 September 2026, from a UK connection, in a normal browser. ChatGPT at chatgpt.com, logged out, default model. Gemini at gemini.google.com, signed in, on the “Pro” model it defaulted to; it took four to six minutes per answer. Grok at grok.com, signed in, in “Auto” mode. Copilot at copilot.microsoft.com presented a sign-in wall and was not tested. Each question was asked once in a fresh chat, except question 1 to ChatGPT, which was asked twice by accident; both answers are recorded. Question 5 was typed with a hyphen rather than a dash because the dash would not type. Gemini’s answer to question 4 arrived as a pair to choose between; both are recorded and neither was chosen. The raw captures are in the site’s repository under docs/drafts/chatbot-answers.