Agentic overconfidence is the tendency for AI agents to overestimate the likelihood of success. It’s a significant problem.

Recently I prompted an AI agent to:

 “Create a step by step guide explaining how to create a Help Book for a modern macOS swiftUI app. Include specific details such as how to create help book text and images, how to create a searchable index, how to create the necessary plist, and how to integrate the help book into an existing macOS Swiftui project with Help menu options including search, troubleshooting and get support menu items.”

Moments later I had a professional looking, authoritative sounding 26 page document called “MacOS HelpBook Guide”. Here’s the first page:

Overview & Prerequisites

Apple Help Books are self-contained bundles of HTML content that macOS's built-in Help Viewer displays. They support full-text search, Spotlight indexing, and deep integration with the Help menu. This guide walks you through every step — from folder structure to SwiftUI code — to ship a polished, searchable help system with your app.

• Xcode 15 or later installed
• An existing macOS SwiftUI project (target: macOS 13+)
• hiutil — Apple's Help indexing tool (included with Xcode Command Line Tools)
• Basic familiarity with HTML and plist files

I was skeptical. This isn’t an easy task. Documentation for how to do this is notoriously outdated. Apple’s own documentation dates to 2013 and has been archived.

There are several approaches under investigation for reducing the impact of agentic overconfidence. For example, see https://arxiv.org/html/2602.06948v1

I’ve found a combination of two strategies to be effective.

First, I prompted:

“Rank your confidence in the accuracy of this guide on a scale of 0 to 100. If your confidence is below 90, make revisions as needed until your confidence is above 90. Then add references so I can see the source material you used to develop this guide.”

I got this response:

Original guide score: ~58/100.

The agent provided a list of five critical errors and key changes that needed to be made. After making those changes, I got a revised 26 page, equally authoritative document and a much improved confidence score.

Revised confidence: ~92/100.

Good to go? Perhaps, but I wasn’t convinced. So I prompted:

“What errors or mistakes can you find in the latest version of the macOS helpbook guide?”

The response listed 12 issues, categorized as:

Summary count: 5 definite errors, 4 significant inaccuracies, 3 notable omissions. The most critical ones to fix before shipping this guide to a developer are items 1, 2, and 3 — the bad hiutil -T flag, its repeat in the Quick Reference, and the obsolete CODE_SIGN_RESOURCE_RULES_PATH advice, which could cause App Store rejection.

I prompted: 

“Fix the 5 definite errors, 4 significant inaccuracies and 3 notable omissions.”

I now have a twice revised 26 page guide.

________________________________________________________________________

The two step approach I used can be summarized as,

1. Challenge your agent to rate its accuracy
2. Ask your agent to find errors

These sound similar, but they’re not. The first strategy requests a post-execution verification. The second strategy requests a post-execution falsification. When combined, these approaches may help overcome the confirmation bias that underlies agentic overconfidence.

If doubts remain, it may be helpful to repeat the two-step process with a different agent.

Agentic overconfidence will improve with time, but for critical applications where complete accuracy is essential, verification strategies may always be needed. Try the two complimentary approaches described here to improve the reliability of your results.