Cutting Through the Agentic Hype Using CAIR
If the cost/risk of getting the wrong answer is high, agentic solutions are not the answer, for example, with (mis)understanding tax forms. The CAIR metric quantifies this, predicting the degree of customer adaptability (low/medium/high) of the AI product. Consider computing this metric when evaluating Agentic startups to help you gauge how well the team understands their customers, industry, and market demand. Here’s how it works.
Definition
Assaf Elovic and Harrison Chase developed the CAIR metric, which stands for “Confidence in AI Results,” to better understand the barriers to customer adoption of AI products [1]:
(Value of Success) / [ (Perceived Consequence of Error) x (Effort to Correct Error) ], where
Value: (low/medium/high) benefit to customers when an AI product succeeds
Consequence: (low/medium/high) cost when the AI product makes a mistake
Correction: (low/medium/high) effort needed to fix the AI product’s mistake(s)
High CAIR implies that customer adoption for the product will be high; low CAIR suggests that product adoption will be low, regardless of how shiny the AI/solution is [1].
Examples
The authors calculate this metric to explain why Cursor AI took off: High ÷ (Low × Low) = Very High, because [1]:
Value of Success: High, because it helps developers become more efficient.
Consequence of Error: Low, as the code is generated inside the Cursor’s platform (and not in a customer-facing environment).
Correction Effort: Low, as the user controls what code to keep before migrating it into a customer-facing environment.
I would caveat the authors’ calculation by assuming the developer generating the code is experienced enough to understand and debug the output and is using Cursor AI to develop a fairly standard app/implementation. Otherwise, the generated code will contain many hallucinations, and each debugging iteration will produce more and more code, making it harder to debug.
Assaf Elovic and Harrison Chase also calculate CAIR for the scenario of using LLMs to auto-fill tax forms. As you may remember from my prior Substack, “Magic” LLM Models, Real IRS Penalties, I was opposed to using LLMs out of the box for understanding tax forms. The CAIR metric shows why: High ÷ (High × High) = Very Low, where [1]:
Value of Success: High, because it helps customers (theoretically) become more efficient.
Consequence of Error: High, as there are real IRS (monetary) penalties for getting the dues wrong.
Correction Effort: High, as every entry needs to be checked to ensure the form is filled out correctly. And LLMs aren’t great at math or know tax preparation rules (yet) [1].
Questions to Ask in Diligence
