The Limits of AI: When the intelligence is artificial, common sense is a superpower.

This article is featured in the 2024 edition of the Appraisal Buzz Magazine. Some other features in our magazine include funny Buzztoon comics, industry trends, as well as crazy stories from appraisers and readers like you! Read all these articles and more in the latest edition HERE. If you want to make sure you are receiving the print version of the Appraisal Buzz magazine in your mailbox, sign up HERE.

At some point, all of us will integrate AI tools into our business practices. Whether it automates mundane and repetitive tasks, generates narrative text, or assists with analyses, the power of AI to save time is real.

So, too, is the risk that results will be poor. Just ask the attorneys who had to explain to a Manhattan district judge why their brief cited six fictitious court cases. They hadn’t anticipated that ChatGPT would invent legal precedent out of whole cloth. But chatbot fabrications, known as “hallucinations,” are a common and thorny problem with generative AI tech.

The good news for valuation professionals is they have lots of experience spotting data points that don’t jibe with what they know. Another way to say this is that they have common sense. It’s a basic requirement for doing the work. Of course this 3-story overbuilt McMansion is probably not a comparable for a Cape Cod two miles away. That sale was under atypical conditions and, at minimum, needs adjustments to even be considered. You get the idea. 

Appraisers also need to deploy common sense when deciding whether and how to adopt AI tools into their businesses. Understanding a tool’s capabilities and flaws can help with these decisions. ChatGPT could help with neighborhood descriptions, but I’d better use a paid version built on current information. Grammarly can proof my reports, but will it try to change terms or incorporate prohibited words or phrases? Jasper can generate blog content for marketing, but what if it makes stuff up that’s just factually wrong?

Like any other technology that has entered the zeitgeist, AI will overpromise and underdeliver for a long time to come. Hype will outpace effective practical application because we users don’t know the precise bounds of work we can assign to AI tools without sacrificing quality, veracity, and credibility in our work product. But that doesn’t mean you can’t start adopting some aspects of AI today — you can, using well-honed common sense. 

When I consider how to adopt new tech, I go through this process:

1. Do I understand how this product works?

Much like using AVMs as a tool to support or cross-check assignment results, any conversation with AI must start by understanding what a tool can accomplish and, more importantly, what it cannot. Current AI products are, generally, not good at complex math. Asking ChatGPT to perform a regression analysis is beyond its current skill level. Don’t push the boundaries of what the tool can offer. Stay close to its core purpose, and then experiment with broader use cases.  

Lastly, have a passing knowledge of how the tool works — no one expects you to possess AI expertise, but being able to explain how a tool creates its outputs can help overcome hesitance to use the tools and show you’re not blindly trusting AI.

2. Do I trust what the AI is telling me?

Accepting AI outputs without any skepticism is a recipe for disaster. Approach AI like a detective interviewing witnesses: trust, but verify. Basic internet searches can quickly fact-check results — or raise enough red flags that you reject what is being offered. No state board will accept the argument, “But ChatGPT said,” and neither should you. 

3. Should I be using AI for this task?

Just because you can doesn’t mean you should — commit this phrase to memory. You will have to own everything in your report, and if too much of the product is driven by AI tools, you may be asked: “Well, what exactly did you do here?” Pride of authorship should be a driving ethos for when and how AI gets used. Delegating large swaths of your report elsewhere invites skepticism. Both the individual report and the macro profession need your services. 

Appraising requires a level of instinct, as in other professions where you must form an opinion, even as discretionary points that can be problematic from the perspective of fair housing or discrimination are limited or removed. Since you can’t delegate instinct to a chatbot, consider those factors critical to forming a credible opinion of value too important to hand off to AI tools.

One last point: Be sure your use of the tools aligns with USPAP’s requirements — a beneficial use case doesn’t obviate the need to follow USPAP. Expect ongoing guidance on this topic from the ASB as more real-world examples of use come into view.

Common sense is a superpower that can protect you from dire consequences as you experiment with AI. It’s tempting to be spellbound by new AI tools that seem miraculous — and to let down your guard of common sense. Instead, I recommend a heightened sense of caution: The tools are only as good as the people who craft them and the inputs provided by the users. AI hallucinations are still unpredictable, inevitable failure points, which means any “facts” and analyses it supplies should always be verified — it’s just common sense