INTRODUCTION
This article contains three sections – 1) A summary of statistical studies showing bias and variance in appraisals, 2) Exposing the lack of thinking and support in the appraisal trade, and 3) initial thoughts on how we eliminate the appraisal trade and move forward the Valuist© profession.
Recently, I was at a conference attended by appraisers, investors, lenders, and government agencies and one of the presenters said ‘We don’t think the appraised value is particularly important.’ The general public and real property participants have long held a similar sentiment about appraisals.
People arrive at their negative perception of the appraisal trade through personal encounters with poor quality reports or the media. In support of the general perception, several entities have published studies using actual data.
FNC STUDY
On July 12, 2012, Yanling Mayer of FNC posted the results of an analysis to examine how appraisal valuations respond to local market conditions. The study sampled purchase-mortgage appraisals on single-family homes and condos completed between January and June 2012. The number of appraisals sampled is not stated in the post, but is likely in thousands based on the volume of transactions handled by FNC. Results find ‘nearly 25% are appraised above contract by 3.0% or more. Combined with another 8-9% appraised at below contract by 3% or more, a third of the purchase-loan appraisals contain a market value opinion differing at least 3% in value from the contracts.’ In addition, the overall conclusion is ‘the overall distribution of appraised value relative to contract price shows significant upward bias in appraisals.’
The FNC analysis revealed 87.7% of appraised values were equal to or higher than the contract prices. This supports other studies we have heard about that found 90%-95% of appraised values were at the contract price or higher. Obviously, a true distribution not influenced by contract price would have a 50%/50% split above and below the contract price.
PLATINUM DATA SURVEY
In December, 2014, Bill King published ‘Purchase Price and Valuation Variance in Residential Real Estate.’ Mr. King analyzed two sets of data. First, he found a subdivision that had 21 homes of the same builder model – square footage, number of bedrooms and bathrooms, etc. were all the same and the homes initially sold within a 12-month period. The data included sale prices for these homes from 1993 to 2014. Mr. King observed price differences from 0% to 15% over that period and he concluded a variance of +10% was appropriate with all factors being held the same.
The second set of data studied 69,134 appraisal reports done for purchase transactions nationwide from the 4th quarter of 2013 through mid-October 2014. Mr. King summarizes his results as follows:
“In over 88% of appraisals, the appraised value is equal to or greater than the contract purchase price; in about 75% of cases the appraisal value disagrees with the contract sale price. So, out of the gate we have at least 75% of cases in which some error is present – either the buyer has agreed to a contract price that is not equal to value (assuming the appraisal is right), or the appraised value is not equal to market value (assuming the contract is right).”
YALE UNIVERSITY SCHOOL OF MANAGEMENT WORKING PAPER
In January, 1998, David Geltner and William Goetzmann published a working paper titled ‘Two Decades of Commercial Property Returns: A NCREIF Index Using Independent Appraisals.’ A main focus of the paper was to develop a repeated-measures regression (RMR) index that would be superior to the NPI published by NCREIF. However, a side result of the research led to this conclusion:
“The RMR procedure also allows the direct estimation of the average magnitude of the random component of individual property appraisal error in the NCREIF database. We estimate this individual property appraisal standard error at between 6.5 and 14.5 of the property value, most likely around 10 percent.”
CRE FINANCE WORLD – WINTER 2012
K.C. Conway, MAI and Brian Olasov analyzed 3,276 CMBS loans where the real property had been sold. The appraised value was compared with the gross proceeds from each sale (i.e. no deductions made for sales commissions and other costs). In 2,076 cases, the sale date and date of appraised value were within 12 months of each other.
In aggregate, the study found the sum of appraised values totaled $18.1 billion and the gross proceeds totaled $13.0 billion. This shows a strong upward bias with the average appraised value 39% higher than the underlying asset sale price.
For the subset of 2,076 cases having sale and value dates within 12 months of each other, 60% of the appraisals were within +25% of the sales price. Looking at this from a different perspective, 40% of the appraisals did not fall within 25% of the sales price. This variance is disappointing.
DIGITAL RISK STUDY
In surely the most comprehensive study ever performed, Digital Risk analyzed 725,000 loans by hand. At an average of 4 hours per loan, they invested over 3 million hours in this project.
They engaged several fee appraisers to appraise each property and they also obtained AVM data as available. Staff appraisers monitored the process and performed desk reviews. Appraisals were performed in 2011 and were all single unit residences.
Statistics showed that with a 95% confidence level (i.e. 2 standard deviations), appraised values had a +15% range. Another sample of 200,000 loans between 2010 and 2013 concluded 1 in 7 appraisals contained material error. That was defined as the appraisal value being over 20% higher than true economic value.
At a presentation of this study, the presenter concluded by saying “Current appraisal methodology is not sufficiently precise/robust for client’s problems. The issue is systemic.”
MANN’S DUE DILIGENCE EXPERIENCE
Since 1992, I have been part of over 40 due diligence/merger & acquisition projects. The projects have been coast-to-coast and involved portfolio valuations ranging from $100 Million to $7 Billion. Properties included both commercial and residential.
For all of the projects, appraisals were concluded to be 17% to 22% overvalued. In all cases, the primary component of the average error was an inferior appraisal report. A small part of the difference could be attributed to time between the appraisal and the due diligence project.
My data tracking looked for any significant differences among property types, geographic location, and small versus national firm performing the appraisal. No significant differences were noted for any of these factors.
THE WRITINGS OF EUGENE PASYMOWSKI, MAI
Mr. Pasymowski has five articles available for download at his website www.RealStat.com. Two articles most pertinent to this discussion are ‘Econometric Solutions for Real Estate Valuation’ and ‘How to Discredit Most Real Estate Appraisals in One Minute.” The latter title is sure to get your attention.
Instead of regurgitating his arguments, we will present some of his conclusions and statements below.
“Unfortunately, in most instances real estate appraisers make subjective, anecdotal, arbitrary, and unscientific “adjustments” to comparable sales market data without objective market-based support.”
“Regression analysis is superior with regard to the comparable sales data selection. The ‘traditional’ methods contain a high potential for data bias, because the appraiser often engages in the highly questionable practice of ‘data mining’ by selecting comparable sales to support a preconceived value conclusion. In contrast, regression relies on an unbiased random selection of comparable sales.”
“If fee appraisers use AVM technology, it can be their friend. If they ignore AVM technology, their appraisal services will become obsolete.”
SUMMARY OF STUDIES
The conclusions reached above provide some consistent observations:
- Appraised values are at best accurate to a range of +10% to 15% with one study suggesting it is as high as 25%;
- Sale prices have a minimum variance of +10%;
- Both commercial and residential appraisals have strong upward biases ranging from about 20% to 40%;
- Residential appraisals hit or exceed the contract price 88% to 95% of the time (thus, the industry is asking clients to pay a ‘customary and reasonable fee’ for an answer the client has already obtained for free!).
Individual studies always take criticism for what they did NOT consider i.e. this one was local, this one was residential only, this one compared price to price or price to value or value to value or whatever! The good part about the information above is it is coast-to-coast, covers both commercial and residential, analyzes over 30 years of data in aggregate, addresses the price and value issues in all different ways, etc. Naysayers can naysay, but the evidence is overwhelming and conclusive. Mathematically, the conclusion is to shut down the current appraisal industry and start all over.
THE NAKED TRUTH ABOUT APPRAISALS AND APPRAISAL REPORTS
99%+ of residential appraisals have no support for adjustments. Adjustments are supported only by peers’ adjustments and peer behavior was a contributing factor in the bubble effect in the early 2000’s. Many reviewers also force appraisers to use peer adjustments and to exclude market-based observation adjustments, thus reinforcing herd mentality rather than independent thinking. Under threat of removal of the appraiser (duress) the appraiser succumbs to peer pressure (bullying) and the appraisals all begin to chase prices rather than values.
95%+ of commercial appraisals have no support for adjustments. The same issues mentioned above also apply to commercial appraisals. Even when proof is available, there is no room to move outside of boundaries set by peers, regardless of property nuances and value.
Many appraisers reconcile to an average with only 3-6 sales, when most times there are 10 times that many RELEVANT data points that should be measured and considered. So, are appraisers really that magically talented? Or does peer pressure dictate how comparison grids are loaded and adjusted? Even if appraisers are that good, why then do we round our results. We are either that good, or not that good. Again peer pressure, which is advocated and brought upon by USPAP.
Many appraisers don’t verify sales. Many conveyances are not available for verification, while many are. Appraisers have become frustrated at the level of an untrusting and unresponsive set of market participants. But, real property is an investment with a level of privacy that is intended, so the verification process may or may not prove to be valuable. Most sales from a typical MLS system are fairly accurate, some are not. But, there are mathematical solutions to these problems that can wash out outliers and contain the central elements of value. Are we spinning our wheels in verification after confirmation? Maybe in some intended uses. In some uses, verification may be necessary. Such as government-related direct purchases where the large bulk buyer data is possibly baked into the analysis. In this case, participant verification is required by the clients usually and appraisers of these reports tend to provide this service for a commensurately larger fee for the time it takes to sort and purify the data set.
Many appraisers do not measure the subject improvements. Where was the information about the subject property procured? It may be okay to have several crosschecks, whereby a trainee physically measures and the supervisory appraiser confirms with Google or STDB. But, the owner should be asked, the public records researched and all should be combined to provide the read sources and crosschecks to explain the data used and how it was refined. Foul weather, conversations at the site visit, uneven terrain, landscaping, clutter around the structure, can cause outdoor physical obstacles impossible to accurately measure a large structure. Crosschecking and refining the data for the subject is imperative and several available methods have recently removed the excuse.
In general, the vast majority of commercial appraisals and almost all residential appraisals contain unsupported assumptions, unverified data, and biased (to the upside) values. No one can blame the public for having no faith in the industry when most of what it produces is not remotely reliable. Public Trust has long been violated by biased individuals.
FROM TRADE TO PROFESSION
Appraisers have been well indoctrinated to remain tradesmen, not to evolve past present minimal standards (we speak to the industry, not individuals in all cases) relying upon reputation of having special skills in this mystical investment math. When, in fact, the risk industry provided for the past 100 years much better, accurate and profitable results in insurances, investment performance and in business and public sector development; the general proof of this growth of our trade away from the public trust is the lack of mediation work asked of appraisers in court cases. Appraisers are seen by the public in court cases as expert witnesses (hired guns) to go in and argue different and beneficial results to one side of a court battle. Appraisers, if we are what we say we are, it would seem would be most paid for service to mediate a court battle if they were truly trusted by the public in this regard. With the recent housing debacle, there were very few cases settled by mediation where an appraiser was asked to preside. But there are now thousands of case settlements where dozens of appraisers are brought forth to testify on a single case and all arguing what is VALUE.
The insurance actuary profession sees just the opposite treatment due to its prowess of math and risk and unfettered fear of telling the truth. Actuaries are still compensated even if proven wrong by future unpredictable events. It is not to say that appraisers should take on an egghead persona which actuaries tend to project, but there should be a client care factor that allows for modern mathematics with understanding; instead of selling the client that we can produce wanted results and our math is better. Time to take off the Wizard’s Robes and begin to demonstrate, explain and defend with math of at least the 20th century and data sources of the new 21st century; what the market is providing in evidence. If the market evidence is weak, that should be proven. If there is strong evidence, that should be proven, also.
Weather forecasters cannot predict the climate, but the public trusts a 30 second weather report daily based on predictive analytics. The math isn’t provided because the public trusts the presenter. Behind the weather forecast is analytics that help the Navy, Merchant Mariners, Construction, Agricultural and Manufacturers make trillions of dollars of decisions each year in the US economy alone.
Appraisers are trusted less than the daily weather forecast in most cases. This is why review processes are embattled rather than constructive and court cases are contentious, rather than resolute, as with a coroner’s opinion. But, instead we are pounding each other out of existence. The claims that no one of us is correct or that if incorrect; banishment is the only solution for disagreement is preposterous. We look like crabs in a hamper dismembering one another (not that the legs won’t grow back) but the interminable suffering is not necessary. We must prove to the public trust that we know what it is that we are doing and how in OTHER tangent studies and financial arenas these methods have worked and evolved.
Math (even in simple form) is the four letter word that most people hate to engage in, but accept as truth in a final analysis when techniques that well cover a topic are brought forth. Take for example a routinely visible trait in other value studies; a rated medical study, whereby the participating researchers and performers of the study (are emboldened by their peers) to rate their results as meaningful and reliable or if it were designed so, a cursory study with little dependability but may be a worthwhile start to another follow up research study with greater detail, data points and measurement. So, in medicine, where the stakes are quite high, we see cross checks abundantly and self-review within the work itself which is a true advantage to the public trust! While for our purposes complex and proven hypotheses (statistical proofs, quantitative methods and actuarial techniques) are baseline in our work files. The explanation can still be provided by examples in a simple adjustment grid of comparables or graphical presentations that show the meaningful results, and then other review appraisers could quickly crosscheck the underlying work files mathematics for accuracy and reliability but withholding a judgment of correctness. The work file contains the provable math, while a report that is understandable for the client is produced. Data point purity and reliability is rated accordingly, with consideration given to whether the data is raw, confirmed or verified. A simple scale rating is used to account for the level of available development of the data point within the overall data set used in the report.
Ideally, a value data point might have at minimum important or telling aspects of value, cost, sustainable rents, conveyance information, land and building relationship ratios, acid tests of sustainable rents to cost and conveyance data, etc. These aspects are regressed properly (not singularly which is currently taught) in a well-developed multivariate manner with proof tests of fit, significance, and correlation. There is need for a distinguishing name for people who use higher math and methods to develop values.
The future Valuist© professional must give clients exactly what they need in regard to the reported and produced product. This does not mean giving up proprietary datum, methods, and developmental reasoning as the other professions hold as confidential (The NOAA does not provide its weather reporting to everyone in the same form, as it could at some level interfere with military strategy, its original purpose), but the public uses some of the weather results each day, non-confidential.
But, the future profession cannot have one set of standards or products that apply to all clients. The black Model T Ford as the only option is unacceptable. We must provide a wide range of makes and models with optional accessories.
This will not preclude clients from still requiring a certain report content. In fact, the goal is for this to occur. Valuists© would have significant leeway to meet clients’ needs. Form over substance must be replaced with substance over form.
Transparency is key to the future of valuation. It is time we make this more of a science so others can replicate the results within a reasonable range. The art argument has failed us for 80+ years.
An upcoming paper will detail our vision for the new profession of Valuists©. Until then, we leave you with some pertinent definitions that are still being discussed.
Valuation Analyst – A Valuation Analyst is competent in techniques and methods of extracting useful information from large and small amounts of numeric data relating to real or personal property. A Valuation Analyst would not necessarily be a Valuist©, as a Valuist© must exhibit competency in market theory, valuation analysis, and standards of practice when performing a service. (David Braun)
Valuist© – Noun [val-u-ist] A real or personal property valuation professional that excels in the use of quantitative methodology and/or econometrics to prove a property’s defined value; a person who estimates value, especially one skilled in real or personal property valuation. (David Braun, Bruce Cummings, George Mann, Eric Moskau)
Valuistics© – The application of statistics to the realm of valuation analysis methodology. (David Braun)
Bennu….
THE END
