My recent article, providing a guide to the current portfolio of providers of online property valuations models (see properazzi.co.nz), triggered the inevitable question “just how accurate are they?” So, I thought I would do some desk research. However, before I unleash a barrage of criticism stating that there are heaps of examples where the automated valuation models (AVMs) are so wide of the mark to make them laughable, let me simply say this. There’s over 1.5 million AVMs or potential AVMs for New Zealand properties – there will always be outliers and extremes. As I do not have time to review thousands or even hundreds of properties. I chose to select just 12 properties.

The method I have used, is to track the latest auction results (as published by the team at Interest.co.nz) when the auction year started after Christmas. I simply took the first 12 I saw which comprised eight properties in Auckland and four in Tauranga. So, again, I acknowledge that my sample is hardly representative nor truly random. It is made up of auction sales only, the sales are only for these two areas of the country, and represented a quiet period of the year.

With these 12 property sales results I went to each of the five providers:

• Homes.co.nz
• Trade Me Property Insights
• Realestate.co.nz Property Profile
• MyValocity (you do need to create an account to use the AVM)
• QV – mobile app for iOS and Android

I knew none of these providers had updated their valuations to take into account any of these 12 sales. Neither would the sale records have been picked up through local council sales or agent reporting so there was no bias of an AVM being influenced by these recent sales.

Another point to note is the analysis compared the sale price at auction to the mid-point of the price range of the AVM. So here is the table of results (top right). The colour code used is yellow where the AVM equalled the sale price exactly, blue signifies an AVM below the sale price with green where the AVM is above sale price. Finally, grey indicates that the provider had no AVM for the property.

‘The visual skew towards blue indicates that, based on this sample set, most AVMs were below sale price’

As you can see, the visual skew towards blue indicates that, based on this sample set, most AVMs were below sale price.

In the original version of this article I used an average variance measure, after receiving valuable feedback I have now used the calculation of gross median error.

All providers achieved a gross median error of less than 10%, with Realestate. co.nz achieving less than 5%, which is impressive. I would deduce that a factor in their accuracy is that they benefit from the very latest REINZ data, each month, of unconditional sales. Whilst all other providers rely largely on settled sales which come through at least a month to two months later.

Another perspective I was keen to examine in respect of the accuracy of AVMs, was the indicative range they provide to reflect the level of confidence. For each provider, for each property I assessed, the range is detailed as a percentage of the midpoint price.

This analysis (table 2) is very illuminating. The provider with the tightest range (in theory indicating confidence factor) is MyValocity, closely followed by Homes.co.nz, both just under 10%. This effectively meaning that their AVM range is 5% below the midpoint to 5% above which I would judge as fairly acceptable, given this is a computer based estimation with no detailed knowledge of the specifics of the property.

Of interest in this analysis is the very wide margin in the range from Trade Me Property, at close on 30%, with their tightest range being for a single property at just 19%. Similarly Realestate.co.nz seem to apply a standard c.21% to all AVM’s.