4th Annual Internet MLS Customer Satisfaction Survey

The report is also available for download in Adobe Acrobat format (PDF).
You must have Adobe's Free Acrobat Reader software to view the document.

Introduction

In January through March of 2005, Clareity Consulting conducted its fourth annual survey of America’s leading Multiple Listing Services. The purpose of the study was to learn more about MLS customer satisfaction, system performance, and vendor responsiveness. This survey was designed as a follow up to Clareity’s previous Internet MLS Customer Satisfaction surveys and summarizes how well MLS Executives rated their vendor’s performance during 2004. This year, a record 169 MLSs participated, representing 759,971 subscribers. Several MLS vendors encouraged their customers to respond to this year’s survey and we thank those vendors for their cooperation. Clareity also thanks each of the MLSs that participated.

Executive Summary

The following MLS software vendors had enough responses where they are the primary MLS vendor to be included in the primary vendor survey results.

* Stratus has two large MLS customers where it is the primary vendor and both responded, so Stratus has been included in this year’s survey.

** Technology Concepts (average MLS response size = 373) and FBS (average MLS response size = 548) were included in the head-to-head comparisons with the larger vendors; however it should be noted that larger MLS customers generally require more customization and a higher level of service. FBS has been installed in two larger MLSs.

The following MLS vendors received only one response each as primary MLS vendors, and so are not included in the parts of the survey pertaining to primary MLS vendors: ARIS, eProperty Data, Quest Technologies, and Tarasoft. The following MLS vendors received no responses: Advanced Marketing Services, Coherent, dynaConnections, Hillside Software, RealGo, Superlative, and Systems Engineering Inc. (SEI). FBS originally only had one customer respond when the survey results were first analyzed, but several FBS customers requested the opportunity to participate, so Clareity has updated the report to include FBS and its additional customer responses. This updated version of the report also includes one additional larger Rapattoni MLS customer that requested to be included.

As one can see in the chart above, some vendors serve primarily large markets, some only serve small and medium markets, and some service a wide variety of market sizes. Certain vendors, such as Stratus and MarketLinx provide custom programming and special services for their customers, and some vendors do not. There are many factors to consider in selecting a MLS vendor. Every year Clareity leads a number of MLSs though a structured selection process, involving the:

• determination of member needs via surveys and focus groups,
• a request for proposal (RFP) and competitive bid,
• an analysis of the RFP where vendor capabilities are measured against member needs,
• system demonstrations,
• hands-on testing,
• and other processes designed to ensure the best system and vendor selection is made by each MLS.

Clareity’s MLS satisfaction survey is only one means of evaluating a vendor or the capabilities of their system, and that said, we hope you find this report valuable.

Customer Satisfaction

Always of the most interest to our survey readers, the following chart represents ‘end user satisfaction today’:

* MLS Owned responses are displayed separately from vendors throughout this report.

While some individual MLS accounts experienced end user satisfaction issues, all of the vendors scored well this year. EVERY MLS vendor received a total of 85%   ‘Good’ or ‘Excellent’ by its customers in this overall rating. This is an improvement from the previous year, where that figure was only 75%  . Also, for the first year, no vendor received a ‘Poor’ end user rating, so it is a pleasure to report that the vendors as a group are improving.

Three years ago when Clareity first performed this survey, the group of “new” MLS companies, led by MarketLinx, Stratus, and Rapattoni, outscored the established MLS vendors at implementation, end-user satisfaction, and vendor responsiveness. Last year, they were joined by FBS as a customer service leader.

This year:

• Stratus maintained industry-leading service of its two large-scale MLS customers
• FBS also maintained its above-average customer service record
• Technology Concepts, a vendor that primarily services smaller (sub-1000) member MLSs, made a strong initial showing in the survey.
• Solid Earth, Offutt Systems, Rapattoni, and Interealty followed in a very close grouping. Solid Earth and Offutt generally service small and medium size MLSs, while Rapattoni and Interealty have proven they can service MLSs of various sizes.

The following chart represents the trend in ‘end user satisfaction today’. Of note is the dramatic increase in satisfaction with Fidelity MLS, the substantial increase for Offutt Systems, as well as some disillusionment with MLSs owning and operating their own MLS and the decreases seen for some other MLS vendors.

Numeric score calculation:((3 x Excellent %  ) + Good %   - Fair %   - (3 x Poor %  ))

Clareity asked MLSs whether they measure satisfaction via surveys, and found that 44%   do so. Clareity asked this question to determine if the use of member surveys would correlate to how MLS executives answered the end-user satisfaction related question. The following cross-tab analysis shows that similar percentages of MLSs rated overall end-user satisfaction along a spectrum of Excellent to Poor, regardless of whether surveys were performed or not - so it appears most MLS executives have a high level understanding of their end-user’s satisfaction.

Surveys are an important part of keeping MLS leadership in alignment with subscriber needs and satisfaction levels. Clareity performs a number of surveys for MLSs every year, creating benchmarks for various aspects of MLS service and regularly measuring against that benchmark so an MLS can quantitatively evaluate and monitor its own performance and customer satisfaction of its various services.

When asked whether the MLS was hosted by the MLS vendor or locally hosted, 82%   responded that the system was hosted by the vendor and the remaining 18%   hosted the system locally. Clareity analyzed these results to determine if there was any correlation between hosting and end-user overall satisfaction and found there was no significant relationship.

There was, however, a relationship between hosting and satisfaction with system performance and uptime, where there was an eight point increase in the ‘Excellent’ rating when the system was locally hosted and the only ‘Acceptable’ rating was given to a vendor-hosted system.

Also, as seen in the chart below, there was clearly a relationship between hosting and system response time during peak periods, with the strong advantage again going to locally hosted systems:

76%   of locally hosted systems were rated as having ‘Excellent’ system response time during peak periods, compared with only 60%   of vendor-hosted systems. 7%   rated the response time of locally hosted systems ‘Acceptable’, while vendor-hosted systems were rated almost twice that percentage ‘Acceptable’ and ‘Poor’.

The following chart represents ‘Overall staff satisfaction’ with the systems:

As in previous years, in the chart above we see some differences from the end user satisfaction ratings – but the systems that satisfied the end users the most were given the best ratings by MLS staff, too. There are other aspects of the system and service that affect the staff and end-user ratings and those will be illustrated in the sections below.

Both overall ratings tied closely to how respondents answered the question, “If you could do it over again, would you select this vendor again?” the results of which are illustrated in the chart below:

Last year, when this question was asked, 30%   of FNIS (now Fidelity MLS) customers answered the question negatively, and this year that percentage is reduced to 9%   - a definite improvement.

There was only one case where the previous question was answered negatively, but the question, “If you could do it over again, would you select this Internet MLS system again?” was answered positively. In that case, the vendor experience is positive, but the respondent seems to desire one of the other systems offered by the vendor and Clareity assumes they may elect to migrate to another system rather than to another vendor.

System Features, Performance and Reliability

As MLSs continue to expand in functionality, maintaining a simple, easy-to-use user interface can be difficult. Ease of use has an impact on the amount of training and support a system requires, and is an important component of end user satisfaction. Systems were rated on ‘User interface (i.e. ease of use, minimal keystrokes to accomplish tasks)’ as follows:

Generally, the MLSs are satisfied with the user interface of the system that they selected, but Clareity has been called on during the past year to help identify and resolve usability issues, as spurred by end user complaints to the leadership of regional MLSs.

The following chart illustrates ‘Response times during peak periods’. Some of the systems were well rated, but over the past few years, several systems are receiving lower ratings in this area. Either customer’s expectations have grown greater, or the systems are not performing as well. Clareity suggests that the vendors may wish to track system performance during peak times and make improvements or, if applicable, gauge expectations and figure out a way to reverse this customer perception.

All vendors were rated exceptionally well for ‘Reliability and system up-time’. The trend in this area has continued to improve over the past few years, and this year all vendors scored 96%   or higher as 'Excellent' and 'Good'.

System Features

The MLS vendors all work hard to make core features such as Search and Listing Maintenance robust and conform to customer needs, but there are a number of other functional areas where they differentiate. The following are the ratings for each of these features:

Public records usability and integration

Mapping

Reports (CMAs, Statistics, and other print reports)

IDX Solution - Implementation and Data Conversion

IDX Solution – Overall Satisfaction

RETS

Stratus was not rated by its customers on RETS. It is interesting to note that the MLS Owned and Operated systems that responded are apparently not having an easy time with RETS.

Clareity asked respondents to rate their MLS vendor on the “Pace of software development and enhancements,” and the ratings were as follows:

When asked if their MLS vendor “Keeps pace with evolving technology (e.g. RETS, Mobile Devices)” the ratings were as follows:

Clareity wonders if Marketlinx was penalized in the previous two ratings for the time taken to develop its new 4.0 release, during which time it may have decreased the development pace of its previous version. Likewise, Fidelity MLS has focused much of its resources on its flagship Paragon product. Otherwise, vendors were rated well with regard to their development pace. The MLSs that own and operate their own systems rated themselves well in terms of keeping pace, but this is inconsistent with some of their other self-ratings, especially regarding RETS.

If a vendor does not provide all the features required by an MLS, sometimes the MLS core functionality can be supplemented by using 3^rd party software and information providers. The following chart illustrates how vendors were ranked on their ability and willingness to “Cooperate with other information and technology vendors”.

Vendor Responsiveness and Customer Service

The following chart illustrates how vendors were rated on “responsiveness to system change requests”.

The ratings above are similar to, but distinct from, the respondents rating of ‘Vendor's willingness to customize the system to your local needs’, which has more to do with initial customization of the system to meet specific local market needs. It can be difficult to make a transition to a new MLS system if one is unable to specify distinctive market requirements and have them implemented. Vendors rated as follows:

One can also see a clear correlation between the customization and staff satisfaction. 40%   of those respondents rating a vendor as ‘Poor’ on customization rated their staff satisfaction as ‘Acceptable’ and the remaining 60%   as ‘Poor’.

MLS vendors were rated on end-user support as follows:

MLS vendors were rated on staff support. Offutt and Fidelity MLS showed improvement over previous years, while some other vendors did not rate as well as in the past. However, it is important to note that every vendor received 85%   or higher combined ‘Excellent’ and ‘Good’ ratings.

The respondents also rated the vendors on ‘Timely and comprehensive communications regarding system and support issues’.