Financial advisers are charged with the responsibility of acting in the best interests of their clients. To fulfill their fiduciary duty, advisers must make suitable investment recommendations in accordance with clients’ investment profiles. Typically, advisers use surveys to assess a client’s financial risk tolerance (FRT), a critical component of profiling clients.

FRT scores may vary over time in response to changing demographic or economic events or because an individual is inconsistent in her survey responses. The authors argue that when investment advice is based on survey responses, it is incumbent on advisers to consider the extent to which such surveys provide reliable FRT scores. Understanding the degree of reliability in FRT scores pertains not only to the traditional adviser/client relationship but also to robo-advice services that rely heavily on automated client data collection.

The authors examine whether an association exists between clients’ variability of responses in FRT survey items and clients’ FRT scores over time and whether this inter-item response variation (IRV) could be used as an indicator of the reliability of a client’s FRT scores.

The authors use data from an established, reliable proprietary psychometric test instrument designed to proxy the FRT trait based on a 25-item survey. The scale has been used globally by investors, advisers, financial institutions, and researchers. The authors use survey responses obtained from several thousand clients of advisory firms located in the United States, United Kingdom, and Australia/New Zealand over the period January 2001 through July 2009. They control for the changes in economic circumstances arising from the global financial crisis. The authors also use demographic and socioeconomic information.

A total of 3,110 participants provide at least two completed FRT surveys each. The authors calculate each client’s IRV score as the standard deviation of the scaled response scores for each client’s FRT survey at each point in time. Client IRV scores are sorted into quintiles ranging from low (Quintile 1) to high (Quintile 5). The authors measure IRV and FRT scores within and between surveys.

The authors find a high correlation between IRV scores and FRT stability over time. Individuals with lower IRV scores tend to have follow-up FRT scores that correspond more closely with initial FRT scores and are thus more predictable. Similarly, individuals with high IRV scores tend to have more unreliable FRT scores with more variation over time. The more unreliable the FRT score, the greater the changes in the scores over time. IRV scores overall tend to be lower for the second surveys, suggesting a “learning effect” between survey completions.

The authors note a marked difference in FRT scores between Quintiles 4 and 5. The disparity in FRT scores for the clients in Quintile 5 are found to be significantly different from those in other quintiles. The disparity in successive FRT scores among the high IRV group is not readily explained by characteristics that commonly influence FRT, such as demographic or economic changes.

The authors suggest that advisers can better exercise their fiduciary duty by evaluating the variability in a client’s FRT survey responses, thereby identifying clients at risk for unreliable FRT scores. As a simple guideline, the authors recommend resurveying clients who fall into the top quintile of variability—those whose standard deviation of component scores is in the top 20% of all clients—with the caveat that professional judgment should still be exercised. The authors also suggest that survey providers publish statistics from their own databases to provide norms advisers can use to more readily calibrate client responses.