The Multi-Rater Goes Super Qualitative – With Major Impact for Senior Leaders

Senior Leader Assessments

Multi-rater, multi-source, 360 degree feedback – whatever you call them, these surveys that capture feedback on a leader's behaviours have been around for 30 years. They have really taken off given the ability to now easily administer them online. But a new multi-rater process is emerging – what at HFL we call a Qualitative 360 or Qual360 for short.  The Qual360 is particularly effective in providing deeper and more effective feedback at critical employee levels – general managers and above, or high potential mid-level talent.


We asked HFL Principal Consultant Maria Leske to explain the process and application of Qual360s.


Firstly, Maria, can you explain how a Qual360 works?


With a typical multi-rater survey, a larger group of respondents (maybe 15) are sent an online survey in which they score the subject from 1 to 5 against a pre-defined scale. Most good surveys these days provide an opportunity for the raters to add context to the score, but in our experience – and HFL has executed thousands of these surveys – the raters don’t add much context.


The Qual360 executes a similar style questionnaire, but live, face to face. Typically we interview fewer people (perhaps between 6 and 8), but spend an hour with each of them, asking them to add lots of meat to the bones of their score. As with most qualitative research, the Qual360s are creative enquiry.


OK. So who conducts the interviews, and are people willing to provide their time?


The Qual360 is structured and deployed in much the same way as a normal 360 – the subject is asking for feedback from those who they work with most (direct reports, key peers, their manager) in order to be able to execute their role more effectively. In our experience, raters are more than happy to provide the time, particularly since the interview is always conducted by an external consultant, anonymity is guaranteed, and the extent to which the subject is going to get real feedback is usually admired by the raters.


The external neutral consultant seems to work very well in these situations, and is critical at the feedback stage.


A Qual360 is significantly more expensive that a typical online 360 survey. There must be some serious advantages of this methodology ...


The purpose of any feedback mechanism is to provide a vivid picture of how that leader is showing up to the people they interact with. What is that impact? What is positive? What is maybe negative?  What needs to change and why, and what will it take? Qualitative insights, as opposed to a long debate about the metric of a 3.6 score, clearly have way more impact, and at a senior level, where so much more is on the line, we believe – as do our clients who are using the methodology – that it is very much worth the investment.


In fact recently, after we executed a Qual360 for a CFO, and after the feedback, that CFO ordered Qual360s for all four of his direct reports. That’s how effective and useful he felt the feedback was.


So what makes it worth the extra investment?


I think there are three major things that make it worth the money.


Firstly, the feedback is context specific – it is in the form of examples or stories from raters, so it can be more specific in illustrating what the raters have observed, making it easier for leaders to interpret the feedback and decide what to do differently (if required). It’s not just a number!


The interviews are designed to look beyond the percentages to gain an understanding of the raters’ feelings, impressions and viewpoints. Gaining such insight into the hearts and minds of people is best acquired through the use of smaller, highly targeted samples. Expert interviewers, unencumbered by the strict time and structure constraints of a quantitative survey, use a multitude of techniques to obtain in-depth information. Interviews are often at times as long as one and a half hours, allowing the interviewer to elicit extremely candid, highly complex responses. The result is rich, in-depth data laden with insight unobtainable from quantitative online 360s.


Secondly, Qual360s are an investigative process. Using open-ended, face-to-face questioning provides an opportunity to probe and explore – getting to the heart of the matter. We are able to discover the “why” behind certain behaviours. Because the results are seen or heard first-hand, leaders relate to the findings easily. Using the language of the organisation, illustrating behaviour in practice through concrete examples coming from the day to day working environment has enormous impact on quickly accepting the feedback (ownership) and moving into action (inspired to act) to change.


Thirdly, Qual360s value show up most markedly in the feedback sessions – which can go for quite a while. There are no discussions about why someone might have scored a leader a 2 against a particular question – in the Qual360 the consultant will know exactly why a leader has been scored down, and have several examples at hand to back it up, often from different raters. This is extremely powerful, cut through, no escape feedback – and really helps leaders focus on changing behaviours rather than second guessing the data.


You seem to be describing a very in-depth, and perhaps sometime very personal, discussion.


Generally, the comment we receive after feedbacks for Qual360s from leaders is that they have never had such useful, honest feedback, ever before. And it is very often transformational for them. It is extremely transformational for those who provided – once removed – the feedback, because they can perhaps say things they wouldn’t have felt comfortable saying directly to their colleague, or was perhaps not sure whether this behaviour was exhibited in front of other colleagues. So it’s a real opportunity for a team to get on the same page as their leader, focus on important behaviour breakthroughs, and build team work and business results on the back of an in-depth dive into how a leader leads, and how they could possibly lead better.


Is it for everyone?


Certainly not, and indeed some clients who have enquired about the Qual360 have ended up using a normal 360 survey. But for critical talent – and particularly critical senior talent an organisation wants to keep and develop – Qual360s offer really in-depth insights, help map the readiness of the executive for next role (the trend we see at HFL is that most subjects think they are much more ready than they usually are), and can produce very actionable development plans that make a significant difference to the engagement of the entire team, not just the subject.


For more information about Qual360s please contact Maria Leske or Katelijne Pee at HFL.





Multi-raters image
HFL Leadership

About Us


HFL Network