From my work with search committees, one question that often comes up is how to evaluate whether a search firm is driven by data with a focus on the competencies required for success in a given role. Staffing Advisors’ entire approach to hiring is competency-driven. But how can that be proven or compared to another search firm?

A Firm’s Retention Rate Isn’t the Best Indicator

There are several elements to consider. Sadly, a firm’s stated retention rate cannot be one of them. Because it cannot be independently verified or observed, anyone can claim any retention rate on their placements. For example, our rate is 85% retention at 3 years, but there is no realistic way a client could verify that number. It would require knowing every single one of our searches, reaching out to every single client, and confirming the employment dates for every placement we’ve done.

Even if someone did all that, it still wouldn’t be a perfect comparison. Even if you assumed an honest intent to report retention rates accurately, the math to calculate retention can be fuzzy, with wildly varying definitions between firms. For example, we have an 85% retention rate on our placements at 3 years. But to calculate our 3-year retention rate, we only consider people who we placed at least 3 years ago. If we looked at everyone we placed within the past 3 years, our rate would be much higher because it would include people who started work last week.

What to Ask an Executive Search Firm to Determine Their Approach

In the early search committee meetings, what will be discussed? A rigorous evaluation of the business problem to be solved or a general discussion of strategic goals and ideal candidate attributes? When speaking with stakeholders, is the information gathered in a similar vein?

Is the search committee discussing problems to be solved and discussing the tangible, measurable competencies needed to drive results (knowledge skills and abilities that can be tested)? Or is the search committee primarily discussing attributes? (Attributes are generally opinions that can be open to interpretation depending on who is doing the evaluation, such as bold, courageous, innovative, mature, etc.) Attributes can be a problem because research has shown that more than half of your evaluation of another person is a reflection of you and not the person being evaluated.

Are they generally similar to each other or meaningfully different? Are they full of vague generalities that might apply to a variety of organizations, or do they clearly signal the search committee’s intentions? Do they identify the business problem to be solved, indicate the nature of the challenges, and outline the amount of running room the new leader will have? Are the messages interesting or even inspiring to read? And most importantly, is it clear to you what performance is expected? Top performers want to know that the board (or search committee) has thought deeply about what is needed and is clear about performance expectations; otherwise, your job could be a massive career risk for them.

Data-driven recruiting offers the additional benefit of being more inclusive of diversity – a “Moneyball” approach to recruiting. Diversity comes from the right message being delivered to the right audience right from the start. Diversity comes from thinking expansively about the underlying pool of who will be contacted. It is not about awarding points to certain people or just posting ads on so-called “diversity job boards.”

Listen for if the search firm is using selection criteria that actually predict success on the job. (More on that here.) Recruiting often looks like a modern integrated marketing campaign. There are metrics, A/B testing, a variety of message delivery mechanisms, and a feedback loop that includes the search committee. Be concerned if the candidate selection criteria veer toward elements that are not predictive of success on the job (such as education, years of experience, the prestige of the candidate’s current employer, confidence, being known by the search consultant, etc.) These elements invariably exclude some very well-qualified candidates.

And be positively alarmed about vague answers suggesting that “Everyone is in our database,” “Everyone takes the call when it is from our firm,” or any reference to “Good people know good people.” Those “good people” mostly tend to know demographically similar people, and that can be remarkably limiting to the development of your candidate pool.

When it’s time to select candidates for an interview, what information will be shared with the search committee beyond the resume? Will the search committee receive some kind of pretty “candidate pitch” document (like a PR agent might produce), or receive some type of briefing from the search consultant (also designed to pitch the candidate), or will you see something else?

We prefer to share information specifically related to the competencies the search committee has discussed that are most likely to result in success on the job. The candidate submits this written supplemental information on their own, so it is both a writing sample and an evaluation of how they think, what they value, and what they prioritize. No fluff, no pitch, just data that is likely to predict success on the job.

Interview questions are often suggested by board members or other stakeholders and may not be specifically related to competencies that drive results. This can result in nearly meaningless questions like, “What is your philosophy about leadership?” It is best to avoid the realm of theoretical questions in the interview process. Talking about work and doing work are quite different skills.

Does the search firm suggest ways to add rigor to the hiring sequence with work-sample testing? Do they help you create an environment where candidates might fail, or do they suggest a less rigorous hiring sequence? Are job competencies also evaluated in reference checks?

The search firm should be current with the latest research on hiring practices and well aware of the fact that typical hiring practices are laden with bias and prone to error. Whatever data-driven approach they suggest should be designed to zealously guard against the very human nature to prefer “people like us” and help the committee focus on factors that predict success on the job and avoid reflexively gravitating toward the person “you’d like to have a beer with.”

If a search consultant is surfacing issues that make you stop and think at regular intervals, you are probably on the right track. (Then you know they are thinking equally hard.) But if they readily accept simple answers to complex questions, you might want to find another consultant.

This article outlines some of the latest research on hiring. and how to guard against the deeply-rooted problem of interview subjectivity.

Hopefully, you found this information useful. Please feel free to contact us to learn more about our work with search committeesbest practices for search committees, or our process for recruiting senior staff members.