How do you effectively interview for a set of skills that is only just emerging, in an area where most candidates lack a formal record of success? By getting specific about your expectations and staying open to how candidates show their abilities during interviews.

Our guidance for assessing AI capabilities is the same we recommend for any skill set. Define what you need first and build each step of the interview sequence around that. It’s an effective, proven way to assess candidates and can be adapted when new skills and needs arise, as they inevitably will.

The following framework is designed to help you assess the AI capabilities of mission-driven professionals whose roles increasingly require them to understand and apply AI to their work.

Define Results and Where AI Is Likely to Show Up in the Work

Asking whether a candidate knows how to use AI is like asking whether they know how to use the internet. It’s too vague. You want to learn whether the candidate will be able to use AI tools—or understands how AI impacts the function, field, or profession—to work toward specific goals. Start by defining:

  1. The results you expect the person in the role to achieve. Think outcomes and impact, not activities. What problem do you want your new hire to solve?
  2. How you expect them to use AI, or apply an understanding of it, to reach that goal.

If you’re not clear on what that looks like yet, it may help to see how other organizations are thinking about this work. In the roles we’ve recruited for over the past year, AI capability tends to show up in two main areas: responding to how AI is changing the external environment and using AI to change how your organization works internally. Here are a few examples of what we are seeing.

External Impact

  • Helping Members Navigate Change. Association leaders are expected to understand the implications of AI for their communities and help members navigate it. Business impacts here might be strengthening the member value proposition, positioning the organization as a trusted resource or authority in the field, and ensuring that the organization remains relevant to target audiences. We see this most often in roles with oversight of education strategy, such as CEOs and executive directors, scientific leaders, and education leaders.
  • Adapting Digital Strategies to Changes in User Behavior. Many nonprofits rely on web traffic and digital channels to connect with their constituents and raise funds. As AI changes how people consume information and interact online, implementing generative engine optimization strategies and rethinking user experience are priorities. Business impacts could include greater visibility in AI-generated search results and increased engagement from human users. We see this in senior roles that set digital strategy and marketing and web teams that carry it out.

Internal Impact

  • Improving Internal Workflows. Mission-driven organizations operate with limited resources and increasing demands, and they are interested in how AI can help staff work more efficiently. Business impacts here might include reducing routine tasks to make room for higher-value work, facilitating more accurate information sharing from meetings, and improving documentation to support continuity. We see this most often in roles responsible for process improvement, including chief operating officers, operations directors, and program leads.
  • Financial Analysis and Reporting. As many organizations explore how to strengthen financial sustainability and diversify revenue, finance teams are interested in how AI-enabled tools can create efficiencies, reduce errors, and support more robust reporting that helps leaders make decisions. Business impacts here might be identifying financial risks earlier, supporting forecasting and scenario planning, and improving assessment of program performance. We see this broadly across finance leadership roles, including chief financial officers, VPs and directors of finance and accounting, and controllers.

Related article: Embracing AI with Kraig Conrad of NCMA

Build Interview Questions Around Results and Stay Open to How Candidates Respond

Once you’ve defined results and where AI may fit into the picture, develop interview questions around those expectations. Ask how candidates would approach the problem you need solved and where and how AI relates to that work. Here are a few examples from recent searches:

  • Expected result: Cut staff time spent on pulling reports and reconciling records across systems to create time for more member-facing work.
    Interview question: Share any systems (processes and technology) you have introduced to increase efficiency, including enabling AI and automation capabilities.
  • Expected result: Modernize and improve the effectiveness of the association’s certification program.
    Interview question: How have you leveraged technology, including AI, and innovative learning modalities to enhance education and training and expand delivery methods?
  • Expected result: Keep the association relevant to members as AI reshapes the profession, strengthening the value of membership renewal.
    Interview question: Describe a time when you successfully led a membership organization through a major industry shift, and how you managed innovation and change during that process.

In some cases, candidates will have experience that directly relates to your needs. But because organizations vary widely in how much they’ve adopted AI, if you only select those candidates to move forward, you may miss others with great potential.

At its core, this is interviewing for innovation. During interviews, listen for the habits of mind that lead to success when doing something that hasn’t been done before. The most promising candidates will:

  • Show curiosity and interest when discussing this work. If they haven’t led formal AI-related initiatives for their former employers, they will still be eager to share what they’ve tried on their own, researched, or discussed with other experts in the field.
  • Offer specific examples of how they’ve used technology to solve problems or helped others understand or adopt something new. They can explain what they did, why they did it, and the impact of that work with plenty of detail and metrics.
  • Demonstrate humility in the face of doing something new, shown by examples of approaches that failed and what they changed or would change next time.
  • Describe their approach to gathering data and assessing risk before moving forward and how they know when it’s time to launch or stop an experiment.

Related article: What Does “Qualified” Look Like in the Age of AI?

Use a Collaborative Exercise in the Interview Sequence

We’ve long recommended a collaborative work sample test as part of every interview sequence. The practice can be especially valuable for assessing AI competency, which can encompass a range of skills and is specific to the role and how your organization operates.

After initial interviews, give finalists a scenario similar to the work they’d do on the job and ask them to share their thinking when you meet next. Then discuss their approach with them and work through their ideas just as you would with a colleague. The best candidates will help you think about the work in ways you haven’t before. Find specific guidance on developing a properly scoped work sample here.

A few AI-related examples from recent senior leadership roles (edited to remove identifying details):

  • Systems Integration: We have many robust data systems (some are more user-friendly than others), but we lack full integration. How would you approach integrating our databases so both internal and external stakeholders (budget owners, development team, HR, partner organizations, board members) can seamlessly access the data? What factors would drive your decision-making? Where and how would you consider AI in these efforts?
  • Strategic Prioritization: You have access to a $18 million annual operating budget and $500,000 designated for strategic investments, plus a potential drawdown of up to $1 million from reserves. Your team includes 60 staff across departments. Priorities include branding, expanding partnerships, diversifying revenue, technology/AI strategy, building alignment, and innovation. What would you do with these assets and why? Where does AI strategy fit among these priorities, and how would you approach it? How would you engage the board in a conversation around your ideas?
  • Member Support: We’ve identified a strategic priority to help members understand and use AI and emerging data tools in their work, but progress on planning and implementation has stalled due to a lack of consensus around what’s most valuable to the field. How would you help the organization align around a shared vision and ensure efforts match member needs?

You Don’t Have to Be an AI Expert to Interview Effectively

There is a lot of hype around AI right now and we are all still figuring out how to adjust to new ways of working and interacting in the world. It’s easy to feel intimidated by the prospect of hiring people to do work you may not fully understand yourself. But you don’t need to be an AI expert to run a good process. This approach will help you compare candidates against your needs and (hopefully) take some of the guesswork out of hiring in this environment.