How to Write Good Evaluation Criteria
Last updated
Last updated
Similar to ChatGPT, the AI is usually only as good as the natural language you write as criteria (a.k.a. your prompt). The following are examples of good and bad criteria.
A good piece of criteria language is singular, clear and specific. A smart human should be able to read this and determine if the candidate has that by either explicitly seeing it on their resume, or inferring it in the case of them being a subject-matter expert (which Endorsed’s AI is).
You’ve split out the criteria into individual line items.
For example, instead of adding one line like this:
“Experience with Python, Matlab, as well as front-end web technologies like Javascript and Tailwind”
Add three like this:
“Experience with Python”
“Experience with Matlab”
“Experience with front-end web technologies like Javascript and Tailwind”
The reason this is better is that the AI can more easily determine whether the candidate meets the criteria, and you’ll also have more granularity in the ranking. If a candidate meets 2/3 of the criteria, then you can still see that and they’ll rank higher than someone who is only 1/3.
If you enter in “fuzzy” language, you may not get the results you’re after. For example, say you were looking for talent that “Has an entrepreneurial background”, this could mean any number of things. So you could enrich the criteria language by saying “Has an entrepreneurial background, specifically founding a company versus just starting a club”.
Here are some more examples of clear criteria:
“Microsoft certifications (e.g., MCSA, MCSE)”
“Experience working in a customer supporting role like an Integrations Engineer”
“Marketing email campaign management experience”
“Experience with large Cloud providers like AWS or Azure”
You’ll have to give the AI a little more granular of instructions than a human for it to perform well. Here are a few examples of how to convert non-specific language into more specific language
Entrepreneurial background
Has founded a company
Worked for a Fortune 500 company
Explicitly mentions working at a Fortune 500 company by name
Experienced in complex project management
Experience in managing projects across a team or multiple teams
Has experience in leading development teams
Has held an Engineering Management, Director, or CTO role previously
Strong knowledge of Blockchain technologies
Strong experience in Blockchain technologies
Is experience with Next.js
Has 5 years of Next.js experience
AI is known to make mistakes occasionally with a few criteria. It often involves numerical calculations. These will be continuously improved over the coming months. For right how however, we recommend double checking accuracy on calculations like these:
Time zone calculations - i.e. “Within 3 hours of GMT-4”
KPI calculations - i.e. “Has cleared $200k sales quotas 4 years in a row”
Bad criteria language is ambiguous and unclear. Unfortunately, many line items in job descriptions are bad pieces of evaluation criteria. So we strongly recommend against blindly copy/pasting in each piece of criteria from job descriptions without filtering out bad criteria and copy-smithing (a.k.a prompt engineering) the others.
Examples of bad criteria are:
“Strong troubleshooting and problem-solving skills”
“Great communication skills”
“Blockchain or AI industry graphic design experience”
