Ask or search…
K

How to Write Good Evaluation Criteria

Similar to ChatGPT, the AI is usually only as good as the natural language you write as criteria (a.k.a. your prompt). The following are examples of good and bad criteria.

Examples of Good Criteria

A good piece of criteria language is singular, clear and specific. A smart human should be able to read this and determine if the candidate has that by either explicitly seeing it on their resume, or inferring it in the case of them being a subject-matter expert (which Endorsed’s AI is).

Singular

You’ve split out the criteria into individual line items.
For example, instead of adding one line like this:
  • “Experience with Python, Matlab, as well as front-end web technologies like Javascript and Tailwind”
Add three like this:
  • “Experience with Python”
  • “Experience with Matlab”
  • “Experience with front-end web technologies like Javascript and Tailwind”
The reason this is better is that the AI can more easily determine whether the candidate meets the criteria, and you’ll also have more granularity in the ranking. If a candidate meets 2/3 of the criteria, then you can still see that and they’ll rank higher than someone who is only 1/3.

Clear

If you enter in “fuzzy” language, you may not get the results you’re after. For example, say you were looking for talent that “Has an entrepreneurial background”, this could mean any number of things. So you could enrich the criteria language by saying “Has an entrepreneurial background, specifically founding a company versus just starting a club”.
Here are some more examples of clear criteria:
  • “Microsoft certifications (e.g., MCSA, MCSE)”
  • “Experience working in a customer supporting role like an Integrations Engineer”
  • “Marketing email campaign management experience”
  • “Experience with large Cloud providers like AWS or Azure”

Specific

You’ll have to give the AI a little more granular of instructions than a human for it to perform well. Here are a few examples of how to convert non-specific language into more specific language
Non-Specific Language
Specific Language
Entrepreneurial background
Has founded a company
Worked for a Fortune 500 company
Explicitly mentions working at a Fortune 500 company by name
Experienced in complex project management
Experience in managing projects across a team or multiple teams
Has experience in leading development teams
Has held an Engineering Management, Director, or CTO role previously
Strong knowledge of Blockchain technologies
Strong experience in Blockchain technologies

Examples of (Temporarily) Bad Criteria

There are a few known things that the AI often gets wrong, and are on our roadmap to fix over the coming weeks. One cluster in particular are any criteria involving numbers like:
  • Years of experience calculations - i.e. “5 years of Next.js experience”
  • Time zone calculations - i.e. “Within 3 hours of GMT-4”
  • KPI calculations - i.e. “Has cleared $200k sales quotas 4 years in a row”
For the time being, we would recommend leaving these out, and then letting the Endorsed team know what you had to artificially remove to improve quality. That way we can prioritize our roadmap most effectively.

Examples of (Always) Bad Criteria

Bad criteria language is ambiguous and unclear. Unfortunately, many line items in job descriptions are bad pieces of evaluation criteria. So we strongly recommend against blindly copy/pasting in each piece of criteria from job descriptions without filtering out bad criteria and copy-smithing (a.k.a prompt engineering) the others.
Examples of bad criteria are:
  • “Strong troubleshooting and problem-solving skills”
  • “Great communication skills”
  • “Blockchain or AI industry graphic design experience”
Last modified 2mo ago
© 2024 Endorsed