case study · 2025
Rubricate
An agentic AI voice-interview and evaluation platform — the entire platform, built solo — that vets domain experts at scale with adaptive voice interviews backed by a full automated evaluation pipeline. The expert network behind Dataclap.
Adaptive real-time voice
Interview mode
Automated rubrics
Scoring
Source → onboard
Funnel coverage
What needed solving
Vetting domain experts at scale is a bottleneck: every candidate needs a real conversation with someone qualified to judge them, and that doesn't scale. Screening was consuming expert time that should have gone to actual work, and consistency between interviewers was impossible to guarantee.
How it fits together
A real-time voice loop connects each candidate to an interview agent that writes its next question from the last answer. Every transcript flows into an evaluation pipeline that scores against explicit rubrics, and the scores drive automated workflows across sourcing, screening and onboarding.
Candidate
Candidate
live voice session
Voice
Voice loop
real-time turns
Agent
Interview agent
adaptive follow-ups
Scoring rubrics
defined criteria
Evaluation
Eval pipeline
rubric scoring
Workflows
Sourcing
automated
Screening
automated
Onboarding
automated
// stack
The calls that shaped it
Follow-ups generated from the answer, not a script
Each question is written from what the candidate actually said, so the interview probes real depth instead of checking off a fixed list.
trade-off · No two interviews are identical, so consistency has to come from the scoring side: every transcript is graded against the same rubric.
Explicit rubrics for every evaluation
Every candidate is judged against the same defined criteria. That level of consistency is impossible to guarantee across human interviewers.
trade-off · Every new assessment domain needs its own rubric written and validated before the pipeline can score it.
A latency budget for the voice loop
Turn-taking has to feel like a conversation rather than a form, so the real-time loop was engineered around response latency first.
trade-off · Streaming speech-to-text, LLM and text-to-speech in parallel adds real complexity, and turn detection needs a model of its own rather than a simple silence timer.
One platform for the whole funnel
Sourcing, screening, evaluation and onboarding run as automated workflows in one system, so a score moves a candidate forward without a manual handoff.
trade-off · One system means one place to break, so error tracking and session logging cover every stage of the funnel.
Where it landed
Rubricate runs in production as the expert network behind Dataclap, conducting adaptive voice interviews at a volume that would be impossible to staff manually, with consistent rubric-based scoring across every candidate. The voice AI, the evaluation pipeline and every workflow around them are one platform, built end to end by one engineer.
Adaptive, real-time voice
Interview format
Rubric-based, structured
Scoring
Source → onboard
Funnel automated
One engineer, solo
Built by
What I'd do differently
- →I'd measure latency per stage from the first prototype. End-to-end numbers told me the conversation felt slow, but only per-stage timing told me why.
- →Real candidates pause for a long time while they think. Silence-based turn detection kept cutting them off, and fixing that took a semantic turn detector plus several rounds of tuning.
- →I'd build the human-reviewed golden set earlier. Agreement with careful human graders is the only number that really tells you whether an LLM evaluator can be trusted.
// also offered as freelance services
// next step
Hiring for this kind of work?
I'm open to full-time AI Engineer roles. Grab my resume, or email me and I'll walk you through how this was built.