Describe what you want to learn. Candor recruits participants, generates the right interface, prevents bias, and delivers results — so you can focus on building.
Tasks and display positions are shuffled per participant to prevent position bias. Pairwise items are counterbalanced 50/50 AB/BA.
Tasks grouped into right-sized assignments: 5 for comparisons, 10 for ratings, 20 for labeling. Prevents fatigue.
Rewards estimated from media type + task complexity, targeting $12–18/hr. Audio comparisons priced higher.
10% of tasks are known-answer pairs. Participants who fail are excluded. Only engaged evaluator data reaches results.
Krippendorff’s alpha for inter-rater reliability. Per-pair disagreement breakdowns. Know when you have signal.
All N*(N-1)/2 pairs generated. Display order counterbalanced. Produces ranked leaderboards with win rates.
Learn more →Independent item evaluation on numeric scales. Per-item means, medians, and standard deviations.
Learn more →Custom label taxonomies. Label distributions and confidence scores per item. Inter-annotator agreement included.
Learn more →Open-ended descriptions, transcriptions, and reviews. Each reviewer writes independently.
Learn more →Every task type can be combined with an AI moderator. Choose the right scope for your research — from hands-off structured tasks to a study-level moderator that learns across sessions.
Structured tasks with no AI involvement. Participants compare, rate, or label items independently.
PassiveMic on, voice transcribed, but AI doesn't speak. Captures think-aloud observations as participants browse.
Most popularAI conducts a live voice interview within one session. Adapts follow-ups based on what the participant says and does.
AdvancedAI learns across all sessions. Identifies themes early, probes gaps later. Use coverage reports to track exploration.
Install the CLI and launch a study in under two minutes.