get-convex/convex-evals
Search or jump to... Search code, repositories, users, issues, pull requests... Search Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name Query To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up Reseting focus
Convex Coding Evals Convex is an open-source, reactive database that's the best platform for full-stack AI coding. We ensure that Convex performs well with a large set of models by continuously running evals. Each eval has a set prompts for coding a Convex backend, a set of human-curated solutions, and a script for evaluating the LLM's output. These evals are split up into eight different categories: Fundamentals Data Modeling Queries Mutations Actions Idioms Clients Components The most up to date eval runs can be found on our website. Detailed results from production runs can be…
saved by
related reading
- Demystifying evals for AI agents \ Anthropicanthropic.com
- Your AI Product Needs Evals – Hamel's Blog - Hamel Husainhamel.dev
- AI Model & API Providers Analysis | Artificial Analysisartificialanalysis.ai
- Cookbookcookbook.openai.com
- Best practices for Claude Code - Claude Code Docsanthropic.com
- Introducing Supabase Evalssupabase.com
- LLM evaluation: a beginner's guideevidentlyai.com
- GitHub - open-compass/opencompass: OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.github.com
- Claude SWE-Bench Performance \ Anthropicanthropic.com
- Quantifying infrastructure noise in agentic coding evals \ Anthropicanthropic.com
- Senior SWE-Benchsenior-swe-bench.snorkel.ai
- A pragmatic guide to LLM evals for devsnewsletter.pragmaticengineer.com