Sara Fish • LLM Evals Research Advice
sarafish.com · 1,282 words · saved by 1 readers
LLM Evals Research Advice
Sara Fish • LLM Evals Research Advice [ Back to main page ] Some LLM evals research advice June 9, 2026 Below are some general pieces of advice I would give to anybody conducting LLM evals / behavioral LLM science research. Take it or leave it -- you may find that you work more effectively doing things differently -- these are just some points to consider. (Note: I expect some of this advice to become outdated as AI technology advances, particularly the lines I draw on what AI can and can't be trusted with.) Low-level technical stuff Log everything Logging is important for two reasons. First,
saved by
related reading
- Sara Fish • LLM Evals Research Advicesara-fish.github.io
- Thoughts on AI in academiatheinfinitesimal.substack.com
- Tips and Code for Empirical Research Workflows — AI Alignment Forumalignmentforum.org
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- Your AI Product Needs Evals – Hamel's Blog - Hamel Husainhamel.dev
- Thoughts — Jason Weijasonwei.net
- The machines are fine. I'm worried about us.ergosphere.blog
- Tips and Code for Empirical Research Workflows — LessWronglesswrong.com
- Writing for LLMs So They Listen · Gwern.netgwern.net
- Tips for Empirical Alignment Research — AI Alignment Forumalignmentforum.org
- Tips and Code for Empirical Research Workflows — LessWronglesswrong.com
- A pragmatic guide to LLM evals for devsnewsletter.pragmaticengineer.com