UKGovernmentBEIS/inspect_ai: Inspect: A framework for large language model evaluations ·
github.com · 400 words · saved by 1 readers
Inspect: A framework for large language model evaluations
Welcome to Inspect, a framework for large language model evaluations created by the UK AI Security Institute . Inspect provides many built-in components, including facilities for prompt engineering, tool usage, multi-turn dialog, and model graded evaluations. Extensions to Inspect (e.g. to support new elicitation and scoring techniques) can be provided by other Python packages. To get started with Inspect, please see the documentation at https://inspect.aisi.org.uk/ . Inspect also includes a collection of over 200 pre-built evaluations ready to run on any model (learn more at https://inspect.a
related reading
- Inspectinspect.ai-safety-institute.org.uk
- Why We Built Our Own Background Agentbuilders.ramp.com
- GitHub - brexhq/prompt-engineering: Tips and tricks for working with Large Language Models like OpenAI's GPT-4.github.com
- GitHub - rasbt/LLMs-from-scratch: Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepgithub.com
- llm-wikigist.github.com
- Your AI Product Needs Evals – Hamel's Blog - Hamel Husainhamel.dev
- GitHub - oughtinc/ice: Interactive Composition Explorer: a debugger for compositional language model programsgithub.com
- GitHub - guidance-ai/guidance: A guidance language for controlling large language models.github.com
- GitHub - UKGovernmentBEIS/control-arena: ControlArena is a collection of settings, model organisms and protocols - for running control experiments. · GitHubgithub.com
- GitHub - salesforce/AuditNLG: AuditNLG: Auditing Generative AI Language Modeling for Trustworthinessgithub.com
- GitHub - deepset-ai/haystack: Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.github.com
- GitHub - swyxio/ai-notes: notes for software engineers getting up to speed on new AI developments. Serves as datastore for https://latent.space writing, and product brainstorming, but has cleaned up canonical references under the /Resources folder.github.com