Learning to Replicate Expert Judgment in Financial Tasks - Thinking Machines Lab
With expert-labeled data and fine-tuning on Tinker, a custom model outperforms frontier LLMs on financial information-filtering tasks at a fraction of the cost.
Judging information Outperforming the market is hard. When every investor has access to the same sources of public information, alpha must come from unique insight built on taste and judgment. A strong investor’s judgment is difficult to articulate and teach directly to others, whether human or AI. It comes from experience. Even when we decompose an investor's job into its simplest constituent tasks, those tasks turn out to be surprisingly difficult for LLMs. In this post, we consider a simple special case: filtering and processing financial documents to surface information relevant to investm
saved by
related reading
- On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
- Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
- Training LLMs to Predict World Events (Guest Post with Mantic) - Thinking Machines Labthinkingmachines.ai
- Putting Task Expertise into RL Achieves State-of-the-Art Performance on Text-to-SQLthinkingmachines.ai
- Inkling: Our Open-Weights Model - Thinking Machines Labthinkingmachines.ai
- Anatomy of a Modern Finetuning APIbenanderson.work
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- Shaping capabilities with token-level data filteringarxiv.org
- The bitter lesson of LLM evalsparsed.com
- Fine-Tuning Llama-2: Tailoring Models to Unique Applicationsanyscale.com
- J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learningarxiv.org
- Base Models Know How to Reason, Thinking Models Learn Whenarxiv.org