My experience on starting with fine tuning LLMs with custom data : LocalLLaMA
I keep seeing questions about "How I make a model to answer based on my data. I have [wiki, pdfs, whatever other documents]" Currently I am making a living by helping companies built chatbots fine tuned on their custom data. Most of those are support or Q&A chatbots to answer questions from clients at any hour and day. There are also internal chatbots to be used to train new people joining the company and several other use cases. So, I was thinking to share my experience (it might be wrong and I might be doing everything wrong, but it is my experience and based on this I have a dozen chatbots running in production and talking with clients with few dozen more in different stages of testing). The actual training / fine-tuning, while it might initially seem like a daunting task due to the plethora of tools available (FastChat, Axolot, Deepspeed, transformers, LoRA, qLoRA, and more), I must tell you - this is actually the easiest part of the whole process! All you need to do is peek into t
I've been a .net dev since forever, started coding during the .net boom with asp/vb6. For the past 10 years most of the work has been CMS websites, integrations, services etc. I am very interested in what you're talking about. Right now I'm building my own application with Semantic Kernel and looking into using embeddings as you suggested, but this is my MVP. I think you're on the right track for setting up enterprises with private LLMs. I assume that enterprises will have all of their data, all of it, integrated into a LLM. Every email, transcribed teams conversation, legal paper,…
saved by
related reading
- LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
- Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- Building Effective AI Agents \ Anthropicanthropic.com
- Patterns for Building LLM-based Systems & Productseugeneyan.com
- Building Effective AI Agents \ Anthropicanthropic.com
- Using Large Language Models Effectivelyunsupervisedlearning.substack.com
- Building LLM applications for productionhuyenchip.com
- Andrej Karpathy on X: "LLM Knowledge Bases Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating" / Xx.com
- The Llama Hitchiking Guide to Local LLMs – hackerllamaosanseviero.github.io
- Emerging Architectures for LLM Applications | Andreessen Horowitza16z.com
- Introduction to LLM Agents | NVIDIA Technical Blogdeveloper.nvidia.com