My experience on starting with fine tuning LLMs with custom data : LocalLLaMA

I keep seeing questions about "How I make a model to answer based on my data. I have [wiki, pdfs, whatever other documents]" Currently I am making a living by helping companies built chatbots fine tuned on their custom data. Most of those are support or Q&A chatbots to answer questions from clients at any hour and day. There are also internal chatbots to be used to train new people joining the company and several other use cases. So, I was thinking to share my experience (it might be wrong and I might be doing everything wrong, but it is my experience and based on this I have a dozen chatbots running in production and talking with clients with few dozen more in different stages of testing). The actual training / fine-tuning, while it might initially seem like a daunting task due to the plethora of tools available (FastChat, Axolot, Deepspeed, transformers, LoRA, qLoRA, and more), I must tell you - this is actually the easiest part of the whole process! All you need to do is peek into t

saved by

• Shubham Chandel