leogao's Shortform — LessWrong
Comment by leogao - some lessons from ml research: * any shocking or surprising result in your own experiment is 80% likely to be a bug until proven otherwise. your first thought should always be to comb for bugs. * only after you have ruled out bugs do you get to actually think about how to fit your theory to the data, and even then, there might still be a hidden bug. * most papers are terrible and don't replicate. * most techniques that sound intuitively plausible don't work. * most techniques only look good if you don't pick a strong enough baseline. * an actually good idea can take many tries before it works. * once you have good research intuitions, the most productive state to be in is to literally not think about what will go into the paper and just do experiments that satisfy your curiosity and. convince yourself that the thing is true. once you have that, running the final sweeps is really easy * most people have no intuition whatsoever about their hardware and so will write code that is horribly inefficient. even learning a little bit about hardware fundamentals so you don't do anything obviously dumb is super valuable * in a long and complex enough project, you will almost certainly have a bug that invalidates weeks (or months) of work. being really careful and testing helps but slows down velocity a lot. unclear what the right equilibrium is. * feedback loop time is incredibly important, if you can get rapid feedback, you will make so much more progress. * implementing something that is already known to work is always vastly easier than inventing/researching something new. * you will inevitably spend a lot of time doing things that have no impact on the final published work whatsoever. like not even contributing that much useful intuition. this is unfortunate but unavoidable * oftentimes you will spend a lot of time being fundamentally philosophically confused about what to do, and only really figure out halfway through the project. this is normal. * directio
Explore this link on the map →