@adlrocha - How I built an IPFS crawler with libp2p

And lessons learnt

I already mentioned in my publication dedicated to my financial dashboard side-project that, for me, the most effective way of learning something new is embarking myself in a “pet project” that motivates me enough not to abandon it halfway through. I force myself to develop the project using the new technologies I want to learn so in the process I end up with a brand new side project and a whole bunch of new skills. The financial dashboard was an excuse to learn GraphQL and ReactJS (to be honest, since then I’ve been actively using ReactJS more and more, so it paid off). Today I am going to…

related reading

• Building a web search engine from scratch in two months with 3 billion neural embeddingsblog.wilsonl.in
• Building a BFT JSON CRDTjzhao.xyz
• The Architecture of a Web Crawler: Building a Google-Inspired Distributed Web Crawler. Part 1 | by TonyWang | Mediummedium.com
• How I side project | thesephist.comthesephist.com
• Optimistic Provide: How We Made IPFS Content Publishing 10x Fasterprobelab.io
• Let’s Handle 1 Million Requests per Second, It’s Scarier Than You Think!youtube.com
• Crawling a billion web pages in just over 24 hoursandrewkchan.dev
• The sad state of personal data and infrastructure | beepb00pbeepb00p.xyz
• Profiling Go programs with pprofjvns.ca
• Peer-to-Peer Ordered Search Indexes – 0 FPS0fps.net
• How to Build a Web Crawler in Pythonzenrows.com
• GitHub - amirgamil/apollo: A Unix-style personal search engine and web crawler for your digital footprint.github.com