flâneur

Monolith – CLI tool for saving complete web pages as a single HTML file | Hacker News

news.ycombinator.com · 5,882 words · saved by 1 readers

My own https://shot-scraper.datasette.io/ tool (which uses headless Playwright Chromium under the hood) has a command for this too: I use that for things like accepting cookie banners, but using it to scroll down to trigger additional loading should work too. There's also a --wait-for option which takes a JavaScript expression and polls until it's true before taking the shot - useful for if there's custom loading behavior you need to wait for. Documentation here: https://shot-scraper.datasette.io/en/stable/screenshots.html You can come up with workarounds for each, but it's still hacky and there's always going to be other pages that need special treatment. 1 - https://simpatico.io/acceptance.js 2 - https://simpatico.io/acceptance That's handy for when you're developing a front end and IT/devops hasn't approved/enabled the the CORS settings on the backend yet, or if you're just hacking around and want to get data from somewhere that doesn't allow cross domain requests. I tried this and

Well this is fun... from the README here I learned I can do this on macOS: /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \ --headless --incognito --dump-dom https://github.com > /tmp/github.html And get an HTML file for a page after the JavaScript has been executed. Wrote up a TIL about this with more details: https://til.simonwillison.net/chrome/headless My own https://shot-scraper.datasette.io/ tool (which uses headless Playwright Chromium under the hood) has a command for this too: shot-scraper html https://github.com/ > /tmp/github.html But it's neat that you can…

saved by

related reading