loyce's and bpip's scraper also seems to be down? (paging @LoyceV, @suchmoon, @ibminer).
True, I can't scrape anything - getting 503 errors.
1. If Cloudflare pops up, calls a function that invokes a
puppeteer page (an instance of chrome) to the forum and waits for CF to clear out.
I have a prototype of a parser that works entirely via Playwright/Firefox and even solves captchas but it'd be quite absurd to use it. Not only because it needs an ungodly amount of RAM to work, but also because it looks like we're going against the policy (unwritten as it may be) of Cloudflare and/or Bitcointalk. This needs to be solved properly, i.e. by implementing some sort of whitelist/key/whatever. Not by trying to hack the damn thing.
Edit: my other parser that I use for plagiarism checks is still running albeit intermittently, not sure what the difference is that makes this one kinda sorta work.