Given how often you saw normal search engines in your logs and how often you now see AI crawlers, I can only guess that it's like a death-race of who downloads the whole internet first to train their shitty eliza pro+ LLMs with.
And since their hunger for new data is so immense, they have no time to be kind to webservers. Fuck webpages anyway, in their vision people will not visit webpages anymore in the future, they will just ask some shitshippiddi for everything.
im almost at that point. i dont do web searched barely at all now, for most general questions i use a LOCAL ONLY ai (LM Studio is a good easy starting point for experimenters). i run 20+gig models on a 24 GB vid card with 128 gig system RAM so its not to bad.
but the bestest part.. some models are so good thats you would swear it going out to the net. that resulted in me connected a old school physical mechanical switch that that i hit it and that physically severs the cable connection.. easier than unplugging the ethernet cord alla time. oh and the BT/WiFi radios are pulled from the mobo.
so now actual web searches are pretty rare around here.
raw capitalism baby. nothing like it!
edit: just to reinforce that my AIs are run LOCAL entirely on my box.. as i dont need musks opinions from grok, or zuccs latet visions on their live versionwhater on chatgpt and such that change with wind.
i dl the model, check it meets minimums, and tweak it suitably. and everything stays local.
