Resident Pulser@infosec.pubB to Pulse of Truth@infosec.pubEnglish · 4 months agoWebsites are Blocking the Wrong AI Scrapers (Because AI Companies Keep Making New Ones)www.404media.coexternal-linkmessage-square3fedilinkarrow-up114arrow-down11file-text
arrow-up113arrow-down1external-linkWebsites are Blocking the Wrong AI Scrapers (Because AI Companies Keep Making New Ones)www.404media.coResident Pulser@infosec.pubB to Pulse of Truth@infosec.pubEnglish · 4 months agomessage-square3fedilinkfile-text
minus-squaredrkt@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up9·4 months agorobots.txt is a suggestion, and using it to block malicious actors is pointless.
minus-squaregarrett@infosec.publinkfedilinkEnglisharrow-up6·4 months agoUnfortunately, yeah. AI industry has proven that it’s willing to beg for forgiveness rather than asking for permission.
minus-squareGBU_28@lemm.eelinkfedilinkEnglisharrow-up5·4 months agoScrapers are not new. They’ve been doing this shit for years, the result of the scraping is just more transparent to the general internet public now. They’ve never asked permission.
robots.txt is a suggestion, and using it to block malicious actors is pointless.
Unfortunately, yeah. AI industry has proven that it’s willing to beg for forgiveness rather than asking for permission.
Scrapers are not new. They’ve been doing this shit for years, the result of the scraping is just more transparent to the general internet public now.
They’ve never asked permission.