Resident Pulser@infosec.pubB to Pulse of Truth@infosec.pubEnglish · 3 days agoAI haters build tarpits to trap and trick AI scrapers that ignore robots.txtarstechnica.comexternal-linkmessage-square15fedilinkarrow-up1133arrow-down14file-textcross-posted to: technology@lemmy.worldtechnology@lemmit.online
arrow-up1129arrow-down1external-linkAI haters build tarpits to trap and trick AI scrapers that ignore robots.txtarstechnica.comResident Pulser@infosec.pubB to Pulse of Truth@infosec.pubEnglish · 3 days agomessage-square15fedilinkfile-textcross-posted to: technology@lemmy.worldtechnology@lemmit.online
minus-squareBlaster M@lemmy.worldlinkfedilinkEnglisharrow-up5·2 days agoRather funny that some of these are in code if(useragent == chatgpt) kind of sillyness. You need heiuristics to detect scrapers, because they’ll just report their user agent as the average user’s browser.
Rather funny that some of these are in code if(useragent == chatgpt) kind of sillyness. You need heiuristics to detect scrapers, because they’ll just report their user agent as the average user’s browser.