@WorkingLemmy@lemmy.world to Open Source@lemmy.ml • 3 months agoFOSS infrastructure is under attack by AI companiesthelibre.newsexternal-linkmessage-square24fedilinkarrow-up1374arrow-down10cross-posted to: technology@beehaw.orglinux@programming.devtechnology@lemmy.worldopensource@programming.devtechnology@lemmy.worldtechnology@lemmy.worldhackernews@lemmy.bestiver.se
arrow-up1374arrow-down1external-linkFOSS infrastructure is under attack by AI companiesthelibre.news@WorkingLemmy@lemmy.world to Open Source@lemmy.ml • 3 months agomessage-square24fedilinkcross-posted to: technology@beehaw.orglinux@programming.devtechnology@lemmy.worldopensource@programming.devtechnology@lemmy.worldtechnology@lemmy.worldhackernews@lemmy.bestiver.se
minus-square@Strawberry@lemmy.blahaj.zonelinkfedilink5•3 months agoThe bots scrape costly endpoints like the entire edit histories of every page on a wiki. You can’t always just cache every possible generated page at the same time.
minus-square@jagged_circle@feddit.nllinkfedilinkEnglish1•edit-23 months agoOf course you can. This is why people use CDNs. Put the entire site on a CDN with a cache of 24 hours for unauthenticated users.
The bots scrape costly endpoints like the entire edit histories of every page on a wiki. You can’t always just cache every possible generated page at the same time.
Of course you can. This is why people use CDNs.
Put the entire site on a CDN with a cache of 24 hours for unauthenticated users.