Changes between Version 1 and Version 2 of Ticket #24266, comment 4


Ignore:
Timestamp:
2025-04-18T22:04:18+02:00 (9 months ago)
Author:
stoecker

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #24266, comment 4

    v1 v2  
    66* April 2025 (i.e. half a month only) has 2 million visitors, 8 million pages and 12 million requests
    77
    8 Until recently I reacted to slowness of the server with blocking bad actors (usually whole network segments which i don't like much because of collateral damage). Typically these could be tracked to AI companies which forget all rules how to behave to get data. I'm not opposed to the data scraping they do, only to the method they are doing it. Hey guys: Rather than hammering one server with dozens requests at the same time how about changing your bots to access dozens of servers with one request? Google and the others also learned that, you also can learn to be nice and get your data.
     8Until recently I reacted to slowness of the server with blocking bad actors (usually whole network segments which I don't like much because of collateral damage). Typically these could be tracked to AI companies which forget all rules how to behave to get data. I'm not opposed to the data scraping they do, only to the method they are doing it. Hey guys: Rather than hammering one server with dozens requests at the same time how about changing your bots to access dozens of servers with one request? Google and the others also learned that, you also can learn to be nice and get your data.
    99
    1010The increased number of visitors in April shows that either a LOT of new actors came up or site scraping now is using a botnet. Access comes from all-over the world. There is no more chance to filter that by IP. I had to improve the whole setup.