crawler

seekport Crawler and “Search Engine” by Sistrix

by

Seekport is a German search engine, though the search engine itself is not that popular (778k Global Alexa rank as of January 2021), it’s far more likely you will see the seekport crawler in your logs. The seekport crawler identifies itself with the following user agent string: Mozilla/5.0 (compatible; Seekport Crawler; http://seekport.com/) I’ve witnessed the … Read more

10 GB in a 27 KB Gzip File [My Present To HTTP Scanners]

by

Here’s a gzip bomb I use to redirect http scanners and web scrapers to: 10G.gz Create a PHP file with the following: < ?php header('Content-Encoding: gzip'); echo file_get_contents('10G.gz'); Example: http://rehmann.co/gz-bomb.php How it works: A web-crawler or browser requests the page and sends the "accept-encoding: gzip, deflate, br" header. So long as gzip is accepted, the ... Read more