GreenAsJade Posted August 21, 2018 Share Posted August 21, 2018 Hi All, For the longest time, I've been supporting the community with mods warehouse infrastructure, though I don't play anymore. This is not free, but it is on cheap plans so I haven't minded. Also the occasional kind soul has donated (I think I get about $5 per month this way, which doesn't cover costs). However, this infrastructure can't support bots or automated downloads. They put too much load on it, it runs out of allocation. For some reason, the last few months, we've been running out of allocations at CMx1 Mods Warehouse. I've ponied up a few more dollars this month to go to the next level and get it running again. And I've had a look in the logs. It appears that someone has an automatic script/bot that is trying to download everything, one mod a a time. This is not OK. The allocations we have can support people using it at a moderate rate, but not sustained bot usage. If this doesn't stop, I'll have to turn it off, or ask you all for the money to keep it going. In case it's of any use or interest, the IP address originating the requests is 94.130.18.159. Unlikely we could do anything with that, but just in case. GaJ 1 Quote Link to comment Share on other sites More sharing options...
sburke Posted August 21, 2018 Share Posted August 21, 2018 (edited) The only thing I can tell is it is out of the Ukraine. I ponied up some money even though I haven't touched CMx1 in a decade, hopefully it helps Still this kind of crap just irks me. Geo location Information of your IP address: 94.130.18.159 Country Ukraine () Region City Latitude 49, 32 Net-Range 77.0.0.0 - 95.255.255.255 Edited August 21, 2018 by sburke 0 Quote Link to comment Share on other sites More sharing options...
GreenAsJade Posted August 21, 2018 Author Share Posted August 21, 2018 It _may_ be innocent and well intentioned. If someone has set up something to pull out the mods and save them or publish them somewhere else, that could be a good thing. The only problem is the infrastructure doesn't support it. It could also be some strange way of hacking ... who knows. Thanks for the help ($)! GaJ 0 Quote Link to comment Share on other sites More sharing options...
Mord Posted August 21, 2018 Share Posted August 21, 2018 That must be why I couldn't get on the site last month. Mord. 0 Quote Link to comment Share on other sites More sharing options...
Wicky Posted August 21, 2018 Share Posted August 21, 2018 (edited) 6 hours ago, GreenAsJade said: In case it's of any use or interest, the IP address originating the requests is 94.130.18.159. Unlikely we could do anything with that, but just in case. 94.130.18.159's User Agent Strings ninja-crawler47.webmeup.com Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/) Block the Ip and range / Kindly suggest via robots.txt that it refrains from crawling / email webmeup.com asking them to stop. Problem is there are hundreds of similar 'SEO' webcrawlers operating from all over the world. Can't you make it so only registered users can access and download files? Edited August 21, 2018 by Wicky 0 Quote Link to comment Share on other sites More sharing options...
Wicky Posted August 21, 2018 Share Posted August 21, 2018 Some discussion on webmeup-crawler BLEXBot https://www.webmasterworld.com/search_engine_spiders/4847633.htm Not good - as I saw they use hetzner.de servers which are notorious for no good > http://www.webhostingtalk.com/showthread.php?t=1672855 The only way to protect the files is to make them unpublic to bots. 0 Quote Link to comment Share on other sites More sharing options...
GreenAsJade Posted August 21, 2018 Author Share Posted August 21, 2018 Thanks for hunting this information down. I'll see if I can do something about bots. 0 Quote Link to comment Share on other sites More sharing options...
A Canadian Cat Posted August 21, 2018 Share Posted August 21, 2018 11 hours ago, GreenAsJade said: For the longest time, I've been supporting the community with mods warehouse infrastructure, Thanks for that. 11 hours ago, GreenAsJade said: though I don't play anymore. 0 Quote Link to comment Share on other sites More sharing options...
GreenAsJade Posted August 22, 2018 Author Share Posted August 22, 2018 I'm taking some steps. I'm putting greenasjade.net behind Cloudflare, which will allow me to block crawler IPs that make a nuisance of themselves. It's a bit experimental (for me). If I break it along the way I'm sure you'll tell me GaJ 0 Quote Link to comment Share on other sites More sharing options...
Sequoia Posted August 23, 2018 Share Posted August 23, 2018 GAJ I just wanted to confirm the paypal button on the cmx1 site works. I couldn't find another. 0 Quote Link to comment Share on other sites More sharing options...
GreenAsJade Posted August 23, 2018 Author Share Posted August 23, 2018 It does! Thanks!!! 0 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.