I am using *your* script (1 minute cron job) to restart Apache and Interchange whenever it fails to respond.From: Ron Phipps Sent: Tuesday, December 20, 2005 2:07 PM<snip>Are you using a script to restart your site or do you restart it by hand? If you restart it by hand could you please setup a cgi test domain and hit that site before you restart IC/Apache?
So it sounds like from the work you have done with a cgi test domain that Interchange fails to respond via tlink aswell.
BTW, I feel a bit bad that the subject of this thread is "mod_interchange..." when it may or may not be anything to do with mod_interchange - sorry Kevin :-o. I would post my reply with a new subject but that seems bad form now the thread is underway...should we change the subject of postings to this thread?
I have created a little perl script to try to emulate the awstats GET and xmlrpc POST requests but Interchange seems to cope fine with my script, returning status 200 (and no doubt returning the interchange missing.html page, although I haven't bothered checking the contents of the request response in the script). So I am still at a loss as to exactly what is causing IC to hang.Something in these worms is causing mod_interchange or IC to hang, but I'm not sure I know where to look from here. It'd be great if there was a script that recreated the actions of these worms, but I have not found one yet. If we could recreate the problem on demand then it'd be much easier to find a fix. Unfortunately right now we have to make a change then wait for the worm to attack again.
The only thing I am 100% sure about is that this worm (i..e variants of the Lupper worm) are definitely the culprit - each time, and very shortly before IC hangs I can always see in the log the following three GET requests:
/awstats/awstats.pl
/cgi-bin/awstats.pl
/cgi-bin/awstats/awstats.pl
or on a couple of occasions, the following 2 GET requests
/modules/Forums/admin/admin_styles.phpadmin_styles.php
/Forums/admin/admin_styles.phpadmin_styles.php
More detail in previous post to this thread:
http://www.icdevgroup.org/pipermail/interchange-users/2005-November/044359.html
Does anyone know of any communities where they would post such a script?
There is an analysis of the packets sent (by one of the Lupper variants) at: http://www.philippinehoneynet.org/charts_2005-11-11/awstats.htmlIt should be possible to reconstruct the GET and POST requests from this data, but unfortunately this page is unavailable at the moment - "The server is temporarily unable to service your request due to the site owner reaching his/her bandwidth limit. Please try again later."
Snip from earlier post to this thread by Kevin:
Thanks for posting the packet data. I'll use that to try to recreate the problem locally. I imagine I'll have to throttle the link and/or fire truck-loads of simultaneous requests to get the problem to show itself. If the problem can be recreated on demand then it can be found and fixed. I have an old P200 that I use for performance tests. Test time differences are amplified massively when running Interchange on a P200 with 128MB of memory. :-)
Kevin, did you get chance to do this?
OK, I have just updated our DropRequestList to include /xmlrpc.php. So, if this has stopped your site falling over it does look like it is the contents (or frequency) of xmlrpc.php POST requests that are causing the problem. I will let you know if our site also now stays up.I think what I'm going to do next is add those xmlrpc paths to either the ordinaryfilelist or the dropfilelist of mod_interchange so that the posts are not passed along to IC.The DropRequestList looks like this now in the interchange-handler section: DropRequestList /default.ida /x.ida /cmd.exe /root.exe /xmlrpc.php Since I implemented this, the site has been hit by the worm 6 times, but my script has not detected the site going down.
We have seen a similar pattern to you Ron - site didn't go down at all between 8 December and 16 December (during which time the site was very busy). However, since 16 December it has been brought down (and restarted by your script) a total of 20 times. I have checked the access log each time it went down and always find evidence of the 3 awstats GET requests.
I am surprised that not more Interchange sites have reported being affected by this on this mailing list. So far only 3 of us have reported experiencing this problem which would suggest that it is something peculiar to our installations, and yet I am sure it isn't. If anyone else is seeing the same problem with there Interchange site going down please post a brief reply to this thread - thanks.
I still think there is a problem somewhere, either in apache, mod_interchange or interchange, however I'm not sure how to go about finding the issue without an easy to reproduce case.
Ditto
I'll let you know if the DropRequestList stops the problem - I suppose that will at narrow down the cause to the xmlrpc POST requests... Thanks.Once someone can come up with a reproducible case I will look into a fix closer.
___________________________________________________________ NEW Yahoo! Cars - sell your car and browse thousands of new and used cars online! http://uk.cars.yahoo.com/
_______________________________________________ interchange-users mailing list suppressed http://www.icdevgroup.org/mailman/listinfo/interchange-users
Mail converted by mhonarc 2.6.15
This archive provided courtesy of JSW4.NET, Internet Hosting Services for Small Business.