Nginx securiy problem

Thu Dec 3 12:22:22 MSK 2009

Hello,

I am using nginx for one year. 

Server info :
2 x 8 core -  16GB  (one for web server and other for mysql)
OS : linux RH 5
Nginx version : 0.8.x
web application :  vbulletin 3.8.4 PL1

I have experienced some security issues in last month. My server was under attack with 300Mbit. I don't know what is type of attack. But when I ask my service provider to add my server behind cisco guard, firewall could handle yhese attacks.

By the way my server located in softlayer.  So, they give this firewall only limited time (only 24 hours) adn thenyou have to ask again to add server behind firewall...

At these day, somebody (one of my forum member)  add some files to my server as attachment. I saw that this files contain virusus. I think these files botnet clients. I deleted this forum messages and attachment. (I think some of my other members download this files. :( )

But at that time my server is up with the help of cisco firewall. 
And I began to receive HACKING / MALICIOUS ACTIVITY complaint mails from diffirent locations and they claim that my IP address is attack their server.

below are some log lines that they sent :

#Nov 3 02:00:24 2009 .. Nov 3 02:33:14 2009
# Scan from xxx.xxx.xxx.xxx affecting at least
# 65 addresses targeting TCP:1024, TCP:3072.
#

#Nov 3 01:00:50 2009 .. Nov 3 01:59:00 2009
# Scan from xxx.xxx.xxx.xxx affecting at least
# 104 addresses targeting TCP:1024, TCP:3072.
#

#Nov 3 00:23:25 2009 .. Nov 3 00:59:55 2009
# Scan from xxx.xxx.xxx.xxx affecting at least
# 100 addresses targeting TCP:1024, TCP:3072.
#

#Nov 2 23:00:15 2009 .. Nov 2 23:59:58 2009
# Scan from xxx.xxx.xxx.xxx affecting at least
# 54 addresses targeting TCP:1024, TCP:3072.

UIDL Date Source Destination Port Protocole Nombre ASN Pays
4aefcca000000000 2009-11-02 22:52:03 xxx.xxx.xxx.xxx u-bordeaux.fr 3072 tcp 31 11897
4aefcca000000000 2009-11-02 22:40:53 xxx.xxx.xxx.xxx u-bordeaux.fr 1024 tcp 31 11897
4aef69ee00000000 2009-11-02 22:29:11 xxx.xxx.xxx.xxx lmd.ens.fr 3072 tcp 8 11897
4aefcca000000000 2009-11-02 22:52:03 xxx.xxx.xxx.xxx u-bordeaux.fr 3072 tcp 31 11897
4aefcca000000000 2009-11-02 22:40:53 xxx.xxx.xxx.xxx u-bordeaux.fr 1024 tcp 31 11897
4aef69ee00000000 2009-11-02 22:29:11 xxx.xxx.xxx.xxx lmd.ens.fr 3072 tcp 8 11897 

#Nov 20 06:00:59 2009 .. Nov 20 06:59:51 2009
# Scan from xxx.xxx.xxx.xxx affecting at least
# 58 addresses targeting TCP:1025, TCP:1057, TCP:1537, TCP:1569, TCP:16897, TCP:16929, TCP:17409, TCP:17441, TCP:17921, TCP:17953, TCP:18433, TCP:18465, TCP:18945, TCP:18977, TCP:19457, TCP:19489, TCP:19969, TCP:2049, TCP:2081, TCP:2561, TCP:2593, TCP:3073, TCP:3105, TCP:33, TCP:513, TCP:545.
#

#Nov 20 13:47:47 2009 .. Nov 20 13:59:51 2009
# Scan from xxx.xxx.xxx.xxx affecting at least
# 149 addresses targeting TCP:1, TCP:1025, TCP:1057, TCP:1537, TCP:1569, TCP:16385, TCP:16417, TCP:16897, TCP:16929, TCP:17409, TCP:17921, TCP:17953, TCP:18433, TCP:18465, TCP:18945, TCP:18977, TCP:19457, TCP:19489, TCP:19969, TCP:20001, TCP:2049, TCP:2081, TCP:2561, TCP:3073, TCP:3105, TCP:33, TCP:3585, TCP:3617, TCP:513, TCP:545.
#

Event Date Time, Destination IP, IP Protocol, Target Port, Issue Description, Source Port, Event Count
EventRecord: 20 Nov 2009 11:12:36, 67.34.x.x, 6, 16385, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:12:22, 156.99.x.x, 6, 2561, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:09:26, 64.128.x.x, 6, 3617, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:08:47, 83.170.x.x, 6, 16929, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:07:47, 24.220.x.x, 6, 20001, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:07:40, 173.15.x.x, 6, 19969, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:07:40, 173.15.x.x, 6, 19969, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:06:38, 156.99.x.x, 6, 3585, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:06:12, 194.85.x.x, 6, 20001, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:05:43, 194.85.x.x, 6, 16417, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:05:36, 156.99.x.x, 6, 3617, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:05:20, 64.128.x.x, 6, 19969, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:03:37, 84.12.x.x, 6, 3105, Research Pending , 80, 1
EventRecord: 20 Nov 2009 11:02:34, 84.12.x.x, 6, 16897, Research Pending , 80, 1

33:42.1 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.68, 1537, sbg.fmew.com -
47:31.9 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.71, 2561, mac.fmew.com -
49:40.6 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.5, 1, fmewservices.fmew.com -
51:56.5 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.27, 2593 -
53:23.7 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.37, 18433, jma.fmew.com -
54:37.5 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.42, 17953, mjt.fmew.com -
55:41.4 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.46, 16385, emp.fmew.com -
56:51.6 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.86, 16417 -
57:59.0 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.94, 18977 -
59:21.5 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.21, 1057 -
03:50.4 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.104, 2049 -
04:56.8 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.36, 1057 -
06:13.6 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.79, 16897 -
07:19.8 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.33, 1025 -
10:27.5 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.116, 3585 -
11:34.2 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.126, 17953 -
12:34.7 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.16, 16929 -
13:50.4 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.99, 19457 -
14:57.8 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.110, 545 -
16:15.6 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.13, 20001 -
17:17.1 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.27, 18465 -
20:41.4 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.77, 17409 -
21:52.4 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.81, 17953 -
24:24.6 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.92, 17441 -
29:41.8 TCP checksum error - xxx.xxx.xxx.xxx, 500, WAN - 67.151.154.44, 20001 -

The following is a list of types of activity that may appear in this
report:
BEAGLE BEAGLE3 BLASTER BOTNETS BOTS BRUTEFORCE
DAMEWARE DEFACEMENT DIPNET DNSBOTS MALWAREURL MYDOOM
NACHI PHATBOT PHISHING ROUTERS SCAN445 SCANNERS
SINIT SLAMMER SPAM SPYBOT TOXBOT

etc. ....

Like this tens of mail sent to me and softlayer abuse department.

And softlayer ask me to stop this activity or stop my server.
And I check my server with know security, system auditing tool and rootkit scanners. Rootkit Hunter, lynsis and chkrootkit.

nothing found.

Also third party management company audit my server and give me a report that my server is clean and make hardening on myserver. But they advise me switch back to apache (because they no experience with nginx)

After that I receive complaint mails again.

So, 3 days ago made a os reload, setup a clean system and I switched back to apache and complaint mails stop for 3 days.

But Apache couldn't handle request. my server load is very high over 100, sometimes over 300..
I lose my google indexes also my members complaint about unreachable site.

I want to switch back to nginx. But Softlayer warn me about if they receive this kind od abuse mails cut my server activities.

Have you ever been experiencing this kinf of situation ? What do you advise me ? (sorry for my english)

Best regards

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,27636,27636#msg-27636