Apache :: Security Challenge - Rejecting Specific Requests Without Blocking IP
Jan 21, 2014
I have been trying to solve a big problem for the last 2 weeks with one of our servers.
The client using our system (web based w/ apache and php) is a contact center firm. They have about 120 operators, all connect to our websever with the same IP.
We have been suffering DoS attacks from some of these operators. This are simple, browser attacks , namely 5 or 10 operators will just hold F5 key and bombard the server with requests when they shouldnt.
We did manage to produce a php protection which will recognize the multiple requests and blacklist the user, but its "too late" because the request have already been sent and processed by the webserver.
We use the user ID in the system to control who should be blacklisted, so this is all dependent on our own authentication.
Ideally, we need something EXACTLY like mod_evasive, but for rejecting single requests instead of blocking the IP. Exemplifying : if a user calls the same url, 5 times, in a 3 second spawn, we will reject every next request for 30 seconds, but only the requests by that user.
If the webserver can make any use of it, the user id is stored in a cookie.
I have a guy who can't get to any of the 100 or so virtual hosts on my RHEL3 server.
It's running the latest Apache RPM from RedHat. I also have mod_evasive and mod_security running.
Here's what I know. The guy *CAN* connect via SSH and FTP. The guy *CAN* see the default web page when he hits the IP in his web browser (e.g. he types [url]into the address bar on IE). But when he uses any of the host names on the server he *CAN NOT* see anything. He gets timeout errors.
His IP in NOT in ANY error logs, it's not in mod_evasive or mod_security, it's not in IPTABLES, it's not anywhere I can see.
I must be missing something. Anyone have any ideas?
What would be in front of Apache blocking his requests?
I maintain Java servlet applications on my hosting platform [hosting-q.com] and access the applications from another site [wiredpages.qisoftware.com]. Today, there was a demand problem which caused the hosting server to block access to requests from the other domain.
Do you know if there is an .htaccess directive that can perform this blocking or some sort of system trigger?
The thing is-- only the servlets requested from the external domain and not requests from the originating domain [or hosting domain] were blocked.
I am getting a lot of GET requests from different IPs to 4 nonexistent PHP files on my server. Is there any way to block the requests to avoid the resources use of apache that these requests are generating?
I have installed mod_security but Im not sure about how the block rule should be.
The requests are going to images/log.php, images/log2.php, images/log3.php and images/logi.php of one of the sites hosted on the server, is there any way to block there requests for a specific domain or path?
I have a dedicated box with softlayer and I have noticed at varying times the past few months that with sites we host, sometimes the connection times out (I'll try to access like 5 or 6 sites within 30 seconds or so and they all drop, then a minute later they load fine).
I opened a support ticket and they said it usually has to do with the # of requests Apache can handle, and that this can be modified. They stated they could: "tweak the apache configuration file in this server that can make it possible to handle more requests."
So my question is what should the # of requests be set to? (I'm not sure what it is now, but I assume whatever the default # is).
I have Apache 2.4.2, OpenSSL/1.0.1c, on Windows Web Server 2008 R2 (64 bits)
After 12 hours of heavier load, the SSL requests stopped working/being answered. However if you requested the same page via http instead of https, it worked fine. Restarting the Apache server fixes this, for a while. Again after a few hours of traffic, the https requests stopped working again. I checked the logs, and nothing notable, the mod_ssl entries just...
The site is called only by client developed with Delphi 2007 (CodeGear user-agent). Delphi client use THTTPRIO for sending HTTPS request to SOAP.
So I just upgraded Apache 2.2.22 to Apache 2.4.3 and made sure to go through all the options that had changed and update the conf file accordingly. This included adding the cache module for SSL and changing the SSLMutex option over to Mutex default ssl-cache. We also turned off SSLCompression due to the CRIME attack vulnerability.
We use apache strictly as a loadbalancer to 2 tomcat servers via mod_jk. Apache serves no static content at this time.
After being deployed, everything worked fine until later in the day. After 3 hours of heavier load (our site only takes significant traffic during business hours), the SSL requests stopped working/being answered. However if you requested the same page via http instead of https, it worked fine.
Restarting the Apache server fixes this, for a while. Again after a few hours of traffic, the https requests stopped working again. This time I turned the loglevel up to debug and restarted the Apache server.
As traffic slowed down it took another 6 or 7 hours before SSL requests stopped working again. I checked the logs, and nothing notable, the mod_ssl entries just... stopped. (I don't know for sure its ammount of traffic related, it just seems that way)
I have tried reproducing this in a lab, but have not been able to get it to happen on the lab server.
OS: Windows Server 2008 R2 Apache: 2.4.3 vc9 build with OpenSSL 0.9.8 downloaded from apachelounge.org Mod_JK Version 1.2.37 vc9 also downloaded from apachelounge.
I've spent the last several months working on a huge upgrade of a couple dozen websites. The upgrades include modifying Apache so that visitors who arrive at links pointing to mysite/World/New_York are redirected to mysite/world/new-york. In other words, all my links now default to lower case, and underscores are replaced with dashes.
Unfortunately, publishing it has been an endless series of disasters. My websites are now all crashed, and the server is unbelievably slow. It takes pages forever to load (if they load at all), and I can scarcely publish files online.So the following notice sent to me by my webhost got my attention.
IT appears your own server IP is making GET requests to Apache, causing excessive loading and causing service failures. On today's date, your IP made almost 6,000 connections to Apache:<br><br>
I have a little problem (on my Raspberry) with the maximum concurrent connections.When I open multiple tabs of a webpage which keeps persistent connections, apache is unable to serve more requests.Here is the (shortened) mod_info output (which also takes some time till there is a process kind enough to serve the request):
Code: Server Version: Apache/2.4.10 (Raspbian) OpenSSL/1.0.1k Server MPM: prefork 5 requests currently being processed, 9 idle workers
I have an Apache Server (2.4.3) and a Tomcat Server (7.0.36) and have some Java Applications deployed.Everything works fine, but when we start a quite long Ajax process, I see in my Java Application, that a Ajax request is received and starts processing - everything fine. But during processing of the first request, I see a second request starts after 5 minutes.
I currently have a web VPS hosted with FDCServers.net and after 5 days of switching to it i am getting massive HTTP requests. When i login to WHM and hit apache status i have many requests per second by multiple IP's that are going to pages that simple don't exist. Currently my hostname for the server is set at web-01.optical-hosting.com which is what the requests are being sent to. I am also having a DNS issue because when i put http://web-01.optical-hosting.com in the web browser it displays the first account's site under "list accounts" in cpanel. Can someone please help me fix both of these issue's? i will post an apache log in a second post as it is long. Also, these are from overseas. please someone help me with this i have Aim and Msn.
I'm looking to pass the entries to a web form, via Apache, to an external process (listening on a port say 4321) running on the same host as Apache.Is there a way to "coerce" Apache into doing this?
My Linux Server's Http Daemon (Apache) would stop serving websites ever so often, as soon as apache is restarted the error fixes iteself only to resurface within few hours.
The apache process would still be running i.e. apache does not die but no websites hosted on my server would be accessible from browser. And when this happens the apache logs do not log any http requests.
Instead when this happens all http requests to my server would be redirected to some weird Trojan website and my Norton Antivirus would show an Alert/Warning, for example; "Browser exploit at www.xxx.xxx was blocked" Risk Name: MSIE WebViewFolderIcon ActiveX Control BO
or another error like; "Auto-Protect has detected Trojan.Fakeavalert".
At first i thought the problem could be with my Laptop/ISP so i logged on to the server via SSH and opened try to open a website using command line "lynx mywebsite.com" and it shows following error; "Alert!: HTTP/1.0 503 Service Unavailable".
Now if i assume my laptop were to be infected, then as soon as i restart my apache and visit mywebsite.com eveything returns to normal with no such warnings. Why do i see those norton error messages only when apache is down with 503, and when apache is down with 503 how come the http requests always get redirected to some suspicious websites and nothing gets logged in apache error log?
I think my server is being attacked causing http to get unresponsive and thereafter http requests to my server are redirected to some malicious website, is this correct?
Also, i suspect this is a php script exploit as some customers have reported that google have blocked their website due to security reasons, i found <iframe> tage inserted in some php pages which i fixed.
Also, another thinh i noticed; when apache responds with the 503 it is referencing PHP 5.1.4 in the header response:
[root@]# curl -I xxx.xxx.xxx.xxx (my server ip) HTTP/1.0 503 Service Unavailable Server: Apache X-Powered-By: PHP/5.1.4 Retry-After: 20
I am running PHP 4.3.9m why does apache responds with PHP 5.1.4 when this 503 error surfaces?
Also, since my apache was dowan with 503 error a customer mailed in today saying; "It seems that my site www.xxxx.com is regularly down, and the winlogon virus is involved."
I suspect this is again due to the fact that http requests start getting redirected?
So I've got a problem where a small percentage of incoming requests are resulting in "400 bad request" errors and I could really use some input. At first I thought they were just caused by malicious spiders, scrapers, etc. but they seem to be legitimate requests.
I'm running Apache 2.2.15 and mod_perl2.
The first thing I did was turn on mod_logio and interestingly enough, for every request where this happens the request headers are between 8000-9000 bytes, whereas with most requests it's under 1000. Hmm.
There are a lot of cookies being set, and it's happening across all browsers and operating systems, so I assumed it had to be related to bad or "corrupted" cookies somehow - but it's not.
I added "%{Cookie}i" to my LogFormat directive hoping that would provide some clues, but as it turns out half the time the 400 error is returned the client doesn't even have a cookie. Darn.
Next I fired up mod_log_forensic hoping to be able to see ALL the request headers, but as luck would have it nothing is logged when it happens. I guess Apache is returning the 400 error before the forensic module gets to do its logging?
By the way, when this happens I see this in the error log:
request failed: error reading the headers
To me this says Apache doesn't like something about the raw incoming request, rather than a problem with our rewriting, etc. Or am I misunderstanding the error?
I'm at a loss where to go from here. Is there some other way that I can easily see all the request headers? I feel like that's the only thing that will possibly provide a clue as to what's going on.
I'm running Apache 2.4.4 on Windows Server 2008 R2. It's already happened many times that Apache stopped responding to requests. The last entry in the error.log:
[Wed Mar 27 06:22:07.043600 2013] [mpm_winnt:notice] [pid 1736:tid 256] AH00354: Child: Starting 64 worker threads. [Wed Mar 27 06:52:34.521200 2013] [mpm_winnt:error] [pid 1736:tid 1656] AH00326: Server ran out of threads to serve requests. Consider raising the ThreadsPerChild setting
I've been trying to figure out some IP blocking with no success. The environment is UNIX and Apache version is 2.2.22-14.
The site is on a hosted solution and doesn't have a firewall due to the virtualization software limitations. I've tried setting something similar to the following:
Code:
<Directory /home/username/mysite.com> #IP Blocks Deny from 1.2.3.1/24 Deny from 1.2.4.5 etc..
but with no success. I've also tried it in the <Location> tag with no success.
The way I'm testing this is editing the conf and then bouncing the apache server.
I'm trying to prevent unnecessary GET requests from being processed by my CMS that originate from mutating IP address locations. This is sucking up server resources when the request is processed by the app, and so if possible, I'd like to block them with HTACCESS so that the request is stopped before anything is intensively-processed.
What happens is that an IP address will make a GET request for, say, "blah/test" or "blah/test2" but nothing else (no site assets like images or CSS/JavaScript files or even other pages). After this request, another IP address will then make an equivalent kind of request, and so on, and so on... All of them have similar if not identical user agent strings but they're always worthless requests that do nothing but waste CPU and RAM. I'm assuming it's just some idiotic SPAM bot because of this.
I am having a problem with blocking bots using .htaccess. I think I tried all possible syntax variants, yet all the bots that I am blocking get HTTP 200 response instead of 403 (I can verify it using access log).
I am using Apache 2.4 running on Ubuntu 14.04.2 with Plesk 12.0.18.
My AllowOverride is set to allow the use of .htaccess files, so .htaccess file gets loaded: when I make an error in .htaccess sysntax I can see the error in the error log and the webpages don't load. Besides, I have some "Deny from [IP address]" directives in the .htaccess and I see that these IPs get HTTP 403 response when access my site.
I spent hours trying different variants of .htaccess syntax (see below) and neither seems to work...
variant 0:
SetEnvIfNoCase User-Agent LivelapBot bad_bot SetEnvIfNoCase User-Agent TurnitinBot bad_bot Order allow,deny Allow from all Deny from env=bad_bot
If I know the IP range that I want to block the best option is to block it with IPTABLES. This works well when you want to block entire countries. But what happens when you want to block specific IPs rather than ranges? Is iptables still more effective than "deny from [IP]" in .htaccess? I read that you don't want iptables to grow too big as it slows performance, but I guess it is still more effective than having big .htaccess..?
When it comes to blocking spam bots or referrers, robots.txt is just a suggestion for bots, when I looked at my traffic logs I noticed that most bots don't even look at robots.txt file. As far as I understand the only option here is to use .htaccess
1. I am currently using this in my .htaccess: SetEnvIfNoCase User-Agent *ahrefsbot* bad_bot=yes SetEnvIfNoCase Referer fbdownloader.com spammer=yes ... SetEnvIfNoCase Referer social-buttons.com spammer=yes Order allow,deny Allow from all Deny from env=spammer Deny from env=bad_bot
2. Apparently, there is another approach as per below: # Deny domain access to spammers RewriteEngine on RewriteBase / RewriteCond %{HTTP_USER_AGENT} queryseeker [OR] RewriteCond %{HTTP_REFERER} ^(www.)?.*(-|.)?adult(-|.).*$ [OR] ... RewriteCond %{HTTP_REFERER} ^(www.)?.*(-|.)?sex(-|.).*$ RewriteRule .* - [F,L]
Which approach is better #1 or #2? Any better alternative?
Finally, somebody suggested that you need to have both (as per example below). Is it true?
I have a website on a linux-server working fine with PHP/Apache. The page loads a lot of css/js/image-stuff (total 84 requests, 220k), it takes about 4 secs to load via internet.
Now I'm testing the same page locally on a Win7-64-system (Apache 2.2, PHP 5.4). The system is not very slow (8 GB RAM, SSD, i7-CPU), but loading the same page as above takes about 50 secs.
The Q is: What might be the problem?
- I turned off firewall and anti-virus.
- I used mod_status: 150 threads, max. 11 seems to be used during the loading of the page.
- I tried php5apache2_2.dll with TS-PHP 5.4 and mod_fcgid.so with NTS-PHP 5.4, but the loading-time kept almost the same.
Looking at the "network-tab" in FF or Chrome, I found that a lot of subqueries get a timing like this:
Blocking: 11.96 s Sending: 0 Waiting: 1 ms Receiving: 6ms
So the loss of time seem to be in the "blocking"-section. I first thought of something like "limited number of TCP-Connections", but as said above, on the same system the page is remotly loaded fast enough almost without these "blocking"-parts.
... and nothing works. My webserver gets hosed, and my otherwise working system gives me an error when I try to get a page from it. I am, by the way, using Mac 10.6.8, with Apache 2.2 on both server and client.
configure Apache server to handle users requests using condition based on where he come from so i can redirect him . what i need if user came with from site start with xn--* redirect him to virtual host and if he came from any other link then go to another virtual host
