Apache :: Server Ran Out Of Threads To Serve Requests - ThreadsPerChild Settings
Mar 27, 2013
I'm running Apache 2.4.4 on Windows Server 2008 R2. It's already happened many times that Apache stopped responding to requests. The last entry in the error.log:
[Wed Mar 27 06:22:07.043600 2013] [mpm_winnt:notice] [pid 1736:tid 256] AH00354: Child: Starting 64 worker threads.
[Wed Mar 27 06:52:34.521200 2013] [mpm_winnt:error] [pid 1736:tid 1656] AH00326: Server ran out of threads to serve requests. Consider raising the ThreadsPerChild setting
I have a little problem (on my Raspberry) with the maximum concurrent connections.When I open multiple tabs of a webpage which keeps persistent connections, apache is unable to serve more requests.Here is the (shortened) mod_info output (which also takes some time till there is a process kind enough to serve the request):
Code: Server Version: Apache/2.4.10 (Raspbian) OpenSSL/1.0.1k Server MPM: prefork 5 requests currently being processed, 9 idle workers
I need to serve domain subfolder from different Apache 2 server.
1. Squid 3 reverse proxy is pointin to 80.241.222.151 for domain [URL] .... 2. Apache 2 server is on 80.241.222.151 with all files in the root folder so all requests are served from here 3. Now im creating new server at 80.241.222.149 with all files from [URL] ... transferred to the root 4. On Squid 3 server [URL] .... is still pointing to 80.241.222.151 5. But now i need to call [URL] .... from 80.241.222.149 in the root.
What is the example to do this? mod_proxy on Apache 2 server at 80.241.222.151?
I've spent the last several months working on a huge upgrade of a couple dozen websites. The upgrades include modifying Apache so that visitors who arrive at links pointing to mysite/World/New_York are redirected to mysite/world/new-york. In other words, all my links now default to lower case, and underscores are replaced with dashes.
Unfortunately, publishing it has been an endless series of disasters. My websites are now all crashed, and the server is unbelievably slow. It takes pages forever to load (if they load at all), and I can scarcely publish files online.So the following notice sent to me by my webhost got my attention.
IT appears your own server IP is making GET requests to Apache, causing excessive loading and causing service failures. On today's date, your IP made almost 6,000 connections to Apache:<br><br>
I currently have a web VPS hosted with FDCServers.net and after 5 days of switching to it i am getting massive HTTP requests. When i login to WHM and hit apache status i have many requests per second by multiple IP's that are going to pages that simple don't exist. Currently my hostname for the server is set at web-01.optical-hosting.com which is what the requests are being sent to. I am also having a DNS issue because when i put http://web-01.optical-hosting.com in the web browser it displays the first account's site under "list accounts" in cpanel. Can someone please help me fix both of these issue's? i will post an apache log in a second post as it is long. Also, these are from overseas. please someone help me with this i have Aim and Msn.
In the last day or two I've been having issues with memory on one of by boxes.
Something eats it all up, so the OS starts swapping, the I/O wait shoots up, and soon the load is up in the hundreds and the thing is totally useless.
During the day today I've tracked the something to occational apache processes. It seems that occationally a thread is started which uses upwards of 150M of memory. These threads are obvioudly doing something heavyweight and take a while to complete. When I get a few of them running together it soons finishes off all my available memory. Below is an extract from top when a couple of these threads are running.
Code: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10230 nobody 25 0 197m 27m 9392 R 45.5 2.9 0:03.27 /usr/local/apache/bin/httpd -DSSL 10231 nobody 25 0 197m 25m 8376 R 52.8 2.7 0:01.60 /usr/local/apache/bin/httpd -DSSL I was wondering if anyone could help out by suggesting some ways of tracking down what page/script it which uses so much memory. It's a cPanel server so it's not really practical to tail -f the apache logs (not knowing which account it is means I don't know which file to watch).
My Linux Server's Http Daemon (Apache) would stop serving websites ever so often, as soon as apache is restarted the error fixes iteself only to resurface within few hours.
The apache process would still be running i.e. apache does not die but no websites hosted on my server would be accessible from browser. And when this happens the apache logs do not log any http requests.
Instead when this happens all http requests to my server would be redirected to some weird Trojan website and my Norton Antivirus would show an Alert/Warning, for example; "Browser exploit at www.xxx.xxx was blocked" Risk Name: MSIE WebViewFolderIcon ActiveX Control BO
or another error like; "Auto-Protect has detected Trojan.Fakeavalert".
At first i thought the problem could be with my Laptop/ISP so i logged on to the server via SSH and opened try to open a website using command line "lynx mywebsite.com" and it shows following error; "Alert!: HTTP/1.0 503 Service Unavailable".
Now if i assume my laptop were to be infected, then as soon as i restart my apache and visit mywebsite.com eveything returns to normal with no such warnings. Why do i see those norton error messages only when apache is down with 503, and when apache is down with 503 how come the http requests always get redirected to some suspicious websites and nothing gets logged in apache error log?
I think my server is being attacked causing http to get unresponsive and thereafter http requests to my server are redirected to some malicious website, is this correct?
Also, i suspect this is a php script exploit as some customers have reported that google have blocked their website due to security reasons, i found <iframe> tage inserted in some php pages which i fixed.
Also, another thinh i noticed; when apache responds with the 503 it is referencing PHP 5.1.4 in the header response:
[root@]# curl -I xxx.xxx.xxx.xxx (my server ip) HTTP/1.0 503 Service Unavailable Server: Apache X-Powered-By: PHP/5.1.4 Retry-After: 20
I am running PHP 4.3.9m why does apache responds with PHP 5.1.4 when this 503 error surfaces?
Also, since my apache was dowan with 503 error a customer mailed in today saying; "It seems that my site www.xxxx.com is regularly down, and the winlogon virus is involved."
I suspect this is again due to the fact that http requests start getting redirected?
I'm trying to redirect some threads in vbulletin from full version to archive version, I just have some problems and I have not been able to resolve it.
we recently switched from Microsoft ISA to Apache as a proxy to redirect customers to computers in DMZ. However, since the switch, our system always down due to the threading issue below
I have installed Apache 2.0.64(Win32) and after repeated tries at adjusting the httpd.conf file, it still is refusing to serve up CGI. So far I have, following the Apache documentation, set my designated CGI directory with AddHandler directive, Option +ExecCGI enabled. Also have added the ScriptAlias directive as per instructions. These directives should work in conjuction with modules mod_alias and mod_cgi, both of which I have made sure were set to be loaded. The log files show no errors. There are no syntax errors being pushed out at start-up. The server stops and starts upon commands, with no problems,
I will post some excerpts from my httpd.conf in the pastebin as soon as it gets up & running--right now the authentication icon is missing.I have Active Perl 5.16 running on this machine, but my Apache download is the binary without SSL and with msi installer.
I am using the latest version of Apache on an Windows XP machine
When my web service is down for maintenance, since Apache is will still be up and running, I would like for Apache to serve an xml file as a response for the appropriate request. I have three operations available, makePayment, calculateFee, and voidPayment.
Is it possible to have Apache determine what type of request is made for example if I have an xml error page for each operation; how will Apache know which xml file to serve based on the operation request from the client
To make it more clear: What is the best practice for modifying apache to know what request is being made in order to serve the appropriate xml file?
I have a dedicated box with softlayer and I have noticed at varying times the past few months that with sites we host, sometimes the connection times out (I'll try to access like 5 or 6 sites within 30 seconds or so and they all drop, then a minute later they load fine).
I opened a support ticket and they said it usually has to do with the # of requests Apache can handle, and that this can be modified. They stated they could: "tweak the apache configuration file in this server that can make it possible to handle more requests."
So my question is what should the # of requests be set to? (I'm not sure what it is now, but I assume whatever the default # is).
I have Apache 2.4.2, OpenSSL/1.0.1c, on Windows Web Server 2008 R2 (64 bits)
After 12 hours of heavier load, the SSL requests stopped working/being answered. However if you requested the same page via http instead of https, it worked fine. Restarting the Apache server fixes this, for a while. Again after a few hours of traffic, the https requests stopped working again. I checked the logs, and nothing notable, the mod_ssl entries just...
The site is called only by client developed with Delphi 2007 (CodeGear user-agent). Delphi client use THTTPRIO for sending HTTPS request to SOAP.
So I just upgraded Apache 2.2.22 to Apache 2.4.3 and made sure to go through all the options that had changed and update the conf file accordingly. This included adding the cache module for SSL and changing the SSLMutex option over to Mutex default ssl-cache. We also turned off SSLCompression due to the CRIME attack vulnerability.
We use apache strictly as a loadbalancer to 2 tomcat servers via mod_jk. Apache serves no static content at this time.
After being deployed, everything worked fine until later in the day. After 3 hours of heavier load (our site only takes significant traffic during business hours), the SSL requests stopped working/being answered. However if you requested the same page via http instead of https, it worked fine.
Restarting the Apache server fixes this, for a while. Again after a few hours of traffic, the https requests stopped working again. This time I turned the loglevel up to debug and restarted the Apache server.
As traffic slowed down it took another 6 or 7 hours before SSL requests stopped working again. I checked the logs, and nothing notable, the mod_ssl entries just... stopped. (I don't know for sure its ammount of traffic related, it just seems that way)
I have tried reproducing this in a lab, but have not been able to get it to happen on the lab server.
OS: Windows Server 2008 R2 Apache: 2.4.3 vc9 build with OpenSSL 0.9.8 downloaded from apachelounge.org Mod_JK Version 1.2.37 vc9 also downloaded from apachelounge.
I have an Apache Server (2.4.3) and a Tomcat Server (7.0.36) and have some Java Applications deployed.Everything works fine, but when we start a quite long Ajax process, I see in my Java Application, that a Ajax request is received and starts processing - everything fine. But during processing of the first request, I see a second request starts after 5 minutes.
I'm looking to pass the entries to a web form, via Apache, to an external process (listening on a port say 4321) running on the same host as Apache.Is there a way to "coerce" Apache into doing this?
So I've got a problem where a small percentage of incoming requests are resulting in "400 bad request" errors and I could really use some input. At first I thought they were just caused by malicious spiders, scrapers, etc. but they seem to be legitimate requests.
I'm running Apache 2.2.15 and mod_perl2.
The first thing I did was turn on mod_logio and interestingly enough, for every request where this happens the request headers are between 8000-9000 bytes, whereas with most requests it's under 1000. Hmm.
There are a lot of cookies being set, and it's happening across all browsers and operating systems, so I assumed it had to be related to bad or "corrupted" cookies somehow - but it's not.
I added "%{Cookie}i" to my LogFormat directive hoping that would provide some clues, but as it turns out half the time the 400 error is returned the client doesn't even have a cookie. Darn.
Next I fired up mod_log_forensic hoping to be able to see ALL the request headers, but as luck would have it nothing is logged when it happens. I guess Apache is returning the 400 error before the forensic module gets to do its logging?
By the way, when this happens I see this in the error log:
request failed: error reading the headers
To me this says Apache doesn't like something about the raw incoming request, rather than a problem with our rewriting, etc. Or am I misunderstanding the error?
I'm at a loss where to go from here. Is there some other way that I can easily see all the request headers? I feel like that's the only thing that will possibly provide a clue as to what's going on.
I have been trying to solve a big problem for the last 2 weeks with one of our servers.
The client using our system (web based w/ apache and php) is a contact center firm. They have about 120 operators, all connect to our websever with the same IP.
We have been suffering DoS attacks from some of these operators. This are simple, browser attacks , namely 5 or 10 operators will just hold F5 key and bombard the server with requests when they shouldnt.
We did manage to produce a php protection which will recognize the multiple requests and blacklist the user, but its "too late" because the request have already been sent and processed by the webserver.
We use the user ID in the system to control who should be blacklisted, so this is all dependent on our own authentication.
Ideally, we need something EXACTLY like mod_evasive, but for rejecting single requests instead of blocking the IP. Exemplifying : if a user calls the same url, 5 times, in a 3 second spawn, we will reject every next request for 30 seconds, but only the requests by that user.
If the webserver can make any use of it, the user id is stored in a cookie.
I'm trying to prevent unnecessary GET requests from being processed by my CMS that originate from mutating IP address locations. This is sucking up server resources when the request is processed by the app, and so if possible, I'd like to block them with HTACCESS so that the request is stopped before anything is intensively-processed.
What happens is that an IP address will make a GET request for, say, "blah/test" or "blah/test2" but nothing else (no site assets like images or CSS/JavaScript files or even other pages). After this request, another IP address will then make an equivalent kind of request, and so on, and so on... All of them have similar if not identical user agent strings but they're always worthless requests that do nothing but waste CPU and RAM. I'm assuming it's just some idiotic SPAM bot because of this.
I am running a web server (Apache 2.2) on a (production)windows 2008 server. I am encountering a little trouble. I have the domain wheatondev-uk.com pointing to this (production) server. This works fine. I have another domain hosted on another(developement) server. This is domain is wheaton-uk.com I want to have the domain wheaton-uk.com point to the production server. However when i change the DNS settings at network solutions by that I mean i change the settings of wheaton-uk.com to point to the IP of the production server...it times out when trying to go to wheaton-uk.com. If I switch them back to the original DNS settings, all the domains load up fine. I feel like this is an apache thing and I need to change something in the conf files.
my server is still effed up from the MPack attack that I received.
I just received the following email, does anyone know what this means or how it could be done? The client IP is mine, so some how my server is sending that request?
I was able to successfully delete all the files, but how do I now get rid of the directories themselves? When I do: rm -fr "/arcade/images/. /" and then locate ". " I still get:
There seems to be some problem with my server, none of the websites hosted on my server are accessible, the http requests either return a blank page or a page with a red quare on the upper left hand corner.
I am not sure if this is some kind of infection or DNS problem or a problem with memory apache is taking up as i have thousands of virtualhost entries in my access log accumulated over the years out of which only a few 100 websites i am serving presently, but never deleted the non-exitent virtualhost blocks.
At times the websites are opening but most of the times they are not. And when they do not open my http requets are not logged in apacha access log.
Even the customers have reported the same problem.
Also, just four days back i had a strange issue where all http requests to my server would take me to [url].
I can SSH to server, and everything else is working fine.
I just installed Windows 2K3 web on a dual xeon (hyperthreading).
When I check out the cpu threads in the "task manager", it only shows 2 cpu threads.
I have always installed win 2K3 std and it shows the 4 cpu threads. So my question is weather the web edition will only show the 2 cpu threads or am I missing something.
