I've been having problems for the past two weeks. Apache and MySQL kept crashing. I tweaked MySQL, and MySQL stopped crashing. But Apache still crashes, sometimes twice a day, sometimes once every few days. It's driving me crazy. It started around the same time I acquired a new customer, so it's possible the customer has some sort of script or something that is causing problems, but I'm not quite sure how to determine whether the problem is the server itself, or if it's the customer.
So here am I, asking you experts for whatever assistance you feel willing to render. Even if you only merely read it, I thank you for at the very least taking the time to read it. Thanks in advance for any and all assistance given. That being said, here's the information the "How To: Request help from the experts" sticky recommended giving. I also took the liberty of including httpd.conf and my.cnf. I apologize if that resulted in being too much information, and humbly ask a moderator to "white out", or so to speak, (or at least point out) the information that shouldn't be shown.
PHP 5.2.5 (cli) (built: Jan 19 2008 10:08:24)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
with the ionCube PHP Loader v3.1.32, Copyright (c) 2002-2007, by ionCube Ltd., and
with Xdebug v2.0.2-dev, Copyright (c) 2002-2007, by Derick Rethans
Results of "SELECT VERSION;" in MySQL:
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 1134
Server version: 5.0.54 Source distribution
Type 'help;' or 'h' for help. Type 'c' to clear the buffer.
mysql> SELECT VERSION();
| VERSION() |
| 5.0.54 |
1 row in set (0.00 sec)
Results of "ps auxf"
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1628 608 ? Ss 12:41 0:00 init  ....
i have custom dll built with Delphi that used to work on IIS 5/6/7. After i migrated to apache, DLL started to crash after an hour or two and i have to restart the server. The error was something about bad header. Since the developer says DLL is handling reponses OK and i can't get anything with debug logging on apache side can i run DLL as CFG/FastCGI (not that versed with apache so maybe this question does not make sense)?
I've got a complicated problem and I can't figure out what I need to edit to fix this.
I've been messing with the httpd.conf and php.ini files trying to optimize them as best I can. I've managed to get the load to stay pretty low all day, and everything seemed to be working fine. But then apache crashes.
(See screenshot for details)
httpd starts out with 70-150 VIRT memory, and it constantly goes up all day, and I've yet to see it go down. And, eventually it just crashes Apache! In the screenshot it's 502m, but I've seen that get as high as 800! And sometimes I even see a number like this "80376" no "m" or anything, just "80376"; I've no idea what that's about.
So what do I need to edit or tweak to fix this?
I'm on a cPanel linux server, Apache 2.2.8 and PHP 5.2.6.
It's been happening on both my old and new server, was really hoping this would be fixed.
Basically, the httpd/apache service crashes or goes down making domain names and sites unable to load; although cPanel/WHM still accessible. This would go on for about 5 minutes OR until I restart httpd in root.
sites simply do not load or just timeout after a period of time.
This has become extremely annoying for both me and my clients because it makes it seem like we have downtime.
Right now I am using the built-in cpanel backup feature to backup all the accounts on my server to an offsite FTP backup account I have on a weekly basis. The problem I am having is that everytime the backup runs, the load (thats what I'm guessing is the cause) is crashing apache/bind, causing sites to be unreachable until I can restart the services. What options do I have to resolve this issue? Currently I already have spri installed, and it does help out a bit, but its still not enough.
Today on a server of mine mysql keeps crashing and I need to manually restart it over and over, the load as well keeps spiking quite high until I stop mysql for a few minutes. There isn't a extra load of traffic or anything as I have the same amount of users on several other servers doing more then this one.
How do I tell exactly whats happening from then how would I go about fixing it?
Basically it mysql crashes and needs to just be manually restarted via ssh.
My web site keeps crashing for some reason like every 10 minutes. The site will be up for 10 minutes and then down for 10 minutes. Up for anotehr 10 minutes and then down for another 10 minutes.
The odd thing is that the server does not crash, the site itself does. I can access WHM during these periods of downtime and WHM says everything is perfect. All green check marks, low server load, low memory usage. Everything looks perfect.
On top of being able to access WHM during these periods of downtime, I can even access that green page from the IP, 123.456.789 (exampe ip) that says "Great success, Apache is working on your WHM/Cpanel server"
I keep getting a segmentation fault after a few hours when I leave lighttpd on. It doesn't seem like much resources are being used and its serving about 60mbit/s. There's nothing in the error or log file.
I'm using CentOS 2.6.18-92.1.13.el5 and lighttpd-1.5.0-r1992 Here's the config: server.modules = ( "mod_access", "mod_trigger_b4_dl", "mod_status", "mod_proxy_core", "mod_proxy_backend_fastcgi", "mod_uploadprogress" ) upload-progress.progress-url = "/progress" server.network-backend = "gthread-aio" server.max-fds = 15000 server.max-connections = 10000 server.event-handler = "linux-sysepoll" I've actually been having this issue since I've moved to lighttpd 1.5, 1.4 was working fine but I need the features in 1.5. I've tried changing the event handler and network backend to no avail.
How can I at least find out what's causing the error?
Recently, I have switched from Siteground hosting services and everything was fine. Now I have changed over to a dedicated server that resides at my house. I'm routing DNS through FreeDNS (freedns.afraid.org).
For a week or two, the server was fine. Recently, we started working on the website on the new server (posting new articles, changing the theme, adding some plugins, etc.) and it's steadily gotten worse and worse about, when we save changes to something, or post an article, the website will go offline. Eventually, the website will come back.
My question is: What is causing this? The actual server isn't shutting off, so It has to be something between my modem and the user. Is my modem just shutting off because it's overloaded? (that seems sort of improbable) Or, is it a DNS issue? (I have a Dynamic DNS system set up that is working fine)
I cannot figure out why my server is always dead when i wake up most mornings.
I am trying to figure out why my server is crashing nearly every night between 12am and 9am eastern time. When it crashes it is down all night until i wake up and have to reboot the damn server. I am wondering if cron jobs are taking up alot of the cpu or something because the server load isn't that much. here is my specs:
Sites: I am currently hosting 4,500 sites on the server.
The server load usually stays below 1 and is blazing fast. I just cannot figure out why it crashes when i sleep. I dont know if there is a virus or what. PHP mail is disabled so there would be no spamming crashing it. Here is my cron jobs:
I have a FreeBSD server with Cpanel. It keeps crashing every few hours. Data centre swapped RAM, Chassie but no luck. Hard Drive was scanned and no errors found. I can't find anything in the logs (/var/logs/messages). Which direction should I be looking into?
I run a drupal site on a VPS running Lighttpd with 128MB ram and 15GB disk space. Every so often the server crashes and I have no access to even reboot via the shell, I have to get my host to reboot the VPS. They say its most likely due to the server running out of memory. I would have no more than 5 users plus 5 guests online at the same time. So my question is this...
How can I find out exactly what is consuming the memory and causing the server to crash? I have looked at drupal logs and the server logs, but I'm no wiser as to whats causing it. What should I be looking for and what can I do to try and solve this?
One of my servers at FDC Servers is crashing (= kernel panic) every few days since we got it.
The specs of the machine are: C2D E6550, 2 GB
Linux .. 184.108.40.206-64.fc8 #1 SMP Sat Mar 29 09:15:49 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux
I opened several tickets with FDC, after running memtest and clocking down memory they say it's not a hardware issue but software related.
I see that according to this ticket, memtest has been run without error and the memory was even slightly clocked down to avoid problems. It appears that the errors you are reporting are software related...
I already tried several kernels and different application versions.
I've been running 3 Xen servers for the last few months in a production environment, and facing a strange problem with one of them.
Server 1 and 2 run a couple of DomU's and are running just fine.
Server 3 runs one DomU and takes care of all the backups for the DomU's of Server 1 and 2 The DomU on Server 3 is crashing every night, right when the backups are being made. DomU becomes inaccessible, but keeps appearing at xm list with state ------
When I try to connect to the console it wont let me connect. The only way to get the DomU back up is by using xm destroy and xm create again.
For the back-up I use NFS where the DomU on Server 3 is the host, and the other DomU's of Server 1 and 2 connect to it through an internal bridge.
Below you'll find some information about Server 3:
Hardware: HP ProLiant DL380 G5 10Gb Memory ( 8 x 146G SAS10k in RAID 5 )
this is getting out of control. I have a dedicated server that is "unmanaged" meaning, I manage it
I typically can make my way around a server and do most things - but in this case I'm stuck. I host a number of websites on this box and have went over a year with little problems. The past month or so it seems as though the server crashes daily or every other day. It will be running just fine, then all of a sudden the processes and loads will go out of control until the server is just unusable.
What do I need to provide here for you to be of any help to me? I watch the processes using "top -ci" and it's typically small output - but when it's getting ready to crash it's like a ton of processes get backed up and continue running.
as many of you know, FreeBSD is a stable system... I have many other FreeBSD servers (with the same kernel as this one) that doesn't have problems but this server keeps rebooting once or twice a day (EVERY DAY)
it's just a reboot... something very very similar to someone pushing the reset button
1) messages, security, auth or dmesg has no entries just before the reset, so the kernel is not getting aware the server is rebooting
2) the server comes back after around 10 minutes (reboot time + fsck)
this is happening for long time, so I compiled a new kernel... and the problem didn't stop
I request the datacenter techs to replace hardwares and they told me everything was replaced: motherboard, CPU, memories... and yesterday also the power suply so I have no other idea on what to do
in fact I have one... setting a nobreak in this server power suply for 2 or 3 days to see if the problem stops, but the datacenter didn't like this idea
this server is crashing after a few hours... it just got frozen... and after rebooted the server, i was looking at the /var/log/message logs and saw this ( you will see when system restart after the crash ):
Code: Feb 20 17:35:04 server kernel: grsec: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:13280] uid/euid:48/48 gid/egid:48/48, parent /us r/sbin/httpd[httpd:6180] uid/euid:48/48 gid/egid:48/48 Feb 20 17:41:40 server kernel: grsec: From 220.127.116.11: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:27459] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:20166] uid/euid:48/48 gid/egid:48/48 Feb 20 17:45:03 server kernel: grsec: signal 7 sent to /usr/bin/php[php:31710] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:8963] uid/euid:0/0 gid/egid:0/0 Feb 20 17:48:41 server kernel: grsec: From 18.104.22.168: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:11897] uid/euid:48/48 gid/e gid:48/48, parent /usr/sbin/httpd[httpd:8152] uid/euid:48/48 gid/egid:48/48 Feb 20 17:51:04 server kernel: grsec: From 22.214.171.124: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:28508] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:19918] uid/euid:48/48 gid/egid:48/48 Feb 20 17:51:58 server kernel: grsec: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:15615] uid/euid:48/48 gid/egid:48/48, parent /us r/sbin/httpd[httpd:2482] uid/euid:48/48 gid/egid:48/48 Feb 20 17:52:08 server kernel: grsec: From 126.96.36.199: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:4662] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:24468] uid/euid:48/48 gid/egid:48/48 Feb 20 17:52:38 server kernel: grsec: From 188.8.131.52: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:12497] uid/euid:48/48 gid/e gid:48/48, parent /usr/sbin/httpd[httpd:32213] uid/euid:48/48 gid/egid:48/48 Feb 20 17:54:32 server kernel: grsec: From 184.108.40.206: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:18556] uid/euid:48/48 gid/egid :48/48, parent /usr/sbin/httpd[httpd:22809] uid/euid:48/48 gid/egid:48/48 Feb 20 17:55:04 server kernel: grsec: signal 7 sent to /usr/bin/php[php:29694] uid/euid:502/502 gid/egid:502/502, parent /bin/bash[sh:30003] uid/euid:502/502 gid /egid:502/502 Feb 20 18:00:54 server kernel: grsec: From 220.127.116.11: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:10817] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:13549] uid/euid:48/48 gid/egid:48/48 Feb 20 18:01:07 server kernel: grsec: signal 7 sent to /usr/bin/php[php:20901] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:12242] uid/euid:0/0 gid/egid:0/0 Feb 20 18:03:06 server kernel: grsec: signal 7 sent to /usr/bin/php[php:9696] uid/euid:502/502 gid/egid:502/502, parent /bin/bash[sh:23721] uid/euid:502/502 gid/ egid:502/502 Feb 20 18:03:29 server kernel: grsec: From 18.104.22.168: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:917] uid/euid:48/48 gid/egid :48/48, parent /usr/sbin/httpd[httpd:20771] uid/euid:48/48 gid/egid:48/48 Feb 20 18:04:43 server kernel: grsec: From 22.214.171.124: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:10750] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:4130] uid/euid:48/48 gid/egid:48/48 Feb 20 18:05:04 server kernel: grsec: From 126.96.36.199: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:14515] uid/euid:48/48 gid/e gid:48/48, parent /usr/sbin/httpd[httpd:2598] uid/euid:48/48 gid/egid:48/48 Feb 20 18:07:05 server kernel: grsec: signal 7 sent to /usr/bin/php[php:29589] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:7958] uid/euid:0/0 gid/egid:0/0 Feb 20 18:08:31 server kernel: grsec: From 188.8.131.52: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:15335] uid/euid:48/48 gid/egi d:48/48, parent /usr/sbin/httpd[httpd:27788] uid/euid:48/48 gid/egid:48/48 Feb 20 18:08:43 server kernel: grsec: From 184.108.40.206: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:15217] uid/euid:48/48 gid/e gid:48/48, parent /usr/sbin/httpd[httpd:29545] uid/euid:48/48 gid/egid:48/48 Feb 20 18:17:34 server syslogd x.x.x: restart. Feb 20 18:17:34 server syslog: Iniciaci� de syslogd succeeded Feb 20 18:17:34 server kernel: klogd x.x.x, log source = /proc/kmsg started. Feb 20 18:17:34 server kernel: Linux version 2.x.xxgrs-bipiv-ipv4 (firstname.lastname@example.org) (gcc version xxxx) #1 SMP Tue Jan 31 17:34:40 CET 2006 Feb 20 18:17:34 server kernel: BIOS-provided physical RAM map: Feb 20 18:17:34 server kernel: BIOS-e820: 0000000000000000 - 000000000009c400 (usable) Feb 20 18:17:34 server kernel: BIOS-e820: 000000000009c400 - 00000000000a0000 (reserved) Feb 20 18:17:34 server kernel: BIOS-e820: 00000000000ea070 - 0000000000100000 (reserved) Feb 20 18:17:34 server kernel: BIOS-e820: 0000000000100000 - 000000007 Bold date are ( I think ) the crash, and the system booting... Any ideas about what can be causing the crash.. ? Is this kernel compiled with GRSecurity ? may that affect cgis ?
This is a fedora core server, xeon 3.2 GHZ x 4 procs using about 25 MBits per day.
My server just went down for the third time in 3 days. I have the following log entry just prior to the crash and I need some help with identifying the problem as I do not understand the information.
Feb 16 09:52:13 server kernel: loop: loaded (max 8 devices) Feb 16 09:52:16 server kernel: kjournald starting. Commit interval 5 seconds Feb 16 09:52:16 server kernel: EXT3-fs warning: checktime reached, running e2fsck is recommended Feb 16 09:52:16 server kernel: EXT3 FS 2.4-0.9.19, 19 August 2002 on loop(7,0), internal journal Feb 16 09:52:16 server kernel: EXT3-fs: loop(7,0): 4 orphan inodes deleted Feb 16 09:52:16 server kernel: EXT3-fs: recovery complete. Feb 16 09:52:16 server kernel: EXT3-fs: mounted filesystem with ordered data mod e. Feb 16 09:52:47 server lsb_log_message: failed Feb 16 09:56:17 server kernel: ** SSH ** IN=eth0 OUT= MAC=00:12:3f:24:d5:d4:00:1 4:f2:c7:f1:80:08:00 SRC=220.127.116.11 DST=18.104.22.168 LEN=44 TOS=0x04 PREC=0x0 0 TTL=236 ID=63692 DF PROTO=TCP SPT=1765 DPT=22 WINDOW=16000 RES=0x00 SYN URGP=0
I have a VPS from mediatemple.net, and I'm not sure what is going on, but Apache has stopped running every day in the morning and it just stopped running about 2 hours ago and I didn't catch it until now
From the error log, i'm showing:
[emerg] (12)Cannot allocate memory: couldn't grab the accept mutex [alert] Child 3208 returned a Fatal error.../nApache is exciting! [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (22)Invalid argument: couldn't release the accept mutex [emerg] (22)Invalid argument: couldn't release the accept mutex
Any ideas what might be causing this? just came home and my site was down again, and I've lost 15,000 pageviews already today.
Lately on one of my servers I noticed everytime I try to modify an account or quota, the quotaon process goes to 100% cpu, server load gets to about 30, and if I tried to modify multiple quotas it would basically become unresponsive.
It all seemed to have started after stage2 upgrade.
I havent seen any errors from the filesystem or even any errors in messages after it does this.
I've recently had problems where customers will upload PHP scripts that seem to use alot of CPU. I've got PRM installed but when a PHP script uses a lot of CPU, it doesn't seem to kill the processes or do anything to stop it crashing the server. I've checked the logs of PRM and it does kill some processes that use a lot of CPU/RAM though...
The ideal solution would be for PRM or something else to stop people being able to access the script causing excessive CPU/RAM usage. Even suspending for the reason of using excessive CPU/RAM would be sufficient.
For those interested the OS is CentOS 5.3 with cPanel 11, Apache and the latest PHP 5. Average load is always between 0.50 - 1.90.