My server just went down for the third time in 3 days. I have the following log entry just prior to the crash and I need some help with identifying the problem as I do not understand the information.
Feb 16 09:52:13 server kernel: loop: loaded (max 8 devices)
Feb 16 09:52:16 server kernel: kjournald starting. Commit interval 5 seconds
Feb 16 09:52:16 server kernel: EXT3-fs warning: checktime reached, running e2fsck is recommended
Feb 16 09:52:16 server kernel: EXT3 FS 2.4-0.9.19, 19 August 2002 on loop(7,0), internal journal
Feb 16 09:52:16 server kernel: EXT3-fs: loop(7,0): 4 orphan inodes deleted
Feb 16 09:52:16 server kernel: EXT3-fs: recovery complete.
Feb 16 09:52:16 server kernel: EXT3-fs: mounted filesystem with ordered data mod e.
Feb 16 09:52:47 server lsb_log_message: failed
Feb 16 09:56:17 server kernel: ** SSH ** IN=eth0 OUT= MAC=00:12:3f:24:d5:d4:00:1
4:f2:c7:f1:80:08:00 SRC=58.163.33.202 DST=147.202.65.34 LEN=44 TOS=0x04 PREC=0x0
0 TTL=236 ID=63692 DF PROTO=TCP SPT=1765 DPT=22 WINDOW=16000 RES=0x00 SYN URGP=0
Code: Kernel Version2.6.18-4-686 (SMP) Distro Name Debian 4.0 Model Intel(R) Pentium(R) 4 CPU 2.40GHz I've installed the default apache 2, mysql, php using aptitude.
The server load is not at all high, around 1-1.5 max. The apache processes running are at an avg 60-80. It has 1GB ram and Disk Swap usage is 0kB
My server keeps rebooting every now and then. These are the messages I get in the ssh session just before the reboot..
Code: Message from syslogd@xxx at xxx kernel: Bad page state in process 'apache2' kernel: page:c17bc0a0 flags:0xc0000000 mapping:00800040 mapcount:0 count:0 kernel: Trying to fix it up, but a reboot is needed kernel: Backtrace:
I'm using Ubuntu 8.04 Server and during this week I had 2/3 unexpected reboots.
It seems like /var/log/messages is emptied after the reboot, so I don't know how to find out information before the reboot. At first I though of power outage, but I do have an UPS and the other servers were OK.
It seems that it happens always near the same time! I check the crontabs, I don't have any crontab near these times.
I check also the $last log, no one reboot the machine.
How can I find out what is happening? Might be any Kernel Panic error?
Recently, I have switched from Siteground hosting services and everything was fine. Now I have changed over to a dedicated server that resides at my house. I'm routing DNS through FreeDNS (freedns.afraid.org).
For a week or two, the server was fine. Recently, we started working on the website on the new server (posting new articles, changing the theme, adding some plugins, etc.) and it's steadily gotten worse and worse about, when we save changes to something, or post an article, the website will go offline. Eventually, the website will come back.
My question is: What is causing this? The actual server isn't shutting off, so It has to be something between my modem and the user. Is my modem just shutting off because it's overloaded? (that seems sort of improbable) Or, is it a DNS issue? (I have a Dynamic DNS system set up that is working fine)
I cannot figure out why my server is always dead when i wake up most mornings.
I am trying to figure out why my server is crashing nearly every night between 12am and 9am eastern time. When it crashes it is down all night until i wake up and have to reboot the damn server. I am wondering if cron jobs are taking up alot of the cpu or something because the server load isn't that much. here is my specs:
Sites: I am currently hosting 4,500 sites on the server.
The server load usually stays below 1 and is blazing fast. I just cannot figure out why it crashes when i sleep. I dont know if there is a virus or what. PHP mail is disabled so there would be no spamming crashing it. Here is my cron jobs:
I dont know much about cron jobs as I am new to this server hosting thing and just bought an established company. Maybe one of the cron jobs is causing it to crash in the middle of the night?
I have a FreeBSD server with Cpanel. It keeps crashing every few hours. Data centre swapped RAM, Chassie but no luck. Hard Drive was scanned and no errors found. I can't find anything in the logs (/var/logs/messages). Which direction should I be looking into?
One of my servers at FDC Servers is crashing (= kernel panic) every few days since we got it.
The specs of the machine are: C2D E6550, 2 GB
Linux .. 2.6.24.4-64.fc8 #1 SMP Sat Mar 29 09:15:49 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux
I opened several tickets with FDC, after running memtest and clocking down memory they say it's not a hardware issue but software related.
Quote:
I see that according to this ticket, memtest has been run without error and the memory was even slightly clocked down to avoid problems. It appears that the errors you are reporting are software related...
I already tried several kernels and different application versions.
this is getting out of control. I have a dedicated server that is "unmanaged" meaning, I manage it
I typically can make my way around a server and do most things - but in this case I'm stuck. I host a number of websites on this box and have went over a year with little problems. The past month or so it seems as though the server crashes daily or every other day. It will be running just fine, then all of a sudden the processes and loads will go out of control until the server is just unusable.
What do I need to provide here for you to be of any help to me? I watch the processes using "top -ci" and it's typically small output - but when it's getting ready to crash it's like a ton of processes get backed up and continue running.
as many of you know, FreeBSD is a stable system... I have many other FreeBSD servers (with the same kernel as this one) that doesn't have problems but this server keeps rebooting once or twice a day (EVERY DAY)
it's just a reboot... something very very similar to someone pushing the reset button
1) messages, security, auth or dmesg has no entries just before the reset, so the kernel is not getting aware the server is rebooting
2) the server comes back after around 10 minutes (reboot time + fsck)
this is happening for long time, so I compiled a new kernel... and the problem didn't stop
I request the datacenter techs to replace hardwares and they told me everything was replaced: motherboard, CPU, memories... and yesterday also the power suply so I have no other idea on what to do
in fact I have one... setting a nobreak in this server power suply for 2 or 3 days to see if the problem stops, but the datacenter didn't like this idea
this server is crashing after a few hours... it just got frozen... and after rebooted the server, i was looking at the /var/log/message logs and saw this ( you will see when system restart after the crash ):
Code: Feb 20 17:35:04 server kernel: grsec: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:13280] uid/euid:48/48 gid/egid:48/48, parent /us r/sbin/httpd[httpd:6180] uid/euid:48/48 gid/egid:48/48 Feb 20 17:41:40 server kernel: grsec: From 190.73.138.68: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:27459] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:20166] uid/euid:48/48 gid/egid:48/48 Feb 20 17:45:03 server kernel: grsec: signal 7 sent to /usr/bin/php[php:31710] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:8963] uid/euid:0/0 gid/egid:0/0 Feb 20 17:48:41 server kernel: grsec: From 87.219.205.218: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:11897] uid/euid:48/48 gid/e gid:48/48, parent /usr/sbin/httpd[httpd:8152] uid/euid:48/48 gid/egid:48/48 Feb 20 17:51:04 server kernel: grsec: From 85.58.139.135: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:28508] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:19918] uid/euid:48/48 gid/egid:48/48 Feb 20 17:51:58 server kernel: grsec: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:15615] uid/euid:48/48 gid/egid:48/48, parent /us r/sbin/httpd[httpd:2482] uid/euid:48/48 gid/egid:48/48 Feb 20 17:52:08 server kernel: grsec: From 166.114.104.42: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:4662] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:24468] uid/euid:48/48 gid/egid:48/48 Feb 20 17:52:38 server kernel: grsec: From 189.175.50.103: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:12497] uid/euid:48/48 gid/e gid:48/48, parent /usr/sbin/httpd[httpd:32213] uid/euid:48/48 gid/egid:48/48 Feb 20 17:54:32 server kernel: grsec: From 83.53.142.7: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:18556] uid/euid:48/48 gid/egid :48/48, parent /usr/sbin/httpd[httpd:22809] uid/euid:48/48 gid/egid:48/48 Feb 20 17:55:04 server kernel: grsec: signal 7 sent to /usr/bin/php[php:29694] uid/euid:502/502 gid/egid:502/502, parent /bin/bash[sh:30003] uid/euid:502/502 gid /egid:502/502 Feb 20 18:00:54 server kernel: grsec: From 189.141.26.82: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:10817] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:13549] uid/euid:48/48 gid/egid:48/48 Feb 20 18:01:07 server kernel: grsec: signal 7 sent to /usr/bin/php[php:20901] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:12242] uid/euid:0/0 gid/egid:0/0 Feb 20 18:03:06 server kernel: grsec: signal 7 sent to /usr/bin/php[php:9696] uid/euid:502/502 gid/egid:502/502, parent /bin/bash[sh:23721] uid/euid:502/502 gid/ egid:502/502 Feb 20 18:03:29 server kernel: grsec: From 68.26.197.159: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:917] uid/euid:48/48 gid/egid :48/48, parent /usr/sbin/httpd[httpd:20771] uid/euid:48/48 gid/egid:48/48 Feb 20 18:04:43 server kernel: grsec: From 87.219.88.132: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:10750] uid/euid:48/48 gid/eg id:48/48, parent /usr/sbin/httpd[httpd:4130] uid/euid:48/48 gid/egid:48/48 Feb 20 18:05:04 server kernel: grsec: From 189.167.128.26: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:14515] uid/euid:48/48 gid/e gid:48/48, parent /usr/sbin/httpd[httpd:2598] uid/euid:48/48 gid/egid:48/48 Feb 20 18:07:05 server kernel: grsec: signal 7 sent to /usr/bin/php[php:29589] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:7958] uid/euid:0/0 gid/egid:0/0 Feb 20 18:08:31 server kernel: grsec: From 88.64.181.89: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:15335] uid/euid:48/48 gid/egi d:48/48, parent /usr/sbin/httpd[httpd:27788] uid/euid:48/48 gid/egid:48/48 Feb 20 18:08:43 server kernel: grsec: From 201.244.116.46: signal 11 sent to /usr/www/htdocs/mywebsite.com_cgi/script/out.cgi[out.cgi:15217] uid/euid:48/48 gid/e gid:48/48, parent /usr/sbin/httpd[httpd:29545] uid/euid:48/48 gid/egid:48/48 Feb 20 18:17:34 server syslogd x.x.x: restart. Feb 20 18:17:34 server syslog: Iniciaci� de syslogd succeeded Feb 20 18:17:34 server kernel: klogd x.x.x, log source = /proc/kmsg started. Feb 20 18:17:34 server kernel: Linux version 2.x.xxgrs-bipiv-ipv4 (root@kernel.myserver.net) (gcc version xxxx) #1 SMP Tue Jan 31 17:34:40 CET 2006 Feb 20 18:17:34 server kernel: BIOS-provided physical RAM map: Feb 20 18:17:34 server kernel: BIOS-e820: 0000000000000000 - 000000000009c400 (usable) Feb 20 18:17:34 server kernel: BIOS-e820: 000000000009c400 - 00000000000a0000 (reserved) Feb 20 18:17:34 server kernel: BIOS-e820: 00000000000ea070 - 0000000000100000 (reserved) Feb 20 18:17:34 server kernel: BIOS-e820: 0000000000100000 - 000000007 Bold date are ( I think ) the crash, and the system booting... Any ideas about what can be causing the crash.. ? Is this kernel compiled with GRSecurity ? may that affect cgis ?
This is a fedora core server, xeon 3.2 GHZ x 4 procs using about 25 MBits per day.
I have a VPS from mediatemple.net, and I'm not sure what is going on, but Apache has stopped running every day in the morning and it just stopped running about 2 hours ago and I didn't catch it until now
From the error log, i'm showing:
Quote:
[emerg] (12)Cannot allocate memory: couldn't grab the accept mutex [alert] Child 3208 returned a Fatal error.../nApache is exciting! [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (43)Identifier removed: couldn't grab the accept mutex [emerg] (22)Invalid argument: couldn't release the accept mutex [emerg] (22)Invalid argument: couldn't release the accept mutex
Any ideas what might be causing this? just came home and my site was down again, and I've lost 15,000 pageviews already today.
what happened not sure really, but asked volumedrive to reboot the machine like an hour ago and it has still not come online, anyone know or has experience with VD and how long it usually takes them to reboot machine? Or is there any outages anyone is aware that VD is facing right now ?
Damn i got some upset users already ready to shoot the bunny, baah this proxy hosting is kinda harder than one would imagine
I've recently had problems where customers will upload PHP scripts that seem to use alot of CPU. I've got PRM installed but when a PHP script uses a lot of CPU, it doesn't seem to kill the processes or do anything to stop it crashing the server. I've checked the logs of PRM and it does kill some processes that use a lot of CPU/RAM though...
The ideal solution would be for PRM or something else to stop people being able to access the script causing excessive CPU/RAM usage. Even suspending for the reason of using excessive CPU/RAM would be sufficient.
For those interested the OS is CentOS 5.3 with cPanel 11, Apache and the latest PHP 5. Average load is always between 0.50 - 1.90.
My server had been crashing for while with Blue Screen of Death (BSOD) and bug check error code as Stop 0x00000050 PAGE_FAULT_IN_NONPAGED_AREA.It would literally stop by business till I reboot it again.So I tried pull up all information I could get and fix this.
Here is what I found-Possible causes:A faulty driver recently installed Faulty RAM Antivirus Corrupted NTFS file system
I checked the system logs and found errors related to NTFS. Well, my disk needed a chkdsk /r /f to fix this.Ran it at the command prompt and since it required a reboot to fix on the system drive (C, had to reboot. Came back successfully.It has been 14 days and it has not recurred.
I have a VPS , I recently upgraded because the server kept crashing due to lack of memory. I actually upgraded about two hours ago and the damn thing crashed again! I went into virtuozzo and it said I had 12 mb of memory left.
So I restarted the container, the site loads faster than ever, and says I have 16% of memory left
Assuming a linux server / apache / php / mysql setup
I'm wondering 1) how you can find out which php script is causing problems, i assume infinite looping, crashing a server and 2) are there any measures you can take (maybe in the php or apache configuration) that can prevent such a thing from happening in the first place, other than writing good code obviously?
I have had a node with Justedge since July last year, they seem great for the most part, until things go wrong and you need their help.
Here are their responses to reboot tickets which I have sent in, all critical, all with things like "Reboot Needed" "Server reboot" "Server down" as the headers, all with either "Emergency" or "Critical" as the priority.
Here are the response times, when I sent in the ticket and got the first reply. I thought it might of been an isolated incident at first that they took a while to respond to an urgent ticket like this but it has seemed to not be the case.
Ticket submitted -24 Jul 2008 06:40 AM Response - 24 Jul 2008 09:12 AM (Took 2hrs 31 minutes for a response)
Ticket submitted - 09 Jan 2009 02:46 AM Response - 09 Jan 2009 10:50 AM (Took 8hrs 4 minutes for a response)
Ticket submitted - 10 Jan 2009 04:13 PM Response - 10 Jan 2009 04:18 PM (Score, actually a quick one)
Ticket submitted - 10 Jan 2009 04:43 AM Response - 10 Jan 2009 09:03 AM (Took 4 hrs 20 minutes for a response)
Ticket submitted - 03 Apr 2009 07:04 PM Response - 04 Apr 2009 11:06 AM (Took 16 hrs 2 minutes for a response, luckily the server started working again in this time, aparently there were small power issues with something, although no detailed reason actually given)
I have a server with several sites hosted on it. Randomly, a site user will find that they can no longer connect to any of the sites on the server - but they can connect to the rest of the Internet. When this happens, the sites themselves are fine and everyone else can connect without any problems.
The only way the affected user can view sites on the server again is to reboot their router - at which point everything is fine.
why this would happen? I have a CentOS (Red Hat Enterprise) Linux server with DirectAdmin, Apache, PHP/MySQL - the usual. I'm thinking that there must be some kind of network setting which is allowing this to happen, but I can't for the life of me work out what it is.
I have a brand new A8i server from 1&1 that has crashed twice today and could only be rebooted through the 1&1 control panel (not Plesk). Out of the blue the server is not accessible from the Plesk CP and is not responding to pings. I have looked through the logs and do not see anything that stands out (I am not very Linux savvy). Is there somewhere specific that I can look that would tell me why it is locking up/crashing? I have only had the server about 2 weeks and am only running email on it (no websites besides webmail and Plesk CP). It is possible that there is a hardware issue, but I cannot have 1&1 check it until after hours tonight.
Server is an 8 core Intel Atom 2.6GHz with 8GB RAM.
Our server is running; Plesk 11.0.9 and CentOS 5.7 it has a Q8200 CPU @ 2.33GHz and 2GB of RAM. Now there are just two websites on the server plus a couple of redirects/forwarding domains, although lots of domains are still on the server but turned off in Plesk. Both websites are OSCommerce sites and I just need to keep these sites going until the end of the year when we will switch to our new Joomla based website.
We have seen an increasing number of server crashes and after various checks of the logs, fitting a new BIOS battery, check of the hardware by EasySpace who host the server, installation of ClamAV, LMD and RKHunter (which did find some Trojans and Suspect software), I have traced it down to some external Http activity that is taking all of my CPU time and RAM. Here is a screen capture of the Htop listing and when I killed these processes the CPU and RAM went back to normal. The problem is that I usually have to restart the HTTPD service and sometimes things get so bad that the server crashes and I have to request a power cycle.
Today on a server of mine mysql keeps crashing and I need to manually restart it over and over, the load as well keeps spiking quite high until I stop mysql for a few minutes. There isn't a extra load of traffic or anything as I have the same amount of users on several other servers doing more then this one.
How do I tell exactly whats happening from then how would I go about fixing it?
Basically it mysql crashes and needs to just be manually restarted via ssh.
