My server is constantly crashing (halting to dead) and needing reboot literally every few hours. I cannot trace the cause of this whatsoever. Please help out.
CPU/Memory/MySQL Usage shows no accounts in red or yellow zone ....
I have in my possession a new server which is running cPanel. For some unknown reason it keeps crashing about every 4-6 hours where I must get a remote reboot done.
Its starting to annoy me that I'm unable connect to anything. What I suspect is high disk red/write. As what I'm finding is over 10k blocks being written per second with only 300 being read per second. I also am not seeing any bandwidth out/in usage being high.
Hi over the last week ive been having numerous problems with hosting accounts on 2 different servers which has lead me to think that my 'security' is not 'secure' and a malicious user is at play. im in the uk on broadband on a private connection to the internet - no-one else should be sharing this connection. This is the traceroute from my connection at home to the server ive had the most problems with - is this normal?
Traceroute has started ...
I have a dedicated server with NyNOC and earlier on today the server dissapeared off the net, wouldn't respond to ssh, http etc, nor pings
I submitted a ticket and the server got rebooted, went through the FSCK check and came back up ok, but now I'm concerned as to what caused it
Andy the technician said he couldn't find out the reason why and neither can I
I can only put it down to PERHAPS faulty hardware somewhere?
The /var/log/messages file doesn't say anything useful, the last entries show..
Quote:
May 27 06:41:17 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:xxxxxxxx SRC=xxxxxxxx DST=255.255.255.255 LEN=153 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=8778 DPT=11777 LEN=133
May 27 06:56:43 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:xxxxxxxx SRC=xxxxxxxxx DST=255.255.255.255 LEN=153 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=8778 DPT=11777 LEN=133
May 27 07:02:56 server kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:14:d1:14:a7:c0:00xxxxxxxxx SRC=xxxxxxxxx DST=xxxxxxxx LEN=622 TOS=0x00 PREC=0x00 TTL=48 ID=0 DF PROTO=UDP SPT=54284 DPT=1027 LEN=602
May 27 07:03:13 server kernel: Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=00:14:d1:14:a7:c0:xxxxxxx SRC=xxxxxxxx DST=xxxxxxxx LEN=48 TOS=0x00 PREC=0x00 TTL=117 ID=47647 PROTO=TCP SPT=21173 DPT=22 WINDOW=65535 RE$
May 27 13:01:45 server syslogd 1.4.1: restart.
May 27 13:01:45 server kernel: klogd 1.4.1, log source = /proc/kmsg started.
May 27 13:01:45 server kernel: Linux version 2.6.18-128.1.10.el5PAE (mockbuild@builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Thu May 7 11:14:31 EDT 2009
May 27 13:01:45 server kernel: BIOS-provided physical RAM map:
May 27 13:01:45 server kernel: BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
May 27 13:01:45 server kernel: BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
May 27 13:01:45 server kernel: BIOS-e820: 00000000000e7000 - 0000000000100000 (reserved)
May 27 13:01:45 server kernel: BIOS-e820: 0000000000100000 - 00000000affb0000 (usable)
May 27 13:01:45 server kernel: BIOS-e820: 00000000affb0000 - 00000000affc0000 (ACPI data)
May 27 13:01:45 server kernel: BIOS-e820: 00000000affc0000 - 00000000afff0000 (ACPI NVS)
May 27 13:01:45 server kernel: BIOS-e820: 00000000afff0000 - 00000000b0000000 (reserved)
May 27 13:01:45 server kernel: BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
May 27 13:01:45 server kernel: BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
May 27 13:01:45 server kernel: BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
May 27 13:01:45 server kernel: BIOS-e820: 0000000100000000 - 0000000140000000 (usable)
May 27 13:01:45 server kernel: 4224MB HIGHMEM available.
May 27 13:01:45 server kernel: 896MB LOWMEM available.
May 27 13:01:45 server kernel: found SMP MP-table at 000ff780
May 27 13:01:45 server kernel: Memory for crash kernel (0x0 to 0x0) notwithin permissible range
May 27 13:01:45 server kernel: disabling kdump
You can see it was down for some 6 hours, but I can't find a reason why
Diskspace is ample, there's no odd programs/scripts installed (it's a Centos server running whm with CSF firewall)
I reboot my cpanel server twice last night and it had crash
I am now running e2fsck and it's trying to fix the inode thing.. i've been pressing <y> 10000 times! Poor thing my fingers cramp
I've been thinking to mount my primary drive into another cpanel server (not a fresh server) as secondary drive and restore from there. The question is how? I need detailed steps by steps as it's urgent for me..
I just got a new server the other day, installed cpanel and all my usual stuff in it.
Then I started transferring some accounts off my other server on the same network. It would transfer small accounts ok but I have this one big account,. the sql is like 300 mb. It would get to the restore on it then the server would crash and be totally unresponsive.
It has to be manually rebooted at the datacenter. There is nothing in any logs. At first I thought it was the load getting hi and doing it but I ran the unix bench and got the load to 14 and it didnt hurt it.
It only does this on restoring big sql db. At the dc they say the screen on the console goes blank.
Ive done looked and cant find anything in the box. Its actually the same exact setup I use for my other servers.
So the dc is looking at it today. Has anyone ever ran into a problem like this and if so what was the cause?
What could cause the following on one of our servers?
Rebooted and was fine.
May have happened before, will have to dig deeper into logs.
Running CestOS 4.4 (cPanel/WHM)
Assertion failure in __journal_file_buffer() at fs/jbd/transaction.c:1966: "jbd_is_locked_bh_state(bh)"
------------[ cut here ]------------
kernel BUG at fs/jbd/transaction.c:1966!
invalid operand: 0000 [#1]
SMP
Modules linked in: ipt_state ipt_TOS iptable_mangle ip_conntrack_ftp ip_conntrack_irc ip_conntrack ipt_REJECT ipt_LOG ipt_limit iptable_filter ipt_multiport ip_tables autofs4 sunrpc md5 ipv6 loop dm_mirror dm_mod button battery ac uhci_hcd ehci_hcd e100 mii ext3 jbd ata_piix libata sd_mod scsi_mod
CPU: 1
EIP: 0060:[<f8830dfc>] Not tainted VLI
EFLAGS: 00010216 (2.6.9-42.ELsmp)
EIP is at __journal_file_buffer+0x42/0x221 [jbd]
eax: 0000006b ebx: e5b6d3dc ecx: da83add0 edx: f88361ca
esi: f6da3300 edi: 00000001 ebp: 00000000 esp: da83adcc
ds: 007b es: 007b ss: 0068
Process pdflush (pid: 5216, threadinfo=da83a000 task=cb5ad2b0)
Stack: f88361ca f8835eac f88361b5 000007ae f8836634 e5b6db3c 00000000 dcca5660
e5b6d3dc f7638d68 f7f48200 f8830165 f7f482e4 00000000 00001000 f7638d68
dcca5660 00000000 f8898e42 00001000 00000000 dcca5660 f8898cda dcca5660
Call Trace:
[<f8830165>] journal_dirty_data+0x17d/0x1bc [jbd]
[<f8898e42>] ext3_journal_dirty_data+0xc/0x2a [ext3]
[<f8898cda>] walk_page_buffers+0x62/0x87 [ext3]
[<f889927a>] ext3_ordered_writepage+0xee/0x13a [ext3]
[<f889917a>] journal_dirty_data_fn+0x0/0x12 [ext3]
[<c0178962>] mpage_writepages+0x1c2/0x314
we have dual xeon linux server redhat 9 / cpanel
we have a strange crash .... you cant access ftp /ssh / httpd ...
i need to request a server reboot to get access ... and everything after this run ok...
i have check message log .. nothing in it for example
10:20:10 ftp log
11:15:60 rebootlog
also the same in httpd log ...
so as you see there is a gab between the server crash till reboot....
this happen 3 times till now ...
The Apache server on one of my box crashed last night and the log file simply states:
[Fri Mar 09 23:51:49 2007] [notice] caught SIGTERM, shutting down
One thing puzzled me is that there are a couple of symbolic links created in Apache home directory:
lrwxrwxrwx 1 root root 25 Mar 9 23:51 build;45f23959 -> ../../usr/lib/httpd/build
lrwxrwxrwx 1 root root 19 Mar 9 23:51 logs;45f23959 -> ../../var/log/httpd
They were created around the same time when Apached stopped.
Has anyone seen similar things like this? What could be the root cause?
i want to ask any any setting or script can help medo the purpose?
i hope i can set a cron to trace server load every five minutes,
and help me record the load number on a log file,
by the way,
i can trace the past load of my server when i need the data.
I have a windows 2003 server with Plesk installed. It keeps crashing and requires a reboot every couple of days.
Here are screenshots of event logs and task manager.
Also, its a Pentium HT with 1GB memory, so it isn't too slow for Windows.
I have a shared server (root access) using Cpanel / Centos with suphp enabled.
Twice this week the sever's load skyrocketed and was unable to login to trace teh cause, had to reboot instead.
After reboot, I went to whm > CPU/Memory/MySQL Usage and saw nothing in red aside netstat (21% cpu). I'm not sure if this is the cause, but how can I trace the absolute user or script causing this spike?
I have a lynix root server with 1and1.com and recently the server has been going down I have no access to FTP/the sites or root. Their support tells me to look in var/log but im not sure what file to look in or what im looking for as far as to why the server would crash.
So far my solution has been to re boot the server and it has been working but i need to find the cause of this...any suggestions on where to look or what to look for?
I have server load like 40.
Nothing lags.
Server has RAM usage 20% and CPU usage goes up to 25%
Swap isnt used at all.
Should I be afraid of server crash incase server load goes up to 60 or 80 without ram and cpu usage rising that much?
I have a Linux server running some reasonable setups:
Opteron 180, 4 GB RAM, running 8x 36gb 15k scsi with hardware raid 10 -- this is one of the servers from WebNX advertised here not long ago
Running CentOS 4(?), Apache2/MySql 4/PHP5.2, the normal stuff.
I have only one main site on the server, which runs a pretty old PostNuke CMS in Chinese (0.7.2.3 Phoenix) + PNphpBB2 + Gallery 1.5.7 (all integrated into PostNuke). This site is pretty light in "human" traffic, getting about 20K hits per day.
Now, the problem I have noticed with this site, is related to the many MP3 files stored in the Gallery albums. There are lots of HTTP requests to these files, most maybe from Chinese search engine bots (judging from IP), that slows the server to a crawl and even crashes Apache. This happens in the late hours here when it's day time in China. As a matter of fact I just did a reboot, and in 5 minutes there are more than 1000 HTTP requests to MP3 files resulting in a traffic of 2.1+ Gb. So within minutes, the server is brought to its knees again and I can't even get the "apache status" from CPanel now: "Unable to retrieve apache status".
The company that manages the server for me said there's no security problem here. We have installed an Apache extension to limit the number of simultaneous requests to media files to 1. However that doesn't seem to help.
for months my server has been running fine, since about a week ago the server crashed and was up and down
the fault was found to be faulty ram which has been replaced.
now I have had every tom, dick & harry take a look at the server to get it stable again but im still having issues.
The server seems to run fine but crashes when I try and repair or backup my database.
the database is large like 30gig but the server used to do this fine before but it wont now, it get so far then hangs.
Server:
Dual Core Xeon
4GB Ram
2x 250GB HDD
Raid 1
My RAM is 2G.
The cache memory of my server is eating, After 2 weeks, the cache memory is below 1G, then the server crashed.
After I reboot the machine, the cache memory back to normal. But it starts to eating again.
I have attached the graph.
I am using lighttpd, php, mysql
I have a problem. to buy dedicated server but I don't really know what to buy. I need some kind of stress test for server so that I can be assured that my website won't crash when it will be on front page of digg. Does something like that even exists?
View 7 Replies View RelatedThe company I work for unfortunately runs their site on a IIS server. I want to install some forum software, vBulletin or phpBB onto the server and run it on the same site. PHP of course needs to be installed to do this.
Boss is worried that putting PHP on same server as ASP might take down the site if something goes wrong. What are the odds of this happening if I follow directions and what steps should I take to install?
I am running FC2 with cPanel.
My server stopped responding to everything from http to ping and I had to do a hard reboot on it.
Now, I'd like to figure out what happened but am not sure where to start. I've been sifting through /var/log/messages but can't pin point what I'm looking for.
generate crash dumps for apache 2.4.9 at the time freeze or crash in windows environment 2008 server R2.
View 2 Replies View RelatedI am wanting to get access to 'top -c' for a period of 48/72 hours.
Now I have found the following
Code:
top -c -b -d 15 > top.txt
What this does is every 15 seconds add reply 'top -c' to top.txt
Now I can make this into a shell script then put it into the background and exit SSH while leaving it running.
Now the question is will this use much usage/server load over the 48/72hrs, it will be checking 4 times every minute.
I thought i'd just post a quick message on here to see what you guys make of this situation. It's only happened with what, the past hour or so, but when i check the visitor logs on our website it seems that a "YouMonitor.us" is constantly spamming connections to the website over 1 second intervals (even less than that in some cases).
Furthermore, its coming in from different IP address's all the time and therefore its inappropriate to block everyone as they seem to just constantly change.
Today i was informed that some of Apache instances are vulnerable for serving content while client is constantly pressing F5 button in browsers - once is pressed CPU load is increasing, page became slow etc. (it's dynamic content served by back-end Tomcats). In the same time i see errors with connection between Apache and Tomcats' instances.
Is there any good way to protect Apache against it ?
I have a dedicated RHEL server with cPanel and my server loads spikes about +0.4 (out of 2.0) for about 30 mins every 4-6 hours or so. My regular server load is 0.01, because there is barely any traffic on the server yet, but by looking at my top processes in WHM, I can see that the processes that are spiking the Server Load when it is high, is something like:
sshd: [priv] root
sshd: [priv] root
sshd: [priv] root
sshd: [accepted]
sshd: [priv] games
sshd: [priv] news
sshd: [priv] root
sshd: [priv] root
sshd: [accepted]
...something along these lines. And a lot of times there are 10-20 of these sshd processes at one time.
My server is managed and my dedicated server engineer said it was probably a bot trying passwords. He took one of the IP's, said it was from Taiwan, and blocked that IP in iptables.
However, this is still happening constantly with different IP's. Is there a way to prevent this from happening? I'm the only person (and my host) who should be able to login to my server using SSH... however, I don't have a static IP and I work from multiple locations, so only allowing certain IP's won't work for me.
First off, is this normal? Or am I being attacked or what? What can I do to remedy this? It seems the bots haven't successfully logged in, but they are spiking my server load which is NOT what I want.
I was having issues with what I would like to think as power.
Now, I was wondering if there are any SQL database benchmarks, or something else I can run on the system for a given period of time, that will let me see if I'm still having those power issues.
I keep seeing sa login failure on Event viewer Application log for MS SQL 2005. I'm tired of blocking off the failed attempt ip. A huge portion of them seem to come from China and Estern Euro. Should i just block off those region? Is there a better way for securing SQL2005?
View 1 Replies View RelatedWe first noticed the issue on July 19th, Backups were taking a while to transfer from our own servers, and we were getting tickets from clients asking why this was happening. At the same time, it took about 4-5 minutes to log into the backup console. Issue seemed to correct itself so that backups were possible, and we assumed that scott would be aware of it and would work to resolve it.
Again on the 29th it slowed completely again. We immediately dispatched an email to their support email asking whats up, gave them detailed information such as access to the script and the output so he could monitor it.
The same day a few hours later we got a response that he would run the script and monitor it.
On the 31st, after not getting an update I asked what was going on. He said he ran the script, it seemed to stall for long periods of time and he would know more in the afternoon.
After not getting an update for ~5 days I asked what was going on and I expressed that i was not happy with the lack of communication/updates ont he situation. He updated me later that day saying that we weren't the only ones being affected. Fair enough, he is working on it. Thats all I wanted to hear.
Now ever since that email I received on the 5th (AUG) it has been up and down and all over the place.
After seeing him post on the forums I sent him a PM (10th August), summed up it was basically saying I had an issue, and I need it fixed. I figured since the PM system is usually pretty good at not losing them that he would see it and respond accordingly, that way there was no chance of it being lost from point A to point B as an email.
Well to this day, no response nor resolution.
I really have exhausted all points of communication, and I want to make it clear that I am not bashing them, when it works it's great. But maybe someone else affected can provide more insight..