Since the past few months I am facing cpu load on a server. The server is a Intel Xeon server with 4 gigs of RAM. Today the load average went over 150 which is very high. I could not access the server to check what is causing such a high load. I had to reboot the server using Remote Power switch. Is there a way I can find out the cause of the heavy load after the server is reboot? Is there any log where all the information is stored?
View 9 Replies
In cpanel I usually see my sever load in red zone like from 15-18
how bad is this ? should I think about changing my host?
a good method to have processes controlled in the resources they use, in a shared cpanel environment?
For example, it appears that the cpanel backup (compressed) made the load skyrocket to beyond 60% cpu and 95% memory usage today, and this server is beefy to begin with.
I want some system to kill processes immediately if they exceed X amount of resources or kill processes that spiral out of control (X amount of processes from same user).
I'd like to be alerted of this action as well.
How can I set this up?
my server is running cpanel, dual xeon 2.4GHZ, 2GB ram, only have about 10 small websites, but look at the load:
Quote:
Tasks: 154 total, 1 running, 153 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.1% us, 0.1% sy, 0.0% ni, 0.0% id, 99.8% wa, 0.0% hi, 0.0% si
Mem: 2074940k total, 1110308k used, 964632k free, 105768k buffers
Swap: 4096564k total, 0k used, 4096564k free, 829452k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27230 root 16 0 2168 1000 760 R 0 0.0 0:02.15 top
1 root 16 0 2620 556 472 S 0 0.0 0:00.99 init
2 root RT 0 0 0 0 S 0 0.0 0:00.10 migration/0
3 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/0
4 root RT 0 0 0 0 S 0 0.0 0:00.07 migration/1
5 root 34 19 0 0 0 S 0 0.0 0:00.18 ksoftirqd/1
6 root RT 0 0 0 0 S 0 0.0 0:00.08 migration/2
7 root 34 19 0 0 0 S 0 0.0 0:00.15 ksoftirqd/2
8 root RT 0 0 0 0 S 0 0.0 0:00.21 migration/3
9 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/3
10 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/0
11 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/1
12 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/2
13 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/3
14 root 5 -10 0 0 0 S 0 0.0 0:00.01 khelper
15 root 15 -10 0 0 0 S 0 0.0 0:00.00 kacpid
43 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/0
44 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/1
45 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/2
46 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/3
47 root 15 0 0 0 0 S 0 0.0 0:00.00 khubd
56 root 15 0 0 0 0 D 0 0.0 0:01.24 pdflush
59 root 8 -10 0 0 0 S 0 0.0 0:00.00 aio/0
60 root 11 -10 0 0 0 S 0 0.0 0:00.00 aio/1
61 root 11 -10 0 0 0 S 0 0.0 0:00.00 aio/2
62 root 5 -10 0 0 0 S 0 0.0 0:00.00 aio/3
58 root 15 0 0 0 0 S 0 0.0 0:01.12 kswapd0
135 root 25 0 0 0 0 S 0 0.0 0:00.00 kseriod
212 root 15 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_0
213 root 15 0 0 0 0 S 0 0.0 0:00.00 percraid
234 root 15 0 0 0 0 D 0 0.0 0:06.05 kjournald
1180 root 6 -10 2664 456 380 S 0 0.0 0:00.03 udevd
1346 root 5 -10 0 0 0 S 0 0.0 0:00.00 kauditd
1421 root 8 -10 0 0 0 S 0 0.0 0:00.00 kmirrord
1422 root 8 -10 0 0 0 S 0 0.0 0:00.00 kmir_mon
1442 root 19 0 0 0 0 S 0 0.0 0:00.00 kjournald
1443 root 15 0 0 0 0 D 0 0.0 0:01.93 kjournald
2168 root 15 0 3468 600 512 D 0 0.0 0:00.60 syslogd
2172 root 16 0 2428 484 420 S 0 0.0 0:00.25 klogd
2182 root 16 0 1756 492 412 S 0 0.0 0:00.01 irqbalance
2194 named 18 0 69712 3492 2232 S 0 0.2 0:01.11 named
2268 root 16 0 2644 440 380 S 0 0.0 0:00.00 courierlogger
2269 root 16 0 1940 612 508 S 0 0.0 0:00.00 authdaemond
2270 root 16 0 1940 616 512 S 0 0.0 0:00.00 authdaemond
2271 root 16 0 1940 616 512 S 0 0.0 0:00.00 authdaemond
2272 root 16 0 1940 616 512 S 0 0.0 0:00.00 authdaemond
2273 root 16 0 1940 616 512 S 0 0.0 0:00.00 authdaemond
2274 root 16 0 1940 616 512 S 0 0.0 0:00.00 authdaemond
2303 root 16 0 5392 1012 844 S 0 0.0 0:00.00 rpc.idmapd
2375 root 18 0 1980 552 480 S 0 0.0 0:00.00 acpid
2384 root 15 0 9308 2076 1632 S 0 0.1 0:00.03 cupsd
Anyone know of some good server load testers ( commercial )?
Im not looking for application based load testing, I need real web server load testing... need to see how much traffic this one site can take before it cries.
I'm having the oddest issue. For some reason, some of the websites on my server load fine, and some take a really long time to load (2 minutes).
Now, the server load is fine, and the size of the sites aren't the issue either. I've restarted Apache and a couple more services, and still the same sites seem to load very slow.
What could be causing this since it's only effecting certain websites?
I'm using Virtuozzo on two server.
1) Can I do this with only ONE Load Balancer?
I wish I could have many "LB", like one for the WWW serveur, an other one for the DB serveur, my "personnal" VPS without LB...
i.e. :
WWW -> SRV1 SRV2
DB1 -> DB1 DB2
EXCESSNET -> EXCESSNET1
Do I need 3 LB to do this or I can do it with only one? The screenshot of Barracuda LB seem to show many "virtual LB", so I guest it's a yes?
2) So... it's service only?
On the Barracuda and some page, they are talking about "service" and not "server" LB. Can I LB a whole serveur ? Like all service from an IP xxx.xxx.xxx.234 to a server ? Or I can only do "HTTP -> SRV1" "DNS -> SRV1"? If no, what happen to service that the LB didn't know (like cPanel)?
3) An Internal Network? what About cPanel?
According to my search, LB are working like this:
EXTERNAL IP of SERVICE (xxx.xxx.xxx.200) -> INTERNAL IP SERVER 1(10.0.0.25), INTERNAL IP SERVER 2 (10.0.0.26)
So, I need to configure the server with an Internal IP cause else, it will conflit on the "local network". The probleme is, cPanel will be using the "Internal IP" to make all of his configuration... so if a user add a new website, he will configure it with 10.0.0.25, this won't since the external IP will be xxx.xxx.xxx.200 and Apache won't anwser to this virtual host. What can I do?
Can I do this ? (same internal IP as the external?)
EXTERNAL IP (70.234.125.123) -> SERVER 1 INTERNAL IP (70.234.125.123), SERVER 2 INTERNAL IP (10.0.0.26)
If yes, I will configure the second serveur without cPanel and only sync the configuration and files. So in case of "crash" of the server 1, cPanel won't be working, but website won't be down...
I've been having trouble with my VPS for a while now. In the QoS alerts page in Virtuozzo it seems to be a problem with numtcpsock and tcprcvbuf, mainly numtcpsock.
Copy these into the browser:
i18.photobucket.com/albums/b106/gnatfish/qosnumtcpsock2.jpg
And when i run cat /proc/user_beancounters:
i18.photobucket.com/albums/b106/gnatfish/beancounters2.jpg
This line is particularly scary:
numtcpsock 164 164 166 166 7321
What do i need to do, to get the website running again? It's only one site on the vps a proxy. So i thought a vps would be able to handle one proxy.
Any good and simple script to see Server status, server load, bandwidth, etc...?
is there like a simple script where it shows me the server info?
Like how much RAM is being used, bandwidth, load, etc?
We have CentOS 4 and WHM 11 on server .
Now-a-days server is having too much load due to http and in access logs we see following message :
========================================
127.0.0.1 - - [11/Oct/2008:01:40:02 -0700] "OPTIONS * HTTP/1.0" 200 -
127.0.0.1 - - [11/Oct/2008:01:40:03 -0700] "OPTIONS * HTTP/1.0" 200 -
127.0.0.1 - - [11/Oct/2008:01:40:02 -0700] "OPTIONS * HTTP/1.0" 200 -=============================================
And due to this there is load on server. We are not able to understand why this is happening and how to stop this. So please suggest with some solution.
I am facing a strange issue with two RPG games sites that i am hosting on a 2GB ram server with softlayer, each of these sites was running on a 512MB VPS and they were doing fine but i decided to move them to better server so things run smoother when each has 50+ members online.
So the issue is when i have around 40 online members on each site, they start coughing up mysql errors, sometimes the sites just hang and stop loading, server load goes up to 3 or 4 but memory usage remains around 40%, I also notice that mysql uses the most of the CPU usage, around 75%
What i probably need is for someone to identify the cause of this, can it be an attack of some sort? or is it probably some code issues in the sites? do you recommend anyone that can have a look at this?
So the other day me and friend installed cPanel on a VPS. Everything seems to be working perfectly however the sites don't seem to load.
I've pinged all the domains and they all seem to be fine and the hostname also seems to be setup I'm just not sure why my sites aren't loading.
I've also loaded the site on multiple computers using different dns servers and still no luck.
I haven't touched apache at all, but from what I understand I don't need to.
I have my WHM/cPanel installation configured with daily and weekly backups. I checked at what time of the day the server load was at the minimum and configured the cPanel backup cron to run then.
The problem now is: Backing up a few hundred accounts results in a high server load. My server configuration:
Dual Processor Quad Core Xeon 5335 2.0GHz with 4GB RAM and 2 x 250GB SATA HDD hosted at SoftLayer.
The accounts are located on the first HDD and the backup archives are placed on the second HDD.
What can I do about this? I'd like to take daily backups of all accounts but not if my server load increases up to 10... That kind of renders the cPanel backup feature useless if it doesn't even work on a powerful server like this one...
Would it help if I use an application such as Auto Nice Daemon to give the backup process a lower priority? But then again that won't work on the MySQL dumps? And I think it's not a CPU problem but an I/O wait problem? Other processes have to wait for disk access because the disk-intensive backup process is running?
I have a cPanel VPS so I want to know the 'Server Load' appearing in cPanel, is it for my VPS or the whole server?
View 8 Replies View RelatedWhen my server load is in the 30s:
- My server runs smoothly, doesn't get any slower.
- My memory usage is not over 50%
- My CPUs are not going over 50%
- My hard drives are doing fine
So what is my server load showing?
how much strain does ssl puts on the server? Lets say, a small VPS with ~300 RAM and 2.6GHz.
How many clients could simultaneously have a smooth experience? (only using ssl to access their account and upload files up to 50MB)
my server load just reached 29.32 (2 cpu).
These are my specs:
**************************************
Server Information
Processor Information
Processor #1 Vendor: GenuineIntel
Processor #1 Name: Intel(R) Pentium(R) 4 CPU 3.20GHz
Processor #1 speed: 3192.543 MHz
Processor #1 cache size: 1024 KB
Processor #2 Vendor: GenuineIntel
Processor #2 Name: Intel(R) Pentium(R) 4 CPU 3.20GHz
Processor #2 speed: 3192.543 MHz
Processor #2 cache size: 1024 KB
Memory Information
Memory: 1025772k/1039552k available (2476k kernel code, 13144k reserved, 1098k data, 240k init, 122048k highmem)
System Information
Linux server973.xxxx.net 2.6.17.6.dn2.p4 #3 SMP Sun Jul 23 18:01:11 EDT 2006 i686 i686 i386 GNU/Linux
Physical Disks
SCSI device sda: 234441648 512-byte hdwr sectors (120034 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
SCSI device sda: 234441648 512-byte hdwr sectors (120034 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
sd 0:0:0:0: Attached scsi disk sda
sd 0:0:0:0: Attached scsi generic sg0 type 0
Current Memory Usage
total used free shared buffers cached
Mem: 1026280 994472 31808 0 119472 341092
-/+ buffers/cache: 533908 492372
Swap: 2096440 0 2096440
Total: 3122720 994472 2128248
Current Disk Usage
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 1.5G 754M 647M 54% /
/dev/sda7 82G 10G 68G 13% /home
/dev/sda6 494M 9.0M 460M 2% /tmp
/dev/sda3 9.7G 4.4G 4.8G 48% /usr
/dev/sda2 16G 7.4G 7.3G 51% /var
none 502M 0 502M 0% /dev/shm
/tmp 494M 9.0M 460M 2% /var/tmp
**************************************
Some mysql databases take about 30 seconds to load results...
Is there a way to reduce this load beside adding more ram (something that I'll do right away)...
What's the normal load of a new server-Intel Xeon 3060, CentOS 4.4 32-bit, cpanel, 2x250 SATA II, 2GB of ram.
I saw it go up to .30 I didn't install anything, it's a fresh new server. Should I be concerned?
What is normal server load should be from web hosting servers.
more than 8 is bad?
I'm getting an almost constant server load of between 0.8 and 1.9
Its usually about 1.2 - 1.6 but peaks at 3.0.
It might just be traffic but can anyone check the output from top and see if it looks ok
top - 17:10:59 up 93 days, 23:01, 2 users, load average: 2.16, 1.31, 1.12
Tasks: 116 total, 1 running, 114 sleeping, 0 stopped, 1 zombie
Cpu(s): 11.5% us, 3.5% sy, 0.0% ni, 76.7% id, 8.4% wa, 0.0% hi, 0.0% si
Mem: 1027556k total, 1000260k used, 27296k free, 90060k buffers
Swap: 2048276k total, 160k used, 2048116k free, 568264k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29795 root 16 0 29364 26m 2048 S 6.6 2.6 0:27.34 spamd
305 root 16 0 0 0 0 S 1.7 0.0 719:35.17 kjournald
26305 mailnull 17 0 9608 3108 2036 S 1.0 0.3 0:00.03 exim
722 named 25 0 63124 27m 2032 S 0.7 2.8 223:21.30 named
14110 nobody 16 0 26244 15m 3704 S 0.3 1.5 0:25.16 httpd
26376 root 16 0 2044 988 760 R 0.3 0.1 0:00.02 top
1 root 16 0 1732 528 452 S 0.0 0.1 2:03.99 init
2 root 34 19 0 0 0 S 0.0 0.0 1:33.01 ksoftirqd/0
3 root 5 -10 0 0 0 S 0.0 0.0 3:18.70 events/0
4 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 khelper
5 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 kacpid
21 root 5 -10 0 0 0 S 0.0 0.0 16:38.77 kblockd/0
42 root 12 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0
22 root 15 0 0 0 0 S 0.0 0.0 0:00.00 khubd
41 root 16 0 0 0 0 S 0.0 0.0 16:48.41 kswapd0
188 root 25 0 0 0 0 S 0.0 0.0 0:00.00 kseriod
300 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 ata/0
1351 root 6 -10 1800 500 412 S 0.0 0.0 0:00.01 udevd
1832 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 kauditd
1879 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 kmirrord
1897 root 15 0 0 0 0 S 0.0 0.0 0:00.00 kjournald
2808 root 16 0 4428 532 272 S 0.0 0.1 0:36.65 rpc.idmapd
2880 root 18 0 3244 496 416 S 0.0 0.0 0:00.00 acpid
2976 root 17 0 3032 772 624 S 0.0 0.1 0:00.02 xinetd
3021 root 16 0 6084 936 536 S 0.0 0.1 0:11.98 crond
3047 dbus 16 0 3496 956 812 S 0.0 0.1 0:00.02 dbus-daemon-1
3056 root 16 0 8096 4840 1512 S 0.0 0.5 6:22.22 hald
3063 root 17 0 2932 1252 972 S 0.0 0.1 0:00.00 login
3064 root 16 0 2020 448 388 S 0.0 0.0 0:00.00 mingetty
3065 root 16 0 2084 448 388 S 0.0 0.0 0:00.00 mingetty
3066 root 18 0 2964 448 388 S 0.0 0.0 0:00.00 mingetty
3067 root 18 0 2380 448 388 S 0.0 0.0 0:00.00 mingetty
3068 root 18 0 1580 448 388 S 0.0 0.0 0:00.00 mingetty
3682 root 16 0 5836 1404 1180 S 0.0 0.1 0:00.00 bash
3949 root 16 0 0 0 0 S 0.0 0.0 66:21.77 kjournald
13066 root 16 0 2052 548 456 S 0.0 0.1 13:59.70 syslogd
13070 root 16 0 2028 384 316 S 0.0 0.0 0:00.05 klogd
3404 root 0 -20 0 0 0 S 0.0 0.0 3:02.75 loop0
3405 root 15 0 0 0 0 S 0.0 0.0 0:24.35 kjournald
8596 root 17 0 12484 8032 1080 S 0.0 0.8 1:04.24 chkservd
2051 root 16 0 5172 1136 832 S 0.0 0.1 1:56.43 sshd
26330 root 16 0 6952 1392 1116 S 0.0 0.1 0:00.19 pure-ftpd
26333 root 15 0 6280 896 764 S 0.0 0.1 0:00.03 pure-authd
5719 root 16 0 18784 9.8m 3620 S 0.0 1.0 1:13.73 httpd
30206 root 15 0 0 0 0 S 0.0 0.0 14:45.38 pdflush
12376 root 16 0 8372 4744 692 S 0.0 0.5 0:06.09 cpbandwd
12456 root 39 19 21524 15m 1732 S 0.0 1.6 19:35.48 cpanellogd
12530 mailman 16 0 11532 5016 920 S 0.0 0.5 0:00.00 mailmanctl
12539 mailman 16 0 10228 6700 2620 S 0.0 0.7 1:41.93 python2.4
12540 mailman 16 0 10760 6636 2588 S 0.0 0.6 1:48.33 python2.4
12541 mailman 16 0 11052 6636 2588 S 0.0 0.6 1:42.71 python2.4
12542 mailman 16 0 11720 6700 2620 S 0.0 0.7 1:41.73 python2.4
12543 mailman 16 0 10716 6720 2620 S 0.0 0.7 1:41.25 python2.4
ps aux output.
Code:
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 12 8.8 0.0 0 8 ?? WL 11:54PM 37:44.94 [swi1: net]
root 23 7.9 0.0 0 8 ?? WL 11:54PM 27:31.07 [irq23: vr0]
jeev 55752 0.3 0.6 7560 6492 ?? S 5:10AM 0:11.41 /usr/bin/perl blah.cgi --build-with-new (perl5.8.8)
www 57051 0.1 1.0 14224 9860 ?? S 6:54AM 0:06.72 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 57202 0.1 1.0 14224 9860 ?? L 7:19AM 0:05.42 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 57899 0.1 1.0 14160 9704 ?? L 9:06AM 0:00.13 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 55531 0.0 1.0 14224 9768 ?? S 5:05AM 0:12.46 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 55871 0.0 1.0 14224 9836 ?? S 5:14AM 0:10.94 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 56063 0.0 0.7 14224 6956 ?? L 5:18AM 0:10.29 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 56147 0.0 0.7 14224 6916 ?? S 5:20AM 0:10.61 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 56271 0.0 1.0 14224 9780 ?? S 5:24AM 0:11.43 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 56387 0.0 1.0 14224 9840 ?? S 5:32AM 0:11.01 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 56823 0.0 0.7 14224 6652 ?? L 6:23AM 0:07.40 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 57228 0.0 1.0 14224 9780 ?? S 7:22AM 0:05.17 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 57296 0.0 1.0 14224 9780 ?? S 7:35AM 0:04.65 /usr/local/sbin/httpd -DNOHTTPACCEPT
www 57500 0.0 1.0 14224 9780 ?? S 8:04AM 0:03.03 /usr/local/sbin/httpd -DNOHTTPACCEPT
root 0 0.0 0.0 0 0 ?? WLs 11:54PM 0:00.01 [swapper]
root 1 0.0 0.0 768 248 ?? ILs 11:54PM 0:00.27 /sbin/init --
root 2 0.0 0.0 0 8 ?? DL 11:54PM 0:03.48 [g_event]
root 3 0.0 0.0 0 8 ?? DL 11:54PM 0:29.41 [g_up]
root 4 0.0 0.0 0 8 ?? DL 11:54PM 0:25.58 [g_down]
root 5 0.0 0.0 0 8 ?? DL 11:54PM 0:00.00 [kqueue taskq]
root 6 0.0 0.0 0 8 ?? DL 11:54PM 0:00.00 [acpi_task_0]
root 7 0.0 0.0 0 8 ?? DL 11:54PM 0:00.00 [acpi_task_1]
root 8 0.0 0.0 0 8 ?? DL 11:54PM 0:00.00 [acpi_task_2]
root 9 0.0 0.0 0 8 ?? DL 11:54PM 0:00.00 [thread taskq]
root 10 0.0 0.0 0 8 ?? RL 11:54PM 0:00.00 [idle: cpu1]
root 11 0.0 0.0 0 8 ?? RL 11:54PM 127:04.50 [idle: cpu0]
root 13 0.0 0.0 0 8 ?? WL 11:54PM 6:17.46 [swi4: clock sio]
root 14 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [swi3: vm]
root 15 0.0 0.0 0 8 ?? DL 11:54PM 1:41.91 [yarrow]
root 16 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [swi5: +]
root 17 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [swi6: Giant taskq]
root 18 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [swi6: task queue]
root 19 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [irq9: acpi0]
root 20 0.0 0.0 0 8 ?? WL 11:54PM 0:27.49 [irq20: atapci0]
root 21 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [irq14: ata0]
root 22 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [irq15: ata1]
root 24 0.0 0.0 0 8 ?? DL 11:54PM 0:04.95 [acpi_thermal]
root 25 0.0 0.0 0 8 ?? DL 11:54PM 0:00.06 [acpi_cooling0]
root 26 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [swi0: sio]
root 27 0.0 0.0 0 8 ?? WL 11:54PM 0:00.00 [irq1: atkbd0]
root 28 0.0 0.0 0 8 ?? DL 11:54PM 0:04.04 [pagedaemon]
root 29 0.0 0.0 0 8 ?? DL 11:54PM 0:00.00 [vmdaemon]
root 30 0.0 0.0 0 8 ?? RL 11:54PM 0:04.86 [pagezero]
root 31 0.0 0.0 0 8 ?? DL 11:54PM 0:00.39 [bufdaemon]
root 32 0.0 0.0 0 8 ?? DL 11:54PM 0:01.73 [vnlru]
root 33 0.0 0.0 0 8 ?? DL 11:54PM 0:43.43 [syncer]
root 34 0.0 0.0 0 8 ?? DL 11:54PM 0:00.91 [softdepflush]
root 35 0.0 0.0 0 8 ?? DL 11:54PM 0:38.78 [schedcpu]
root 109 0.0 0.1 1296 652 ?? Is 11:54PM 0:00.00 adjkerntz -i
root 127 0.0 0.0 0 8 ?? DL 11:54PM 0:00.01 [md0]
root 448 0.0 0.0 528 240 ?? Is 11:54PM 0:00.00 /sbin/devd
root 503 0.0 0.1 1440 928 ?? Is 11:54PM 0:00.98 /usr/sbin/syslogd -c -s
root 639 0.0 0.2 3568 2092 ?? Is 11:54PM 0:00.38 /usr/sbin/sshd
root 645 0.0 0.2 6312 2420 ?? Is 11:54PM 0:00.03 sshd: jeev [priv] (sshd)
root 647 0.0 0.2 6312 2424 ?? Is 11:55PM 0:00.03 sshd: jeev [priv] (sshd)
root 653 0.0 0.1 1456 924 ?? Ss 11:55PM 0:00.49 /usr/sbin/cron -s
jeev 712 0.0 0.2 6296 2440 ?? S 11:55PM 0:01.33 sshd: jeev@ttyp0 (sshd)
jeev 723 0.0 0.2 6296 2440 ?? I 11:55PM 0:00.93 sshd: jeev@ttyp1 (sshd)
root 2845 0.0 0.1 1348 640 ?? S 11:57PM 0:03.68 /usr/local/bin/svscan /var/service
root 2846 0.0 0.0 1292 412 ?? I 11:57PM 0:00.00 [readproctitle]
root 2847 0.0 0.1 1300 568 ?? I 11:57PM 0:00.02 supervise dnscache
root 2848 0.0 0.1 1300 568 ?? I 11:57PM 0:00.02 supervise log
dnslog 2849 0.0 0.1 1328 600 ?? I 11:57PM 0:00.50 multilog t ./main
dnscache 2890 0.0 0.2 2632 1932 ?? S 11:58PM 0:00.75 /usr/local/bin/dnscache
root 4888 0.0 0.2 6312 2432 ?? Is 1:19AM 0:00.03 sshd: jeev [priv] (sshd)
jeev 4890 0.0 0.2 6296 2444 ?? I 1:19AM 0:08.65 sshd: jeev@ttyp2 (sshd)
nobody 48192 0.0 0.2 2416 1684 ?? Is 2:50AM 0:01.15 proftpd: (accepting connections) (proftpd)
www 52568 0.0 1.0 14224 9836 ?? S 3:58AM 0:16.60 /usr/local/sbin/httpd -
everything built with 'CFLAGS= -O2 -pipe -funroll-loops' in make.conf
php 5.2 from ports
apache 22 from ports
php5-extensions from ports
nothing in the apache config... maxclients hard limit 4000.
could this be maxrequestsperchild?
Server load is too high, what could be the reason?
top - 02:45:31 up 15 min, 1 user, load average: 2.38, 9.92, 8.01
Tasks: 148 total, 3 running, 145 sleeping, 0 stopped, 0 zombie
Cpu(s): 35.2% us, 5.3% sy, 0.0% ni, 21.6% id, 37.9% wa, 0.0% hi, 0.0% si
Mem: 1027556k total, 1017364k used, 10192k free, 122912k buffers
Swap: 2040244k total, 68k used, 2040176k free, 215112k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3350 popuser 16 0 33500 27m 2328 R 10.0 2.7 0:03.81 spamd
3359 popuser 18 0 33400 27m 2324 S 8.3 2.8 0:04.71 spamd
3202 apache 16 0 98372 57m 6208 S 6.0 5.7 0:02.58 httpd
3713 apache 16 0 92860 53m 5732 S 6.0 5.3 0:02.39 httpd
3206 apache 15 0 102m 62m 5752 S 2.0 6.2 0:01.98 httpd
2730 mysql 16 0 398m 59m 4356 S 0.7 5.9 0:04.14 mysqld
5923 apache 16 0 93344 53m 5192 S 0.7 5.3 0:01.61 httpd
3182 apache 16 0 92948 53m 6084 S 0.3 5.4 0:01.66 httpd
3191 root 15 0 30084 24m 2380 S 0.3 2.5 0:00.80 spamd
3195 apache 16 0 93736 54m 5500 S 0.3 5.4 0:01.54 httpd
3208 apache 15 0 98.2m 59m 5936 S 0.3 5.9 0:02.60 httpd
3212 apache 15 0 93176 54m 6040 S 0.3 5.4 0:01.96 httpd
3298 apache 15 0 96252 56m 5788 S 0.3 5.7 0:02.87 httpd
6170 apache 15 0 99648 59m 5680 S 0.3 5.9 0:01.05 httpd
1 root 16 0 2276 552 472 S 0.0 0.1 0:00.68 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/0
4 root 8 -10 0 0 0 S 0.0 0.0 0:00.00 khelper
5 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 kacpid
19 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 kblockd/0
37 root 15 0 0 0 0 S 0.0 0.0 0:00.00 pdflush
40 root 11 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0
20 root 15 0 0 0 0 S 0.0 0.0 0:00.00 khubd
39 root 16 0 0 0 0 S 0.0 0.0 0:00.08 kswapd0
186 root 25 0 0 0 0 S 0.0 0.0 0:00.00 kseriod
295 root 15 0 0 0 0 S 0.0 0.0 0:00.33 kjournald
1373 root 6 -10 3452 452 380 S 0.0 0.0 0:00.01 udevd
1606 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 kauditd
1648 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 kmirrord
1672 root 19 0 0 0 0 S 0.0 0.0 0:00.00 kjournald
1673 root 19 0 0 0 0 S 0.0 0.0 0:00.00 kjournald
2360 root 15 0 1748 548 456 S 0.0 0.1 0:00.28 syslogd
2364 root 16 0 2332 380 316 S 0.0 0.0 0:00.00 klogd
2787 postgres 16 0 20184 2048 1764 S 0.0 0.2 0:00.00 postmaster
2790 postgres 18 0 10984 472 184 S 0.0 0.0 0:00.00 postmaster
2791 postgres 18 0 9992 560 232 S 0.0 0.1 0:00.00 postmaster
2799 root 15 0 0 0 0 S 0.0 0.0 0:00.05 pdflush
2822 root 16 0 5128 752 628 S 0.0 0.1 0:00.01 couriertcpd
I built an online application using PHP / MySQL and it's pretty optimized (it's a very simple app). I've inadvertently picked up a huge client that could represent a few 1,000 "posts" per minute.
The process goes like this:
Consumer posts mobile data -> third party receives data -> third party sends data to my server -> my php script throws the data into a database.
I've got about a month to prepare things for these nightly "posts" before it's a real-life affair.
My questions are:
1) How can I "stress" or "load" test my machine and script to see what I'm up against.
2) What is my cheapest option for "redundancy" (Would something like MediaTemple's MySQL Grid products be a fix?)
When I say a few thousand posts per minute - each one is around 350 bytes.
I'm sure you may be wondering if I've bitten off more than I can chew - yea kind of but I made it perfectly clear to the client about my uncertainty and they are willing to give me benefit-of-a-doubt since I'm a very hard worker, very easy to deal with and loyal - not to mention my product is unlike any other that we have seen.
I have some serious server load issues on one of my web servers...
Hardware:
Intel Pentium D 3.40GHz
1024 MB DDR RAM
Software:
RH Linux with 2.6.9-42.0.3.ELsmp kernel
cPanel with most recent Apache and PHP 5.2.x
Process list (top output, on a regular moment):
[url]
When everything is running normal (read: server load below 2) there's between 100 and 300 MB of free memory. Though sometimes (this happens about 2 times per day, at random times of the day) the server load dramatically increases.
I wrote a script (the chkApache.sh process in the top list) that constantly checks the server load and if it raises about 4, it will check if the sum of all httpd processes are consuming too much CPU or Memory. In this case, it will force a httpd restart... I need this to prevent Apache from crashing my server almost daily. If I let Apache do its thing without this script, it will happen that tons of httpd processes (50 and up) take in all the RAM and server load increases to 100 or more and the eventually I need to reboot the server using the SoftLayer control panel.
That chkApache script I wrote also sends me a detailed report of the state of the server when it needs to restart Apache, here's a report of an event that occurred today (includes server load info, memory info, top list, httpd processes info, netstat, etc):
[url]
In this case you would think that someone is attacking the youthforums.co.uk domain but I doubt that's the case... It doesn't always happen with that site, in fact I can't seem to find a pattern in the Apache status page so I don't think a single account is causing this...
Today I also used the Apache JMeter to "stress test" my server. I was shocked when I saw how easy it is to use that tool in order to make my server crash... I used 10 threads loading one PHP page (that makes some MySQL queries) and made a loop that kept requesting that information, with 10 connections at a time... The server load rapidly increased to 30 and above... I think it's unacceptable that something like this can happen so easilly...
I tried several things with my Apache configuration settings... Here's what I have at the moment:
Code:
RLimitMEM 67108864
RLimitCPU 60
RLimitNPROC 30
Timeout 5
KeepAlive On
MaxKeepAliveRequests 50
KeepAliveTimeout 2
MinSpareServers 5
MaxSpareServers 10
StartServers 5
MaxClients 40
MaxRequestsPerChild 100
<IfModule mod_limitipconn.c>
<Location />
MaxConnPerIP 3
NoIPLimit image/*
</Location>
</IfModule>
LoadModule rewrite_module libexec/mod_rewrite.so
LoadModule expires_module libexec/mod_expires.so
LoadModule bwlimited_module libexec/mod_bwlimited.so
LoadModule bytes_log_module libexec/mod_log_bytes.so
LoadModule auth_passthrough_module libexec/mod_auth_passthrough.so
LoadModule php5_module libexec/libphp5.so
LoadModule security_module libexec/mod_security.so
LoadModule evasive_module libexec/mod_evasive.so
LoadModule limitipconn_module libexec/mod_limitipconn.so
Does anyone have an idea what's wrong here? Anything I can do to get more detailed information on what's causing this? I've been on this problem for weeks now but I can't seem to find any proper solution.
In case it matters I'm also running: SIM, RPM, BFD, APF and SPRI
how the load on your web hosting machines looks like. I've seen companies that keep the load at values of 10-15 units and the server is still responsive. Though there might be different reasons for the high load and it's a sure indication for an upcoming problem it doesn't seem normal to have this high load on a hosting server. (well actually it would if the machine was running on 16 cpus but it's a normal dual xeon woodcrest for example).
View 2 Replies View Related
