Don't know if it is some kind of attack or this is normal but the Server load goes reaching 20 in most of the times and may be greater!
this is the mysql_process list
jarabhar_JUHyyuh is opening alot of connections in the same time and this is not the normal behavior ....
On one server which i have sometime appear very high load up to 70,and that causing downtime of few minutes(i have historic data webmin module where i saw high load and downtime time matches,but it doesn't show what causing it).That server also have hardware raid in mirror mode.(copies exact data from one disk to another).
View 14 Replies View RelatedI recently started hosting a large vBulletin forum (14,014 registered users... 30-60 online at a time) on one of my dedicated servers. When there's a lot of activity on the forums, things start getting really slow. There will be like 50 php-cgi processes running. Some users get blank pages, others get this message:
Notice: Undefined variable: errfile in [path]/forums/includes/functions_log_error.php on line 163
Specs of the server are:
AMD Athlon 64 3000+
160GB SATA HDD
2GB DDR2 RAM
FreeBSD 6.2 (Hosted at AwkNet)
Could there be any Apache modifications I can make to increase performance, or anything else I could try?
I got a dedicated server which hosted 45 sites, one of those sites contains a vbulletin forum application since january 2007 , but from two weeks i found that there is a high amount of Queries from the this database which leads the server load to be always in RED and exceeds 200 or 300! and the memory is totally consumed!
i tried alot to solve this problem as :
- reinstall the vb.
- change the vb path to another ( considering it as attack)
- installing BFD , APF , ( considering it as attack )
- upgrading to apache 2.2.6 , PHP 5.2.5 , MySQL 5.0.24 ( considering it as a bug)
this is the sample of queries.
Code:
46978hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep40
NULL46980hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep38
NULL46990hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep38
NULL47034hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep30
NULL47086hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep23
NULL47092hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep19
NULL47095hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep23
NULL47099hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep23
NULL47122hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep24
NULL47124hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep24
NULL47126hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep24
NULL47127hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep24
NULL47142hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery22Sending dataSELECT *
FROM datastore
WHERE title IN ('','options','bitfields','attachmentcache','forumcache47143hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery0Writing to netSELECT moderator.*, user.username,
IF(user.displaygroupid = 0, user.usergroupid, user.displaygroup47144hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery22LockedSELECT *
FROM datastore
WHERE title IN ('','options','bitfields','attachmentcache','forumcache47145hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery22LockedSELECT *
FROM datastore
WHERE title IN ('','options','bitfields','attachmentcache','forumcache47147hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery22Writing to netSELECT *
FROM datastore
WHERE title IN ('','options','bitfields','attachmentcache','forumcache47151hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep16
NULL47159hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep17
NULL47166hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep15
NULL47177hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep0
NULL47180hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep16
NULL47211hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep10
NULL47251hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep0
NULL47283rootlocalhostNULLQuery0NULLSHOW PROCESSLIST
Once i disabled the database the server load became normal again
another example for Queries :
Code:
117445 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 49 NULL
117497 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 42 NULL
117667 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 24 NULL SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117790 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 7 NULL
117791 jarabhar_vb localhost jarabhar_vb Sleep 7 NULL
117793 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 7 NULL
117795 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 8 NULL
117798 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 8 NULL
117799 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 8 NULL
117802 unauthenticated user localhost NULL Connect NULL Reading from net NULL
117803 unauthenticated user localhost NULL Connect NULL Reading from net NULL
117804 unauthenticated user localhost NULL Connect NULL Reading from net NULL
117805 unauthenticated user localhost NULL Connect NULL Reading from net NULL
117806 unauthenticated user localhost NULL Connect NULL Reading from net NULL
117807 unauthenticated user localhost NULL Connect NULL Reading from net NULL
117808 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 9 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117818 jarabhar_vb localhost jarabhar_vb Query 8 Sorting result SELECT post.postidntttFROM post AS postntttntttWHERE post.threadid = 52572nttttAND post.visible = 1n
117821 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 8 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117829 jarabhar_vb localhost jarabhar_vb Query 8 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117832 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 statistics SELECT post.*,ntttIF(visible = 2, 1, 0) AS isdeleted,ntttnnttteditlog.userid AS edit_userid, editlog
117833 jarabhar_vb localhost jarabhar_vb Query 7 statistics SELECT IF(visible = 2, 1, 0) AS isdeleted,ntttnntttNOT ISNULL(subscribethread.subscribethreadid) AS
117834 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 statistics SELECT languageid,ntttphrasegroup_holiday AS phrasegroup_holiday,ntttphrasegroup_global AS phrasegro
117835 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Writing to net SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117836 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117840 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT forum.forumid, lastpost, lastposter, lastthread, lastthreadid, lasticonid, threadcount, reply
117841 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117842 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117843 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache
117845 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 2 NULL
117846 hakawyc_SKJI8jj localhost hakawyc_froumvb Killed 0 NULL NULL
117847 unauthenticated user localhost NULL Connect NULL Reading from net NULL
117848 hakawyc_SKJI8jj localhost NULL Connect NULL Writing to net NULL
117849 unauthenticated user connecting host NULL Connect NULL login NULL
117850 unauthenticated user connecting host NULL Connect NULL login NULL
117851 unauthenticated user connecting host NULL Connect NULL login NULL
117852 unauthenticated user connecting host NULL Connect NULL login NULL
117853 unauthenticated user connecting host NULL Connect NULL login NULL
117854 unauthenticated user connecting host NULL Connect NULL login NULL
117855 unauthenticated user connecting host NULL Connect NULL login NULL
117856 unauthenticated user connecting host NULL Connect NULL login NULL
117857 unauthenticated user connecting host NULL Connect NULL login NULL
What is the best way to load balance a vbulletin forum?
View 3 Replies View RelatedI have a site which uses vBulletin. After a long story we ended up with the need to restore it's database from a backup.
The backup proccess was in fact copying the database files - .MYD, MYI; so no mysql dumps.
I've recreated the dabase from phpmyadmin and created a standard user wtih read/write permissions and a dba user.
After that I've copied the database files from the backup and when I've tried to launch the site, it gives me the error:
"Warning: mysql_query(): Unable to save result set in /includes/class_core.php on line 354"
That's all the output I get.
I've modified the files permission to 755 and changed the owner and group; I've modified the vBulletin file to use the dba user account to access the database... all this without any luck.
I've done some searching and I've found that restoring database .MYI/.MYD files is simple - stop mysql, copy files, set permissions and fire up mysql. I've replicated the steps but still doesn't work.
I have a cPanel server with 4GB RAM
My server hangs time to time, once or twice a day. This is the last status when it happens.
Server Load 75.90 (2 cpus)
Memory Used 92 %
Swap Used 65.78 %
When I was still inside SSH when it happens, the processes I can see inside "top" are a lot of httpd processes.
So I "killall -9 httpd", I had to do it 30~40 times
#killall -9 httpd
#killall -9 httpd
#killall -9 httpd
x 30~40 times until no pid process found & the server load is back to normal.
Before that, I check httpd fullstatus, they look very normal, same goes to what I see inside WHM Apache status & cpu usage status.
At first I thought it was a memory problem after consulting with a server admin, so I replaced all 4GB new ram stick (such a waste)
My server is very slow, High CPU, High Apache load....
I got a new server and i had setup a script on it.
No server tweaks has been done.
I run a filehosting site so it gets very slow when traffic is high.
Here is some info:
Code:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4135 filedip 19 0 12812 8892 2212 R 18.9 0.4 0:00.74 index.cgi
3140 apache 15 0 22624 9604 2768 S 17.3 0.5 1:54.45 httpd
3556 apache 15 0 22792 9592 2768 S 17.3 0.5 0:43.53 httpd
3739 apache 15 0 22672 9556 2744 S 17.3 0.5 1:09.67 httpd
3884 apache 15 0 22664 9560 2744 S 17.3 0.5 0:23.98 httpd
3474 apache 15 0 22672 9572 2756 R 16.1 0.5 1:34.09 httpd
3548 apache 15 0 22792 9560 2740 S 16.1 0.5 1:33.67 httpd
3991 apache 15 0 22640 9588 2756 S 15.1 0.5 0:22.44 httpd
3475 apache 15 0 22672 9580 2756 S 13.2 0.5 1:31.42 httpd
3493 apache 15 0 22664 9592 2768 S 12.9 0.5 1:15.52 httpd
3769 apache 15 0 22664 9580 2756 S 12.9 0.5 0:45.19 httpd
3638 apache 15 0 22664 9576 2756 S 12.3 0.5 0:32.82 httpd
3724 apache 16 0 22664 9580 2756 R 12.3 0.5 0:34.25 httpd
3626 apache 15 0 22672 9580 2756 S 11.7 0.5 0:52.54 httpd
4023 apache 16 0 22792 9560 2740 R 11.3 0.5 0:13.16 httpd
3882 apache 15 0 22664 9576 2756 S 10.1 0.5 0:38.56 httpd
3005 apache 15 0 22672 9588 2768 R 9.1 0.5 2:06.61 httpd
I dont know much about server managment...
how to fix this so server doesnt go slow?
I have a site that is eating up my server resources and need to know what the best solution for this is. I'm thinking of getting another server just for mysql but do not know what specs the server should be to handle the current traffic/database load and have the site run smoothly without slowing down to a snail's pace.
An alternative is to get another server just for the videos being served and leave the database and html on the current server. This is where I'm stuck and don't know what route to take with this.
My current server stats:
Dual Xeon 5130
4GB RAM
250GB
50 Mbit/sec
CentOS 3.9
Website traffic stats:
15,000 visitors/day
150,000 pageviews/day
Serving videos
I've attached screenshots of top and bandwidth usage per day. Hopefully with this information you could tell me if I need another server or if there are any things I can do to the current server to help things move faster.
I have one server setup to power my forum and its database. It has been running from this server for a while now and as the site gets bigger i have decided i want to move the database to a seperate server.
I have another server at Ecatel and wanted to know if its possible to do this, as they are in seperate datacenters.. or is it only possible between two servers on a private network in the same DC?
I just added a database server in private network and moved the database for Vbulletin Forum to this server.
But some how, the Forum is loading extremely slow compare to before ( when it was on the localhost). Also, Compare to another website on server (using local database) it is much slower.
One thing good is the load is lower
2 servers are connected via 10mbs private link, both servers are at Softlayer.
webserver: Opteron 170, 2GB RAM, 73GB SCSI 15K
database: Quad Xeon 3220, 2GB RAM, 73GB SCSI 15K.
I am running in a High load problem lately. I have one of those cheap 1and1 servers which was running fine until 2 weeks ago. Once I rebooted accidentaly, it did not come back with some unrepairable kernel errors and I had to re-image it.
I chose to reimage the server with CentOS 5, for better support. The new re-image worked fine for some days, at least so I thought and now I am having high loads. The server crashes if not monitored every moment as the load is unpredictable.
Just a restart of the Apache will bring the server back to normality, but I am not sure if it is apache or some other script to be blamed.
I have beeing monitoring through apache server-status, but I cannot organize something unusual in the high load moments.
At this moment for example:
top - 09:00:36 up 15:43, 3 users, load average: 3.57, 4.60, 5.32
Tasks: 144 total, 2 running, 142 sleeping, 0 stopped, 0 zombie
Cpu(s): 10.6%us, 6.5%sy, 0.0%ni, 18.7%id, 63.5%wa, 0.2%hi, 0.5%si, 0.0%st
12:00:29 AM CPU %user %nice %system %iowait %steal %idle
12:10:01 AM all 9.14 0.00 5.52 44.66 0.00 40.68
12:20:14 AM all 6.83 0.00 3.98 27.88 0.00 61.32
12:30:10 AM all 6.44 0.00 4.20 81.25 0.00 8.11
12:40:09 AM all 5.25 0.00 4.09 81.93 0.00 8.73
12:50:15 AM all 5.11 0.00 3.79 90.74 0.00 0.36
01:00:07 AM all 7.22 0.00 4.52 57.11 0.00 31.15
01:10:13 AM all 6.89 0.00 4.01 55.38 0.00 33.71
01:20:14 AM all 4.37 0.00 3.27 41.88 0.00 50.48
01:30:25 AM all 4.26 0.00 3.29 63.42 0.00 29.03
01:40:06 AM all 27.18 0.00 4.75 58.27 0.00 9.80
01:50:03 AM all 29.64 0.00 6.61 51.50 0.00 12.25
02:00:07 AM all 27.00 0.00 8.48 55.49 0.00 9.03
02:10:10 AM all 19.29 0.00 4.97 73.80 0.00 1.94
02:20:04 AM all 37.85 0.00 6.78 40.70 0.00 14.67
02:30:05 AM all 15.65 0.00 4.80 68.47 0.00 11.08
02:40:08 AM all 9.06 0.00 5.60 37.49 0.00 47.86
02:50:07 AM all 5.36 0.00 3.62 42.29 0.00 48.73
03:00:02 AM all 6.05 0.00 4.08 47.27 0.00 42.60
03:10:02 AM all 4.22 0.00 3.68 38.17 0.00 53.93
03:20:02 AM all 4.06 0.00 3.75 41.37 0.00 50.82
03:30:22 AM all 4.42 0.00 3.93 45.25 0.00 46.41
03:40:11 AM all 4.34 0.00 3.95 39.58 0.00 52.13
03:50:02 AM all 4.67 0.00 4.01 32.53 0.00 58.80
04:00:08 AM all 3.72 0.00 3.87 28.40 0.00 64.02
04:10:02 AM all 13.49 0.00 6.58 20.82 0.00 59.10
04:20:01 AM all 6.70 0.00 4.63 6.06 0.00 82.61
04:30:02 AM all 1.44 0.00 1.21 4.75 0.00 92.59
04:40:01 AM all 12.42 0.00 8.12 7.65 0.00 71.81
04:50:02 AM all 1.43 0.00 1.07 4.02 0.00 93.47
05:00:02 AM all 1.60 0.00 1.40 8.62 0.00 88.38
05:10:10 AM all 3.80 0.00 3.02 17.86 0.00 75.32
05:20:06 AM all 5.10 0.00 4.22 23.34 0.00 67.34
05:30:02 AM all 1.54 0.00 1.40 11.22 0.00 85.85
05:40:05 AM all 1.75 0.00 1.89 13.12 0.00 83.23
05:50:12 AM all 2.15 0.00 2.22 18.92 0.00 76.72
06:00:02 AM all 1.92 0.00 2.01 12.87 0.00 83.20
06:10:02 AM all 2.27 0.00 2.16 11.53 0.00 84.04
06:20:03 AM all 3.56 0.00 3.02 25.26 0.00 68.16
06:30:10 AM all 2.66 0.00 2.05 18.13 0.00 77.16
06:40:02 AM all 2.58 0.00 2.25 22.87 0.00 72.30
06:50:02 AM all 2.68 0.00 1.92 15.77 0.00 79.63
07:00:03 AM all 3.06 0.00 2.48 26.01 0.00 68.46
07:10:03 AM all 3.65 0.00 3.20 36.54 0.00 56.61
07:10:03 AM CPU %user %nice %system %iowait %steal %idle
07:20:03 AM all 4.40 0.00 3.28 43.86 0.00 48.46
07:30:02 AM all 4.10 0.00 3.17 31.30 0.00 61.43
07:40:06 AM all 7.67 0.00 3.95 50.79 0.00 37.59
07:50:02 AM all 4.72 0.00 3.11 44.30 0.00 47.86
08:00:03 AM all 5.57 0.00 3.72 47.15 0.00 43.56
08:10:07 AM all 10.66 0.00 3.59 71.62 0.00 14.13
08:20:17 AM all 5.67 0.00 3.42 58.81 0.00 32.10
08:30:10 AM all 11.12 0.00 3.49 76.71 0.00 8.67
08:40:03 AM all 7.00 0.00 3.36 47.94 0.00 41.71
Average: all 7.53 0.00 3.76 38.90 0.00 49.81
Some configurations:
The reimage partittioning looks like this:
/dev/sda1 950M 138M 765M 16% /
/dev/sda5 4.7G 1.9G 2.9G 40% /usr
/dev/sda7 63G 58G 5.0G 93% /var
/dev/sda6 4.7G 23M 4.7G 1% /home
none 1.0G 500K 1.0G 1% /tmp
KeepAlive On
MaxKeepAliveRequests 300
KeepAliveTimeout 15
<IfModule prefork.c>
StartServers 10
MinSpareServers 20
MaxSpareServers 50
ServerLimit 256
MaxClients 180
MaxRequestsPerChild 2000
</IfModule>
<IfModule worker.c>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-locking
query_cache_limit=1M
query_cache_size=32M
query_cache_type=1
max_connections=250
interactive_timeout=100
wait_timeout=20
connect_timeout=10
thread_cache_size=80
key_buffer=64M
join_buffer=1M
max_allowed_packet=16M
table_cache=512
record_buffer=1M
sort_buffer_size=1M
read_buffer_size=1M
max_connect_errors=10
thread_concurrency=2
read_rnd_buffer_size=1M
myisam_sort_buffer_size=64M
server-id=1
processor : 1
vendor_id : GenuineIntel
cpu family : 15
model : 3
model name : Intel(R) Pentium(R) 4 CPU 2.80GHz
stepping : 4
cpu MHz : 2793.324
cache size : 1024 KB
I have extremely high CPU load average, but when I run TOP command there are not any process with high CPU
CPU load averages:
211.65 (1 min) 227.29 (5 mins) 247.89 (15 mins)
Tasks: 364 total, 2 running, 360 sleeping, 2 stopped, 0 zombie
Cpu(s): 0.0%us, 0.3%sy, 0.0%ni, 0.0%id, 99.7%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2065460k total, 2012372k used, 53088k free, 60964k buffers
Swap: 2040212k total, 3280k used, 2036932k free, 619900k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19871 root 15 0 4944 2488 1656 R 1 0.1 0:02.73 top
1 root 15 0 4208 1320 1136 S 0 0.1 0:01.55 init
2 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0
3 root 34 19 0 0 0 S 0 0.0 0:01.77 ksoftirqd/0
4 root RT 0 0 0 0 S 0 0.0 0:00.01 migration/1
5 root 34 19 0 0 0 S 0 0.0 0:01.16 ksoftirqd/1
6 root 10 -5 0 0 0 S 0 0.0 0:01.04 events/0
7 root 10 -5 0 0 0 S 0 0.0 0:00.65 events/1
8 root 10 -5 0 0 0 S 0 0.0 0:00.00 khelper
9 root 11 -5 0 0 0 S 0 0.0 0:00.00 kthread
35 root 10 -5 0 0 0 S 0 0.0 0:00.15 kblockd/0
36 root 10 -5 0 0 0 S 0 0.0 0:00.00 kblockd/1
37 root 15 -5 0 0 0 S 0 0.0 0:00.00 kacpid
127 root 10 -5 0 0 0 S 0 0.0 0:13.16 ata/0
128 root 16 -5 0 0 0 S 0 0.0 0:00.00 ata/1
129 root 10 -5 0 0 0 D 0 0.0 0:01.15 ata_aux
130 root 10 -5 0 0 0 S 0 0.0 0:00.00 kseriod
158 root 16 0 0 0 0 S 0 0.0 0:00.00 pdflush
159 root 15 0 0 0 0 D 0 0.0 0:01.00 pdflush
160 root 10 -5 0 0 0 S 0 0.0 0:10.68 kswapd0
161 root 11 -5 0 0 0 S 0 0.0 0:00.00 aio/0
So my server is "unresponsive" for abour 18 hours, burst net didnt answer my tickets and I dont know what to do. Ive been with this setup for almost 5 months with no problems, No changes have been made to hardware or software.
Im using Centos 5.0
My VPS holds about 80 domains and low-use accounts.
Every night, from around 1.30am, the load suddenly skyrockets and will usually be around 5 to 10 for a few hours. Occasionally it'll spike to 30+ for a few minutes.
I had some antispam software running, and a couple of other packages (mail queues, mail manage etc), so I disabled all of that and removed all the crontab entries etc.
It's not really made any difference.
I can see the load stats going back 8 hours, as part of the ASSP spam package (I've just left the ASSP server load cron running just so I can continue monitoring it!)
Can the apparent load on my VPS be caused by other VPS's on the same node?? So in reality, my load is fine but is being affected by other people's VPS's?
I hope that makes sense. I'm 99% sure that my VPS is 'clean' (in so far as cron entries)
I'm asking the question because I took a second VPS on the same node and that one too has high loads overnight when there's nothing running on it (ie, no add-on software, no Cpanel accounts added)
I have a VPS with LXadmin control panel and 1024 Ram.
the memory always 333-450 but now it is more than 800 and the VPS stopped I restarted it many times but after restarting it the memory gone crazy again.
So, how to know what causes high load and eating my Memory?
when i install BFD and APF on my server thie processes appears every time on "top" commad on my server and it cuse hight load
------------------------------
4766 root 25 0 100 10:22.56 3.2 165m 130m 420 R grep -vf /usr/local/bfd/tmp/attack.pool.tmp
7747 root 25 0 100 30:20.34 9.8 430m 397m 420 R grep -vf /usr/local/bfd/tmp/attack.pool.tmp
----------------------------
Just signed up for a VPS with solarvps, which so far is lightning fast compared to my shared reseller account with RZ.
I want to create a script which checks the server load and drops me a text message on my mobile if the server load hits a certain level.
Just wondering what I should consider a high server load?
The server is the JupiterLX UK one from solar vps with over just over 800mb of SLM RAM.
Im in a VPS and everyday at the same hour goes very slowly due to high load problems.
The main site is a vBulletin forum and I get this error:
Too many connections
Ive change some details in my.cnf:
Quote:
max_connections=500
max_user_connections=70
But it didnt solve the problem.
This high load occurs at the same hour, I think that it is a cron job related problem, but this is my /etc/crontab
Quote:
# cat crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly
i was previously using maxclients 256 in httpd.conf and the load was normal even when the server was processing 256 requests..
recently i had changed it to 320 coz of which my load had increased a lot.. Now when i have decreased it back to 256 the load is still high even when the no of requests are just 150..
i have trouble with transport from company to another company
and i had problem with the best company ( resellerzoom.com )
i order one month this palnt ( Failover Plans ) then i swa this messeage
[PHP]
Hello,
Accounts usage is very high, MEMORY usage is very high than allowed limit of 3% for Daily average. You still have Cpanel/FTp access.
========================
Account usage Daily average
Average CPU%:- 1.23
Average MEM%:- 3.24
Top Process %CPU 58.0
PHP Code:
Top Process %CPU 54.0 /usr/bin/php forumdisplay.phpTop Process %CPU 52.0 /usr/bin/php showthread.php
i asked him what's can i do?
he toled me
PHP Code:
Upgrade to VPS or Semi-dedicated server
i don't want Upgrade vps with same company
How can you tell if your server was a victim of a DDOS attack? Server load goes so high that you cannot get in via SSH, and the server has to be power cycled to get back online. When it comes back online, load goes back up, we shut apache down, and the let the load come back down and then restart apache and everything is fine. Any way to determine what caused the load to skyrocket? System has APF, BFD, and mod_evasive installed.
View 3 Replies View RelatedI have 2 servers
Dual Quad Core 5310
One for Mysql and the other one for Httpd
I have one large website , when i put this website in one server i mean by one server is making the website work with mysql and httpd in one server no external server it's load seems to be on 1 - 5 but when i but the mysql on other server the load got high and i got high httpd connections , also when i try to see the the amount connections between the servers it's too high in one of them and low in the other and i am always suffering from load .
here the details
192.168.0.1 the mysql server
192.168.0.2 httpd server
[root@httpd etc]# netstat -n | grep 192.168 | wc -l
453
this the mysql server my.cnf conf
[root@mysql ~]# cat /etc/my.cnf
[mysqld]
safe-show-database
skip-name-resolve
skip-innodb
max_connections = 800
key_buffer = 96M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 2500
thread_cache_size = 384
wait_timeout = 30
connect_timeout = 10
tmp_table_size = 128M
max_heap_table_size = 128M
max_allowed_packet = 64M
max_connect_errors = 10
thread_concurrency = 8
read_rnd_buffer_size = 1M
bulk_insert_buffer_size = 8M
query_cache_limit = 8M
query_cache_size = 128M
query_cache_type = 1
query_prealloc_size = 131072
query_alloc_block_size = 65536
[mysqld_safe]
nice = -10
open_files_limit = 8192
[mysqldump]
quick
max_allowed_packet = 16M
[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M
[mysqlhotcopy]
interactive-timeout
[root@mysql ~]#
this the httpd.conf config
Timeout 30
KeepAlive On
MaxKeepAliveRequests 120
KeepAliveTimeout 1
MinSpareServers 15
MaxSpareServers 25
StartServers 15
MaxClients 256
MaxRequestsPerChild 1000
here the total connections in the mysql server
[root@mysql ~]# netstat -n | grep 192.168 | wc -l
25
[root@mysql ~]#
[root@mysql ~]# netstat -n | grep 192.168
tcp 0 0 192.168.0.1:3306 192.168.0.2:47876 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:48923 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:46610 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47142 TIME_WAIT
tcp 0 0 192.168.0.1:3306 192.168.0.2:49209 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:44344 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:44365 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:44359 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:46406 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:44355 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:44352 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:48217 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47208 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:46463 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47229 TIME_WAIT
tcp 0 0 192.168.0.1:3306 192.168.0.2:46459 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47987 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:46469 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:42371 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47259 TIME_WAIT
tcp 0 0 192.168.0.1:3306 192.168.0.2:47252 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:46480 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47275 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47293 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47291 TIME_WAIT
tcp 0 0 192.168.0.1:3306 192.168.0.2:47311 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47304 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47303 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:46530 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:49089 ESTABLISHED
tcp 0 100740 192.168.0.1:3306 192.168.0.2:47318 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47315 ESTABLISHED
tcp 0 0 192.168.0.1:3306 192.168.0.2:47312 ESTABLISHED
[root@mysql ~]#
Do you see the high different number between connections !!
I am sorry for bad english but i am really get more pain daily with the high load
My server kernel
Linux httpd 2.6.18-8.1.8.el5 #1 SMP Tue Jul 10 06:50:22 EDT 2007 i686 i686 i386 GNU/Linux
I use CentOS 5 and httpd 1.3 and php 4.4.7
I try to upgrade them to httpd 2.0 and php 5.2.4 but most users can't browse the site so
i come back to 1.3
I have upgraded the kernel, I have optimized the my.cnf (mysql), the swap file is free.
The memory is not overloaded even then the I/O wait is too high, the top result is as below
The server become very very slow due to high I/O load, I am failed to dins out the reason. Please help me with the commands so I can sort it out
top - 06:09:17 up 13 min, 2 users, load average: 7.51, 8.26, 5.35
Tasks: 179 total, 1 running, 178 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.7% us, 2.0% sy, 0.0% ni, 0.0% id, 95.0% wa, 0.3% hi, 0.0% si
Mem: 1027536k total, 1014232k used, 13304k free, 114232k buffers
Swap: 2040244k total, 0k used, 2040244k free, 242300k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3474 apache 16 0 92232 54m 5780 D 0.7 5.4 0:04.01 httpd
2914 named 18 0 39448 4404 1964 S 0.3 0.4 0:01.44 named
3067 mysql 15 0 283m 42m 4288 S 0.3 4.3 0:03.87 mysqld
3826 popuser 15 0 34572 29m 2348 D 0.3 2.9 0:10.69 spamd
3876 apache 15 0 93716 55m 5512 S 0.3 5.5 0:03.41 httpd
7049 root 17 0 10508 2572 1996 S 0.3 0.3 0:00.05 sshd
7161 root 26 10 12712 5804 2656 S 0.3 0.6 0:00.37 mytop
7651 root 16 0 2784 1020 760 R 0.3 0.1 0:00.03 top
7688 popuser 16 0 6068 932 740 D 0.3 0.1 0:00.01 imapd
1 root 16 0 2588 552 472 S 0.0 0.1 0:00.72 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/0
4 root 9 -10 0 0 0 S 0.0 0.0 0:00.00 khelper
5 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 kacpid
19 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 kblockd/0
20 root 15 0 0 0 0 S 0.0 0.0 0:00.00 khubd
37 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pdflush
38 root 15 0 0 0 0 S 0.0 0.0 0:00.03 pdflush
39 root 15 0 0 0 0 S 0.0 0.0 0:00.01 kswapd0
40 root 11 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0
186 root 25 0 0 0 0 S 0.0 0.0 0:00.00 kseriod
415 root 15 0 0 0 0 D 0.0 0.0 0:00.32 kjournald
1606 root 7 -10 0 0 0 S 0.0 0.0 0:00.00 ata/0
1607 root 7 -10 0 0 0 S 0.0 0.0 0:00.00 ata_aux
1617 root 6 -10 2044 448 380 S 0.0 0.0 0:00.01 udevd
1850 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 kauditd
1915 root 19 0 0 0 0 S 0.0 0.0 0:00.00 kjournald
1916 root 19 0 0 0 0 S 0.0 0.0 0:00.00 kjournald
2698 root 15 0 3540 548 456 S 0.0 0.1 0:00.26 syslogd
2702 root 16 0 1564 384 316 S 0.0 0.0 0:00.00 klogd
2729 rpc 15 0 3560 548 452 S 0.0 0.1 0:00.00 portmap
2748 rpcuser 18 0 2348 724 620 S 0.0 0.1 0:00.00 rpc.statd
2774 root 16 0 5944 344 172 S 0.0 0.0 0:00.00 rpc.idmapd
2845 root 19 0 2568 472 264 S 0.0 0.0 0:00.00 smartd
2854 root 18 0 2928 444 368 S 0.0 0.0 0:00.00 acpid
2863 root 16 0 9548 2008 1456 S 0.0 0.2 0:00.02 cupsd
2957 root 16 0 5440 1132 832 S 0.0 0.1 0:00.04 sshd
eveyday i see my vps high load
1st problem <<<<
now i show this error for load in CPU/Memory/MySQL Usage
--------------------------------------------
/usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/server.nogomhost.net.pid --skip-external-locking
---------------------------------------------
this " mysql " used 90% and 80 % from CPU
And all account used sql 0.0 , why load mysql ?
2nd problem <<<<
some times my vps is down , i don't why
3rd prblem <<<
my vps is not working to some clients , i make allow after user's sent to me ip's by apf program
i'm Disable Backup " Restore only "
top - 00:21:30 up 4 min, 1 user, load average: 20.30, 6.74, 2.42
Tasks: 221 total, 12 running, 209 sleeping, 0 stopped, 0 zombie
Cpu(s): 33.7% us, 4.4% sy, 0.0% ni, 0.0% id, 61.8% wa, 0.1% hi, 0.0% si
Mem: 1025804k total, 639076k used, 386728k free, 1296k buffers
Swap: 2031608k total, 22584k used, 2009024k free, 23804k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3359 apache 16 0 30968 16m 5752 S 11.9 1.7 0:00.61 httpd
3065 apache 16 0 31092 16m 5776 S 5.0 1.7 0:01.38 httpd
2478 apache 16 0 29068 14m 5680 S 4.0 1.5 0:00.80 httpd
2822 apache 16 0 28420 14m 5756 D 4.0 1.4 0:00.59 httpd
2810 apache 16 0 31412 17m 5764 R 3.0 1.7 0:01.72 httpd
3111 apache 15 0 29312 14m 5756 D 3.0 1.5 0:00.76 httpd
3787 apache 16 0 29164 14m 5668 S 3.0 1.4 0:00.09 httpd
2816 apache 16 0 31556 17m 5776 S 2.6 1.7 0:01.20 httpd
3063 apache 16 0 29740 15m 5760 S 2.3 1.5 0:01.65 httpd
3226 apache 15 0 31132 16m 5772 D 2.3 1.7 0:01.62 httpd
3007 apache 16 0 29160 14m 5744 S 2.0 1.5 0:00.59 httpd
3058 apache 16 0 31612 17m 5764 S 2.0 1.7 0:00.92 httpd
3064 apache 15 0 30300 15m 5764 S 2.0 1.6 0:01.59 httpd
3481 apache 16 0 29032 14m 5752 S 2.0 1.5 0:00.15 httpd
3503 apache 16 0 29076 14m 5756 S 2.0 1.5 0:00.48 httpd
2589 apache 15 0 28980 14m 5760 S 1.7 1.4 0:00.94 httpd
3069 apache 15 0 30900 16m 5760 S 1.7 1.7 0:01.92 httpd
httpd.conf
Quote:
ServerType standalone
ServerRoot "/etc/httpd"
LockFile /var/run/httpd.lock
PidFile /var/run/httpd.pid
ScoreBoardFile logs/apache_runtime_status
ResourceConfig /dev/null
AccessConfig /dev/null
Timeout 30
KeepAlive Of
MaxKeepAliveRequests 100
KeepAliveTimeout 5
MinSpareServers 8
MaxSpareServers 18
StartServers 10
MaxClients 100
MaxRequestsPerChild 500
I am using rsync to transfer files (tar.gz) between servers. However, it makes server load increasing 3-4. Normally, server load can be around 1, but when doing the transfer, it can go up to 5+
Is there anyway to reduce the load when doing rsync?
I am just a beginner and it is about a month that the mysql loads on my server is so high , I have checked the mysql processlist via cpanel and there is no account that takes high mysql and make it high , and even I have checked for any possible rotten databases. but still the server load is so high and I can not control it , even when I restart it just after a minute it goes up again.
just the spamd command takes a high CPU usage sometimes. what it should be from? the spamd for a special account!
I need to optimize mysql , and need help. please do not tell me to hire an expert , just help me thanks
and another question is that how can I check which account is sending spam and stop it?
I have to move about 50 GB each night from one server to another.
This is the command I'm using:
Code:
/bin/nice -n +19 scp -c blowfish -l 18000 -P 22222 root@XX.XX.XX.XX:/backup/dailybacks/*.tar.gz ./
There is no private lan so i have to use internet. I'm using blowfish, which will reduce cpu load, and also limiting bandwidth transfer, usign -l 18000. How ever, I still do see some high load averages..
Do you have any other suggestion to optimize CPU performance while running scp?
Why my server - load average is high?
below is output of top:
top - 13:52:53 up 29 days, 5:39, 3 users, load average: 72.05, 68.90, 67.89
Tasks: 857 total, 2 running, 833 sleeping, 20 stopped, 2 zombie
Cpu(s): 4.2%us, 2.5%sy, 0.0%ni, 26.4%id, 63.6%wa, 0.5%hi, 2.8%si, 0.0%st
Mem: 8181908k total, 8135188k used, 46720k free, 35104k buffers
Swap: 4192956k total, 26624k used, 4166332k free, 7321172k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
337 mysql 15 0 327m 56m 4252 S 4.0 0.7 74:18.53 mysqld
24732 root 15 0 13380 1760 808 R 1.0 0.0 0:10.44 top
313 root 10 -5 0 0 0 S 0.3 0.0 90:25.28 kswapd0
22040 nobody 15 0 63148 3272 1572 S 0.3 0.0 0:00.30 httpd
22136 nobody 15 0 63148 3260 1572 S 0.3 0.0 0:00.21 httpd
22291 nobody 15 0 63148 3280 1580 S 0.3 0.0 0:00.18 httpd
22327 nobody 16 0 63148 3280 1580 D 0.3 0.0 0:00.20 httpd
22406 nobody 15 0 63148 3276 1580 S 0.3 0.0 0:00.18 httpd
22522 nobody 16 0 63148 3268 1580 S 0.3 0.0 0:00.10 httpd
23553 nobody 16 0 63148 3244 1556 S 0.3 0.0 0:00.09 httpd
23556 nobody 15 0 63148 3256 1568 S 0.3 0.0 0:00.09 httpd
26260 nobody 16 0 63148 3296 1608 S 0.3 0.0 0:00.13 httpd
26618 nobody 16 0 63148 3244 1556 S 0.3 0.0 0:00.13 httpd
28849 nobody 16 0 63148 3240 1556 S 0.3 0.0 0:00.04 httpd
28903 nobody 15 0 63148 3208 1532 S 0.3 0.0 0:00.02 httpd
29533 nobody 15 0 0 0 0 Z 0.3 0.0 0:00.02 httpd <defunct>
29558 nobody 15 0 63148 3236 1548 S 0.3 0.0 0:00.01 httpd
29587 nobody 15 0 63148 3216 1536 S 0.3 0.0 0:00.01 httpd
29592 nobody 15 0 63148 3208 1532 S 0.3 0.0 0:00.02 httpd
1 root 15 0 10344 576 544 S 0.0 0.0 0:05.36 init
2 root RT -5 0 0 0 S 0.0 0.0 3:58.90 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 6:15.52 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT -5 0 0 0 S 0.0 0.0 0:12.33 migration/1
6 root 34 19 0 0 0 S 0.0 0.0 0:30.07 ksoftirqd/1
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
8 root RT -5 0 0 0 S 0.0 0.0 0:02.57 migration/2
9 root 34 19 0 0 0 S 0.0 0.0 0:02.84 ksoftirqd/2
10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2
11 root RT -5 0 0 0 S 0.0 0.0 0:11.61 migration/3
12 root 34 19 0 0 0 S 0.0 0.0 0:01.12 ksoftirqd/3
13 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/3
14 root 10 -5 0 0 0 S 0.0 0.0 0:00.22 events/0
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.10 events/1
16 root 10 -5 0 0 0 S 0.0 0.0 0:00.16 events/2
17 root 10 -5 0 0 0 S 0.0 0.0 0:00.09 events/3
18 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
87 root 10 -5 0 0 0 S 0.0 0.0 0:00.03 kthread
94 root 10 -5 0 0 0 S 0.0 0.0 0:00.78
