i run a fedora server with plesk 8 on it i was wondering if some one can help me to find the best way to optimize the databases on the server at once my servers contains many clients with unicode data in the database.
So the site got featured on [url]and now the server is drowning...
The Coppermine Gallery usually hovers around 30~50 users daily and now, 1800, and im at a lost as how I should configure mysql to take on such a load. right now it takes about 10 secs or more to load a page and sometimes it would time out. Because it si coppermine, all pages are dynamic and can't be cached -_-"
Here's the my.cnf right now after i played around with the numbers
server spec Opteron 170 (2ghz) 2gb ram 250 7200rpm
# # KeepAlive: Whether or not to allow persistent connections (more than # one request per connection). Set to "Off" to deactivate. # KeepAlive Off
# # MaxKeepAliveRequests: The maximum number of requests to allow # during a persistent connection. Set to 0 to allow an unlimited amount. # We recommend you leave this number high, for maximum performance. # MaxKeepAliveRequests 100
# # KeepAliveTimeout: Number of seconds to wait for the next request from the # same client on the same connection. # KeepAliveTimeout 15
# prefork MPM # StartServers: number of server processes to start # MinSpareServers: minimum number of server processes which are kept spare # MaxSpareServers: maximum number of server processes which are kept spare # ServerLimit: maximum value for MaxClients for the lifetime of the server # MaxClients: maximum number of server processes allowed to start # MaxRequestsPerChild: maximum number of requests a server process serves <IfModule prefork.c> StartServers 8 MinSpareServers 5 MaxSpareServers 20 ServerLimit 200 MaxClients 200 MaxRequestsPerChild 1500 </IfModule>
# worker MPM # StartServers: initial number of server processes to start # MaxClients: maximum number of simultaneous client connections # MinSpareThreads: minimum number of worker threads which are kept spare # MaxSpareThreads: maximum number of worker threads which are kept spare # ThreadsPerChild: constant number of worker threads in each server process # MaxRequestsPerChild: maximum number of requests a server process serves <IfModule worker.c> StartServers 2 MaxClients 150 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule>
I have a fairly high end server in which I have installed SIM. SIM is restarting Apache up to 10 times a day, presumably due to high load causing un-availability.
On restart, Apache / MYSQL is stable until the load / mem usage begins to climb then it is restarted again. Here are my 'load' stats for today:
Load for today High (2:18am): 4.63 Low (3:30am): 1.20 Mean: 1.84 Latest: 1.61
Mem usage for today High (1:36am): 9,192.9 MB Low (1:48am): 7,995.7 MB Mean: 8,683.1 MB Latest: 8,781.7 MB
I have seen it using 20GB RAM before.
I have tried to follow various optimisation guides but these seem tailored to less powerful servers.
The web application I run on this server is almost entirely MYSQL based, with thousands of DB calls a day. Across the entire system I probably get 200,000 bot hits per day or even more. At peak times search engine bots are literally hammering the server.
Can anyone recommend a Apached and MYSQL optimization expert? I just some optimization done on mysql and apache upgraded... but pache wouldn't restart with the following entries:
if i remove above then it starts... but my CPU spikes and I am getting a ton of errors emailed to me:
Invalid SQL: SELECT COUNT(DISTINCT(userid)) AS count FROM vb_session WHERE vb_session.userid>0 AND vb_session.lastactivity>1245294346;
MySQL Error : Out of memory (Needed 8388580 bytes) Error Number : 5 Request Date : Wednesday, June 17th 2009 @ 09:39:06 PM Error Date : Wednesday, June 17th 2009 @ 09:39:06 PM
Its constantly complaining about this same query and i thought i removed all of the vbulletin options...
i am pulling my hair out... cuz my server isn't swapping, ther are 0ver 8GBs in cache.... but my CPU goes up
Here is original problem thread...
[url]
2nd post has recommended config... and my subsequent probs
I am recommending to a friend who is starting a proxy site, to do a successful launch by getting a dedicated server to host the single proxy domain.
He does not want to spend more than $40 per month.
Should he be considering a VPS, or are proxy usually against TOC of a VPS?
I have found a low end server at OLM.net with plenty of bandwidth allocation for $29.95. Is it powerful enough to sustain traffic from a proxy site that is just starting out?
just got a new server and got my site up and running on it perfectly but ive noticed its a bit slower than my old server and i think this is down to my incorrctly optimized mysql setup. ive used the my.cnf off my old server with some minor edits but im wondering what people can suggest i change for the following server spec
AMD Athlon 64 X2 4200+ 2GB DDR2 120gb Sata hd
currently my my.cnf is as follows
Quote:
# This is for a system with little memory (32M - 64M) where MySQL plays # an important part, or systems up to 128M where MySQL is used together with # other programs (such as a web server) # # You can copy this file to # /etc/my.cnf to set global options, # mysql-data-dir/my.cnf to set server-specific options (in this # installation this directory is /var/lib/mysql) or # ~/.my.cnf to set user-specific options. # # In this file, you can use all long options that a program supports. # If you want to know which options a program supports, run the program # with the "--help" option.
# The following options will be passed to all MySQL clients [client] #password = your_password port = 3306 socket = /var/lib/mysql/mysql.sock
according to mysql tuning primer it reports the following
Quote:
-- MYSQL PERFORMANCE TUNING PRIMER -- - By: Matthew Montgomery -
MySQL Version 4.1.20-log x86_64
Uptime = 0 days 17 hrs 52 min 44 sec Avg. qps = 15 Total Questions = 1028682 Threads Connected = 1
Warning: Server has not been running for at least 48hrs. It may not be safe to use these recommendations
To find out more information on how each of these runtime variables effects performance visit: [url] Visit [url] for info about MySQL's Enterprise Monitoring and Advisory Service
SLOW QUERIES Current long_query_time = 5 sec. You have 0 out of 1028694 that take longer than 5 sec. to complete The slow query log is NOT enabled. Your long_query_time seems to be fine
WORKER THREADS Current thread_cache_size = 256 Current threads_cached = 3 Current threads_per_sec = 0 Historic threads_per_sec = 0 Your thread_cache_size is fine
MAX CONNECTIONS Current max_connections = 1000 Current threads_connected = 1 Historic max_used_connections = 4 The number of used connections is 0% of the configured maximum. You are using less than 10% of your configured max_connections. Lowering max_connections could help to avoid an over-allocation of memory See "MEMORY USAGE" section to make sure you are not over-allocating
MEMORY USAGE Max Memory Ever Allocated : 243 M Configured Max Per-thread Buffers : 41 G Configured Max Global Buffers : 75 M Configured Max Memory Limit : 41 G Physical Memory : 1.90 G
Max memory limit exceeds 90% of physical memory
KEY BUFFER Current MyISAM index space = 12 M Current key_buffer_size = 1 M Key cache miss rate is 1 : 14581 Key buffer fill ratio = 44.00 % Your key_buffer_size seems to be too high. Perhaps you can use these resources elsewhere
QUERY CACHE Query cache is enabled Current query_cache_size = 64 M Current query_cache_used = 5 M Current query_cache_limit = 64 M Current Query cache Memory fill ratio = 8.42 % Current query_cache_min_res_unit = 4 K Query Cache is 15 % fragmented Run "FLUSH QUERY CACHE" periodically to defragment the query cache memory If you have many small queries lower 'query_cache_min_res_unit' to reduce fragmentation. Your query_cache_size seems to be too high. Perhaps you can use these resources elsewhere MySQL won't cache query results that are larger than query_cache_limit in size
SORT OPERATIONS Current sort_buffer_size = 32 M Current record/read_rnd_buffer_size = 1020 K Sort buffer seems to be fine
JOINS Current join_buffer_size = 3.00 M You have had 378 queries where a join could not use an index properly You should enable "log-queries-not-using-indexes" Then look for non indexed joins in the slow query log. If you are unable to optimize your queries you may want to increase your join_buffer_size to accommodate larger joins in one pass.
Note! This script will still suggest raising the join_buffer_size when ANY joins not using indexes are found.
OPEN FILES LIMIT Current open_files_limit = 50000 files The open_files_limit should typically be set to at least 2x-3x that of table_cache if you have heavy MyISAM usage. Your open_files_limit value seems to be fine
TABLE CACHE Current table_cache value = 2000 tables You have a total of 908 tables You have 102 open tables. The table_cache value seems to be fine
TEMP TABLES Current max_heap_table_size = 31 M Current tmp_table_size = 30 M Of 3666 temp tables, 0% were created on disk Created disk tmp tables ratio seems fine
TABLE SCANS Current read_buffer_size = 5 M Current table scan ratio = 1544 : 1 read_buffer_size seems to be fine
TABLE LOCKING Current Lock Wait ratio = 1 : 4664 You may benefit from selective use of InnoDB.
The servers load is vey bad, most of the time 7-8+ and the IO WAIT more than 60% which is decreasing server performace. Please help me to optimzie the MySQL specially. If you have the good my.cnf config please give me the complete config file.
[client] port = 3306 socket = /var/lib/mysql/mysql.sock
someone to look at my DB and optimize it as it is running ridiculously high loads of 25. It is vbulletin. I'm only looking for professional companies for this job.
I have a friend which owns a 2k alexa rank forum using VB
The server hangs and he gets mysql connection error in logs
Quote:
Database error in vBulletin : mysql_connect() [<a href='function.mysql-connect'>function.mysql-connect</a>]: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock'
The server is a dual Pentium D with 4GB RAM
currently 600 users online and 20% RAM used
in rush hours there are 2000 users online and 50% RAM used
We want to optimize mysql - this is the current my.cnf
I just switched to a new server - I went from a Dual Opteron 248 with 3GB of RAM to a Dual Clovertown with 4GB of RAM. Yet, despite having mild traffic right now, my load is rather high and my memory usage is like 80% (which is ridiculous, since that would mean I would be maxing out on the old box).
For some reason, my servers usually lose access when I get around 400 people on TEXT-BASED, non-MySQL pages? How is that possible? I have really solid servers, yet people running basic Pentiums with 1 or 2GB never run into this issue.
Is there any good, cheap site that would be able to properly optimize mySQL and Apache to lower the load and figure out why the memory usage is so high? Platinum Server Management never really got this done, so even though they're great for other support issues, please don't recommend them.
On one of my servers (Intel P4 2.8Ghz, 1GB RAM, cPanel), I've been tweaking some of the MySQL values in my.cnf and using the MySQL Tuning Primer shell script to evaluate the results.
This is my current my.cnf. I'm running MySQL 5.0.27.
The tuning primer script is telling me the following.
PHP Code:
Current max_heap_table_size = 16 MCurrent tmp_table_size = 32 MOf 3875 temp tables, 52% were created on diskEffective in-memory tmp_table_size is limited to max_heap_table_size.Perhaps you should increase your tmp_table_size and/or max_heap_table_sizeto reduce the number of disk-based temporary tables
What values should I increase (if at all) max_heap_table_size and tmp_table_size to?
this is what SHOW STATUS; commands give me can anyone provide me any helpful tips on what I can adjust in my.cnf to lower my server load which is above 10 during peak hours and mysql is the only thing eating this serve up I have lots og 3gb ram and a 64bit AMD 2800 server
I'm trying to optimize a server (Intel E8300) with 8Gb of RAM and WHM/cPanel administration panel.
Now the server has only 46 accounts in WHM, and about 10.000 unique visitors / day
As mysql is always on top for CPU ussage I have edited 2 values in my.cnf, but this was done in 10 minutes and not to much research, any more help will be appreciated. On this server as I see it's been using MyISAM for storage engine and maybe just a few tabels with InnoDB ....
i am having quite problem with my server i cant get it to optimize it. During peak times the load goes beyond 10 sometimes 20 to 50 sometimes its just under 1. I have my webserver running on P4 Core 2 6300, 1 GB ram, 250 HDD, on CentOS 4.4 (kernel 2.6.9-42.0.10) etc. It has cpanel/whm running Apache 1.3.7, php 4.4, mysql 4.1, mailscanner, eaccelerator, etc i dont know really what other info to show you.
I host about 200 accounts and there are 4 large php forums (not like WHT) running on it. Sometimes pages stop loading especially these forums have to refresh to get page load.
During day there are approx 700-900 connections.
Previously i had phpsuexec and the load never went beyond 4-5, i have removed it to install eaccelerator.
I hope some of you folks could give me some pointers here. I have a server that is used for a gaming clan's purposes. Nothing major as far as games go, but what I got I want running at peak efficiency.
The most important is the Counter-Strike source. I really need the kernel frequency to be tweaked for this. The number one question is can I merely modify the existing config file and recompile that or do I need to start all over? If I could do that is it something like make old or something to that matter?
I am running a couple other servers as well, but there is never a time where all the servers have anyone in them. I keep everything running. After installing some weird stuffs here and there I got the memory usage to go down. I think. Regarding a scenario of having more than one instance of a game, is there any way to share memory between them?
My Swap. Is there a way to force idling games to move to it instead of using physical memory?
The kernel is what I really need to take care of. I'm still somewhat of a n00b and have just switched from centos to debian to top it off. So any tips would make my day. Thanks for reading me.
Hi"Optimization of computing resources has long been an important management issue. One of its aspects concerns server scalability and the question of whether an organization should scale-up or scale out.Assume that the computing performance of the servers can be measured by variable 0 <=p, that their total cost is given by "c" and the relationship between server performance and cost is defined by c=αp^β"
a. What is the cost-performance elasticity(ђ), precisely?
b. What would be the range of values for ђ that would be expected by moore's law and what are its implications?
c. What would be the range of values for ђ that would lead managers to scale-out? Draw a graph and throughly explain the implications.
I'm on a dedicated server and the httpd service of the server keeps going down which results in downtime for the number of hours until i dont realise that the site is down.
So when I realise the site is down, I ask the hosting support to look into the issue, they tell me the server loads are high and ask me to upgrade even when I just upgraded the server last month.
Generally the forum has around (250 - 300 members) + guests visitors online and is in the alexa top 20k sites.
The first thing I would want to do is hire a person who specializes in server optimization but then I have heard a lot of bad things like the server admin stealing databases and selling it in open market.
The second thing I wanted to do is contact vbulletin for server optimization but they need me to give them server specs and I dont know how to obtain them.
I have purchased a proxy website using phproxy script hosted at dedicated server. Its now about 4 months and everything is fine but i want to move my site to another dedicated server provider because current one has some issues.
When i purchased that site seller told me that he has optimized server and installed some scripts {server scripts} to get best out of proxy website. Now seller is out of my contact.
I am very much pleased with current server settings but i need to move. So i want to make sure that i can optimize new server and install same scripts to keep going with current performance.
I only know about http.conf settings and i have noted it ... plz help me about copying every other settings which that guy had done to optimize the server and how can i know which and where those scripts are installed which has made server quite efficient?
Can anyone recommend a good company or someone that would be able to optimize Apache and MySQL on my server? I've slightly configured Apache so that it works well with my server so I'm mainly looking for someone to optimize MySQL (since it's using a lot of cpu).
The server load is pretty high so I'm hoping to reduce it before having to upgrade to a new server.
I'm sure by now most of you have read the Web Server Optimization Guide by Shaw Networks. It came in handy for me when I was first starting out, I reduced my load & memory usage by tonnes. I thought I would make a new thread with an updated How-To.
[mysqlhotcopy] interactive-timeout To Save: CTRL-X Restart Service: "service mysqld restart" or "service mysql restart" or "/etc/rc.d/init.d/mysql restart"
Use Pico (pico /usr/local/apache/conf/httpd.conf) or Download via WinSCP for editing, Change the following settings in your httpd.conf...
Set "Timeout" value to "Timeout 60" Set "KeepAlive" to "KeepAlive on" Set "KeepAliveTimeout" to "KeepAliveTimeout 3" Set "MinSpareServers" to "MinSpareServers 16" Set "MaxSpareServers" to "MaxSpareServers 32" Set "MaxRequestsPerChild" to "MaxRequestsPerChild 256" Set "HostnameLookups" to "HostnameLookups Off"
Note: These settings will not work under all server environments its recommended that you tweak around with the numbers until your web server is running 100% please read Apache documentation before changing any settings so you know what you are changing [url]
To Save: CTRL-X Restart Service: "service httpd restart" or "/etc/rc.d/init.d/httpd restart"
Installing eAccelerator: eAccelerator is a further development from mmcache PHP Accelerator & Encoder. It increases performance of PHP scripts by caching them in compiled state, so that the overhead of compiling is almost completely eliminated. [url]
If you run CPanel please visit [url] for an auto-installer which will do all the hard work for you.
Run the following in SSH.
Code: cd /
mkdir ea
cd ea
wget [url]
bunzip2 eaccelerator-0.9.5.tar.bz2 tar -xvf eaccelerator-0.9.5.tar
I'm currently in the process of installing a mysql database server on my dedicated game server box. I was using shared hosting before, but i got a complaint from them that i was using too many connections and memory so i have to host the database myself.
Here's the stats of the server:
Dual Core Dual CPU Intel Xeon 3.0Ghz (2 core per cpu & 2 cpu's) 4GB RAM 120GB SATAII Hard Drive Windows 2003 Enterprise Edition SP2 (180 Day Trial)
Currently, im hosting around 10 game servers on the machine with around 30% CPU load. I'm running with the page file disabled (its using 2 GB of RAM instead). Server runs fine like this (i heard its a performance boost, and it is). Over 2GB of ram are free at all times even when most of the game servers are full.
I want the database server to be able to as many connections are need from the game servers but at the same time i dont want to affect performance of the game servers. Any suggestions on what settings i should use?
P.S. The SQL will have 3 databases (AMXBans, SourceBans, & a RP character db). Everytime a player connects to one of my servers, their information is checked against the database if they are an admin. So in total i have around 126 slots for players to connect on.
We just upgraded to VBulletin 3.6.5, and are experiencing strange behavior. My forums are on a dedicated server, 27,000 users.
We'll be running along fine with loads of .5-2.00, etc. Then suddenly, the loads start climbing to 50, 70, 85+. We've been trying to figure out why. We even went on a different (much more robust) server, still the same result. This only started happening after the upgrade. Restarting the server corrects the problem, but only until it happens again. It can happen at anytime - during peak or off peak. The server may run for an hour or two until this happens, or it can happen 10 minutes after a restart.
Late last night when loads were normal we benched the server hard, and it ran just fine. We just can't figure out where this load spike is coming from.
Server info below:
Servers: Mysql: server version: 5.0.27-log PHP: PHP 5.1.6 (cli) Zend Engine v2.1.0, Apache Server version: Apache/2.2.3 Kernel: 2.6.19-1.2911.fc6 #1 SMP
Server: Pentium III with 1 gig of ram 15k RPM SCSI Raid 5 1 Gig RAM
I'm running a Xeon Quad Core 2.5GHz with 2GB RAM. Comments on my setup for shared hosting? (will upgrade ram if people believe that will help with increase page loadtimes.)
Warning: mysql_connect() [function.mysql-connect]: Can't create a new thread (errno 11); if you are not out of available memory, you can consult the manual for a possible OS-dependent bug in /home/user/public_html/file.php on line 400 Can't create a new thread (errno 11); if you are not out of available memory, you can consult the manual for a possible OS-dependent bug
currently im on a VPS server and things are fine normally. However whenever there are like 20 users on my site (browser based text game) things get ugly. The sql queries are all needed and don't think it can get much more effiecent.
Anyways my website is very dynamic all information is changing every second.
For example a user when every page they go to or refresh their account is being set to time() to show that they are online at that second. Processor load gets up to like 6 at times and that is when it is virtually impossible to connect to the site.
Is there anythign you recommend? mysql is the top then when i login via ssh and run 'top'
I have a large proxy, are getting ~3k pageviews/day.
I only got 128 MB RAM on my VPS, so no panels works on it (not even kloxo/lxa). I would guess apache wont do it in the long run either.
So I decided to go with nginx. But I feel it isn't quite good optimized, as it sometimes takes like 1 minute to load the page.
Here is my current config:
####################################################################### # # This is the main Nginx configuration file. # # More information about the configuration options is available on # * the English wiki - http://wiki.codemongers.com/Main # * the Russian documentation - http://sysoev.ru/nginx/ # #######################################################################
#---------------------------------------------------------------------- # Main Module - directives that cover basic functionality # # http://wiki.codemongers.com/NginxMainModule # #----------------------------------------------------------------------
I am facing problem regarding my site. I have dedicated server with WHM/cPanel, i have integrated tomcat with apache , the apache processes are too high aroung 1200-1400, So please guide me howto optimize my apache for better performance.
root@trick [/]# ps aux | grep -c httpd 1403 root@trick [/]# ps aux | grep -c mysql 3 root@trick [/l]# free -m total used free shared buffers cached Mem: 4053 3408 645 0 220 1890 -/+ buffers/cache: 1297 2756 Swap: 4094 0 4094
---------------------------------------------
KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 15