MySQL Optimization - Max_heap_table_size & Tmp_table_size
Jul 8, 2007
On one of my servers (Intel P4 2.8Ghz, 1GB RAM, cPanel), I've been tweaking some of the MySQL values in my.cnf and using the MySQL Tuning Primer shell script to evaluate the results.
This is my current my.cnf. I'm running MySQL 5.0.27.
Code:
[mysqld]
set-variable = max_connections=500
safe-show-database
skip-innodb
long_query_time=5
log-slow-queries=/var/log/mysql/log-slow-queries.log
query_cache_type=1
query_cache_size=16M
query_cache_limit=1M
table_cache=512
key_buffer=150M
thread_concurrency=2
thread_cache_size=80
sort_buffer_size=1M
read_buffer_size=1M
read_rnd_buffer_size=768K
The tuning primer script is telling me the following.
PHP Code:
Current max_heap_table_size = 16 MCurrent tmp_table_size = 32 MOf 3875 temp tables, 52% were created on diskEffective in-memory tmp_table_size is limited to max_heap_table_size.Perhaps you should increase your tmp_table_size and/or max_heap_table_sizeto reduce the number of disk-based temporary tables
What values should I increase (if at all) max_heap_table_size and tmp_table_size to?
View 3 Replies
ADVERTISEMENT
May 11, 2007
I used tuning_primer script to tune up my mysql server.
with tmp_table_size
Its already set to 256M and tuning_primer still shows that 77% of tmp tables were created on HDD. Is there a way to optimalize SQL requests to make it lower ? or what should I do?
View 5 Replies
View Related
Dec 11, 2007
just got a new server and got my site up and running on it perfectly but ive noticed its a bit slower than my old server and i think this is down to my incorrctly optimized mysql setup. ive used the my.cnf off my old server with some minor edits but im wondering what people can suggest i change for the following server spec
AMD Athlon 64 X2 4200+
2GB DDR2
120gb Sata hd
currently my my.cnf is as follows
Quote:
# This is for a system with little memory (32M - 64M) where MySQL plays
# an important part, or systems up to 128M where MySQL is used together with
# other programs (such as a web server)
#
# You can copy this file to
# /etc/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options (in this
# installation this directory is /var/lib/mysql) or
# ~/.my.cnf to set user-specific options.
#
# In this file, you can use all long options that a program supports.
# If you want to know which options a program supports, run the program
# with the "--help" option.
# The following options will be passed to all MySQL clients
[client]
#password = your_password
port = 3306
socket = /var/lib/mysql/mysql.sock
# Here follows entries for some specific programs
# The MySQL server
[mysqld]
old-passwords
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-locking
max_connections=1000
key_buffer = 1M
long_query_time = 5
query_cache_size = 64M
query_cache_limit = 64M
max_allowed_packet = 5M
table_cache = 2000
max_heap_table_size=32M
tmp_table_size=30M
thread_cache_size = 256
sort_buffer_size = 32M
net_buffer_length = 8K
read_buffer_size = 6M
read_rnd_buffer_size = 1M
myisam_sort_buffer_size = 32M
join_buffer_size = 3M
open_files_limit = 50000
#log-queries-not-using-indexes
low_priority_updates = 1
#loonsies
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates
[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
according to mysql tuning primer it reports the following
Quote:
-- MYSQL PERFORMANCE TUNING PRIMER --
- By: Matthew Montgomery -
MySQL Version 4.1.20-log x86_64
Uptime = 0 days 17 hrs 52 min 44 sec
Avg. qps = 15
Total Questions = 1028682
Threads Connected = 1
Warning: Server has not been running for at least 48hrs.
It may not be safe to use these recommendations
To find out more information on how each of these
runtime variables effects performance visit:
[url]
Visit [url]
for info about MySQL's Enterprise Monitoring and Advisory Service
SLOW QUERIES
Current long_query_time = 5 sec.
You have 0 out of 1028694 that take longer than 5 sec. to complete
The slow query log is NOT enabled.
Your long_query_time seems to be fine
WORKER THREADS
Current thread_cache_size = 256
Current threads_cached = 3
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine
MAX CONNECTIONS
Current max_connections = 1000
Current threads_connected = 1
Historic max_used_connections = 4
The number of used connections is 0% of the configured maximum.
You are using less than 10% of your configured max_connections.
Lowering max_connections could help to avoid an over-allocation of memory
See "MEMORY USAGE" section to make sure you are not over-allocating
MEMORY USAGE
Max Memory Ever Allocated : 243 M
Configured Max Per-thread Buffers : 41 G
Configured Max Global Buffers : 75 M
Configured Max Memory Limit : 41 G
Physical Memory : 1.90 G
Max memory limit exceeds 90% of physical memory
KEY BUFFER
Current MyISAM index space = 12 M
Current key_buffer_size = 1 M
Key cache miss rate is 1 : 14581
Key buffer fill ratio = 44.00 %
Your key_buffer_size seems to be too high.
Perhaps you can use these resources elsewhere
QUERY CACHE
Query cache is enabled
Current query_cache_size = 64 M
Current query_cache_used = 5 M
Current query_cache_limit = 64 M
Current Query cache Memory fill ratio = 8.42 %
Current query_cache_min_res_unit = 4 K
Query Cache is 15 % fragmented
Run "FLUSH QUERY CACHE" periodically to defragment the query cache memory
If you have many small queries lower 'query_cache_min_res_unit' to reduce fragmentation.
Your query_cache_size seems to be too high.
Perhaps you can use these resources elsewhere
MySQL won't cache query results that are larger than query_cache_limit in size
SORT OPERATIONS
Current sort_buffer_size = 32 M
Current record/read_rnd_buffer_size = 1020 K
Sort buffer seems to be fine
JOINS
Current join_buffer_size = 3.00 M
You have had 378 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.
Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.
OPEN FILES LIMIT
Current open_files_limit = 50000 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine
TABLE CACHE
Current table_cache value = 2000 tables
You have a total of 908 tables
You have 102 open tables.
The table_cache value seems to be fine
TEMP TABLES
Current max_heap_table_size = 31 M
Current tmp_table_size = 30 M
Of 3666 temp tables, 0% were created on disk
Created disk tmp tables ratio seems fine
TABLE SCANS
Current read_buffer_size = 5 M
Current table scan ratio = 1544 : 1
read_buffer_size seems to be fine
TABLE LOCKING
Current Lock Wait ratio = 1 : 4664
You may benefit from selective use of InnoDB.
View 11 Replies
View Related
Mar 13, 2007
PIV 2.8Ghz, 1GB Ram, MySQL 3.23, RHL E3.0
The servers load is vey bad, most of the time 7-8+ and the IO WAIT more than 60% which is decreasing server performace. Please help me to optimzie the MySQL specially. If you have the good my.cnf config please give me the complete config file.
[client]
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
safe-show-database
innodb_data_file_path=ibdata1:10M:autoextend
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
set-variable = connect_timeout=10
set-variable = interactive_timeout=100
set-variable = join_buffer_size=1M
set-variable = key_buffer=128M
set-variable = max_allowed_packet=16M
set-variable = max_connections=250
set-variable = max_connect_errors=10
set-variable = myisam_sort_buffer_size=16M
set-variable = table_cache=256
set-variable = thread_cache_size=32
set-variable = thread_concurrency=2
set-variable = wait_timeout=15
[mysqldump]
quick
set-variable = max_allowed_packet=16M
[mysql.server]
user=mysql
basedir=/var/lib
[safe_mysqld]
err-log=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log-slow-queries = /var/log/mysql-slow.log
long_query_time = 5
View 7 Replies
View Related
Apr 9, 2009
someone to look at my DB and optimize it as it is running ridiculously high loads of 25. It is vbulletin. I'm only looking for professional companies for this job.
View 1 Replies
View Related
Feb 16, 2008
I have a friend which owns a 2k alexa rank forum using VB
The server hangs and he gets mysql connection error in logs
Quote:
Database error in vBulletin : mysql_connect() [<a href='function.mysql-connect'>function.mysql-connect</a>]: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock'
The server is a dual Pentium D with 4GB RAM
currently 600 users online and 20% RAM used
in rush hours there are 2000 users online and 50% RAM used
We want to optimize mysql - this is the current my.cnf
Quote:
[mysqld]
safe-show-database
old_passwords
back_log = 75
skip-innodb
max_connections = 2000
key_buffer = 80M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 3M
table_cache = 2500
thread_cache_size = 384
wait_timeout = 15
connect_timeout = 10
tmp_table_size = 64M
max_heap_table_size = 64M
max_allowed_packet = 64M
net_buffer_length = 16384
max_connect_errors = 10
thread_concurrency = 8
read_rnd_buffer_size = 786432
bulk_insert_buffer_size = 8M
query_cache_limit = 4M
query_cache_size = 64M
query_cache_type = 1
query_prealloc_size = 262144
query_alloc_block_size = 65536
transaction_alloc_block_size = 8192
transaction_prealloc_size = 4096
default-storage-engine = MyISAM
[mysql.server]
user=mysql
basedir=/var/lib
old-passwords = 1
[mysqld_safe]
err-log=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
nice = -5
open_files_limit = 8192
[mysqldump]
quick
max_allowed_packet = 16M
[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 64M
write_buffer = 64M
[mysqlhotcopy]
interactive-timeout
View 7 Replies
View Related
Jul 2, 2008
My server is running a AMD Athlon(tm) XP 2000+ (1662 MHz)CPU
I only host one site, a vb forum on my server and its killing the CPU with only 80-100 people online.
The load averages are:
23.73 (1 mins) , 20.39 (5 mins) , 10.88 (15 mins)
and these are the processes that are making it slow:
Code:
ID Owner CPU Command
8003 mysql 34.1 % /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-f ...
7777 apache 2.1 % /usr/sbin/httpd
7911 apache 2.1 % /usr/sbin/httpd
8382 apache 2.1 % /usr/sbin/httpd
7857 apache 1.9 % /usr/sbin/httpd
7776 apache 1.8 % /usr/sbin/httpd
8384 apache 1.7 % /usr/sbin/httpd
7849 apache 1.6 % /usr/sbin/httpd
9159 root 1.6 % /usr/libexec/webmin/proc/index_cpu.cgi
7852 apache 1.5 % /usr/sbin/httpd
8790 apache 1.4 % /usr/sbin/httpd
8724 apache 1.2 % /usr/sbin/httpd
8775 apache 1.2 % /usr/sbin/httpd
8779 apache 1.1 % /usr/sbin/httpd
8841 apache 1.1 % /usr/sbin/httpd
155 root 1.0 % [kswapd0]
8752 apache 1.0 % /usr/sbin/httpd
8881 root 1.0 % /usr/bin/perl /usr/libexec/webmin/virtual-server/collectinfo.pl
8961 apache 1.0 % /usr/sbin/httpd
8759 apache 0.9 % /usr/sbin/httpd
8777 apache 0.9 % /usr/sbin/httpd
8956 apache 0.9 % /usr/sbin/httpd
8962 apache 0.9 % /usr/sbin/httpd ..
View 6 Replies
View Related
Jan 27, 2007
I just switched to a new server - I went from a Dual Opteron 248 with 3GB of RAM to a Dual Clovertown with 4GB of RAM. Yet, despite having mild traffic right now, my load is rather high and my memory usage is like 80% (which is ridiculous, since that would mean I would be maxing out on the old box).
For some reason, my servers usually lose access when I get around 400 people on TEXT-BASED, non-MySQL pages? How is that possible? I have really solid servers, yet people running basic Pentiums with 1 or 2GB never run into this issue.
Is there any good, cheap site that would be able to properly optimize mySQL and Apache to lower the load and figure out why the memory usage is so high? Platinum Server Management never really got this done, so even though they're great for other support issues, please don't recommend them.
View 0 Replies
View Related
Oct 30, 2007
i run a fedora server with plesk 8 on it i was wondering if some one can help me to find the best way to optimize the databases on the server at once my servers contains many clients with unicode data in the database.
View 0 Replies
View Related
Mar 20, 2007
I currently have a server for proxy websites only.
I have disabled MySQL, is that ok?
Will my statistics software still work (awstats)?
What else can i disable or do to optimize the server for proxy websites.
I'm using the phproxy scripts.
View 0 Replies
View Related
Feb 16, 2007
Code:
mysql> SHOW STATUS;
+----------------------------+------------+
| Variable_name | Value |
+----------------------------+------------+
| Aborted_clients | 510 |
| Aborted_connects | 0 |
| Binlog_cache_disk_use | 0 |
| Binlog_cache_use | 0 |
| Bytes_received | 441269798 |
| Bytes_sent | 2031527307 |
| Com_admin_commands | 27428 |
| Com_alter_db | 0 |
| Com_alter_table | 4 |
| Com_analyze | 0 |
| Com_backup_table | 0 |
| Com_begin | 0 |
| Com_change_db | 27481 |
| Com_change_master | 0 |
| Com_check | 0 |
| Com_checksum | 0 |
| Com_commit | 0 |
| Com_create_db | 0 |
| Com_create_function | 0 |
| Com_create_index | 0 |
| Com_create_table | 0 |
| Com_dealloc_sql | 0 |
| Com_delete | 16883 |
| Com_delete_multi | 0 |
| Com_do | 0 |
| Com_drop_db | 0 |
| Com_drop_function | 0 |
| Com_drop_index | 0 |
| Com_drop_table | 0 |
| Com_drop_user | 0 |
| Com_execute_sql | 0 |
| Com_flush | 0 |
| Com_grant | 0 |
| Com_ha_close | 0 |
| Com_ha_open | 0 |
| Com_ha_read | 0 |
| Com_help | 0 |
| Com_insert | 5515 |
| Com_insert_select | 379 |
| Com_kill | 0 |
| Com_load | 0 |
| Com_load_master_data | 0 |
| Com_load_master_table | 0 |
| Com_lock_tables | 0 |
| Com_optimize | 0 |
| Com_preload_keys | 0 |
| Com_prepare_sql | 0 |
| Com_purge | 0 |
| Com_purge_before_date | 0 |
| Com_rename_table | 0 |
| Com_repair | 1 |
| Com_replace | 0 |
| Com_replace_select | 0 |
| Com_reset | 0 |
| Com_restore_table | 0 |
| Com_revoke | 0 |
| Com_revoke_all | 0 |
| Com_rollback | 0 |
| Com_savepoint | 0 |
| Com_select | 154467 |
| Com_set_option | 132 |
| Com_show_binlog_events | 0 |
| Com_show_binlogs | 7 |
| Com_show_charsets | 33 |
| Com_show_collations | 33 |
| Com_show_column_types | 0 |
| Com_show_create_db | 0 |
| Com_show_create_table | 0 |
| Com_show_databases | 6 |
| Com_show_errors | 0 |
| Com_show_fields | 0 |
| Com_show_grants | 17 |
| Com_show_innodb_status | 0 |
| Com_show_keys | 0 |
| Com_show_logs | 0 |
| Com_show_master_status | 0 |
| Com_show_ndb_status | 0 |
| Com_show_new_master | 0 |
| Com_show_open_tables | 0 |
| Com_show_privileges | 0 |
| Com_show_processlist | 0 |
| Com_show_slave_hosts | 0 |
| Com_show_slave_status | 0 |
| Com_show_status | 2 |
| Com_show_storage_engines | 0 |
| Com_show_tables | 38 |
| Com_show_variables | 79 |
| Com_show_warnings | 0 |
| Com_slave_start | 0 |
| Com_slave_stop | 0 |
| Com_stmt_close | 0 |
| Com_stmt_execute | 0 |
| Com_stmt_prepare | 0 |
| Com_stmt_reset | 0 |
| Com_stmt_send_long_data | 0 |
| Com_truncate | 0 |
| Com_unlock_tables | 0 |
| Com_update | 38964 |
| Com_update_multi | 0 |
| Connections | 306 |
| Created_tmp_disk_tables | 6302 |
| Created_tmp_files | 3 |
| Created_tmp_tables | 26220 |
| Delayed_errors | 0 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 0 |
| Flush_commands | 1 |
| Handler_commit | 0 |
| Handler_delete | 7984 |
| Handler_discover | 0 |
| Handler_read_first | 10826 |
| Handler_read_key | 80354286 |
| Handler_read_next | 921590052 |
| Handler_read_prev | 0 |
| Handler_read_rnd | 2852179 |
| Handler_read_rnd_next | 358019891 |
| Handler_rollback | 0 |
| Handler_update | 34804652 |
| Handler_write | 3299547 |
| Key_blocks_not_flushed | 0 |
| Key_blocks_unused | 350900 |
| Key_blocks_used | 136652 |
| Key_read_requests | 177777142 |
| Key_reads | 234783 |
| Key_write_requests | 192997 |
| Key_writes | 70499 |
| Max_used_connections | 18 |
| Not_flushed_delayed_rows | 0 |
| Open_files | 126 |
| Open_streams | 0 |
| Open_tables | 89 |
| Opened_tables | 124 |
| Qcache_free_blocks | 1493 |
| Qcache_free_memory | 101831304 |
| Qcache_hits | 247556 |
| Qcache_inserts | 154232 |
| Qcache_lowmem_prunes | 0 |
| Qcache_not_cached | 170 |
| Qcache_queries_in_cache | 2670 |
| Qcache_total_blocks | 6866 |
| Questions | 491633 |
| Rpl_status | NULL |
| Select_full_join | 0 |
| Select_full_range_join | 0 |
| Select_range | 15228 |
| Select_range_check | 0 |
| Select_scan | 42652 |
| Slave_open_temp_tables | 0 |
| Slave_retried_transactions | 0 |
| Slave_running | OFF |
| Slow_launch_threads | 0 |
| Slow_queries | 29 |
| Sort_merge_passes | 0 |
| Sort_range | 26186 |
| Sort_rows | 92603755 |
| Sort_scan | 28212 |
| Table_locks_immediate | 427295 |
| Table_locks_waited | 239 |
| Threads_cached | 1 |
| Threads_connected | 17 |
| Threads_created | 18 |
| Threads_running | 2 |
| Uptime | 9400 |
+----------------------------+------------+
163 rows in set (0.00 sec)
this is what SHOW STATUS; commands give me can anyone provide me any helpful tips on what I can adjust in my.cnf to lower my server load which is above 10 during peak hours and mysql is the only thing eating this serve up I have lots og 3gb ram and a 64bit AMD 2800 server
View 6 Replies
View Related
Jul 27, 2009
I'm trying to optimize a server (Intel E8300) with 8Gb of RAM and WHM/cPanel administration panel.
Now the server has only 46 accounts in WHM, and about 10.000 unique visitors / day
As mysql is always on top for CPU ussage I have edited 2 values in my.cnf, but this was done in 10 minutes and not to much research, any more help will be appreciated. On this server as I see it's been using MyISAM for storage engine and maybe just a few tabels with InnoDB ....
View 13 Replies
View Related
Apr 21, 2008
So the site got featured on [url]and now the server is drowning...
The Coppermine Gallery usually hovers around 30~50 users daily and now, 1800, and im at a lost as how I should configure mysql to take on such a load. right now it takes about 10 secs or more to load a page and sometimes it would time out. Because it si coppermine, all pages are dynamic and can't be cached -_-"
Here's the my.cnf right now after i played around with the numbers
server spec
Opteron 170 (2ghz)
2gb ram
250 7200rpm
[mysqld]
set-variable=local-infile=0
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
#skip-networking
back_log = 75
key_buffer = 256M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 1800
thread_cache_size = 384
wait_timeout = 7200
max_connections = 600
connect_timeout = 100
wait_timeout = 12000
tmp_table_size = 100M
max_heap_table_size = 64M
max_allowed_packet = 15M
max_connect_errors = 9999999
read_rnd_buffer_size = 524288
bulk_insert_buffer_size = 8M
query_cache_limit = 40M
query_cache_size = 150M
query_cache_type = 1
query_prealloc_size = 65536
tmp_table_size = 64M
query_alloc_block_size = 131072
default-storage-engine = MyISAM
[mysql.server]
user=mysql
basedir=/var/lib
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
nice = -5
open_files_limit = 8192
[mysqldump]
quick
max_allowed_packet = 16M
[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M
httpd.conf
Timeout 20
#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive Off
#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 100
#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 15
##
## Server-Pool Size Regulation (MPM specific)
##
# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# ServerLimit: maximum value for MaxClients for the lifetime of the server
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 200
MaxClients 200
MaxRequestsPerChild 1500
</IfModule>
# worker MPM
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule worker.c>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
View 11 Replies
View Related
Jun 24, 2008
I have a fairly high end server in which I have installed SIM. SIM is restarting Apache up to 10 times a day, presumably due to high load causing un-availability.
On restart, Apache / MYSQL is stable until the load / mem usage begins to climb then it is restarted again. Here are my 'load' stats for today:
Load for today
High (2:18am): 4.63
Low (3:30am): 1.20
Mean: 1.84
Latest: 1.61
Mem usage for today
High (1:36am): 9,192.9 MB
Low (1:48am): 7,995.7 MB
Mean: 8,683.1 MB
Latest: 8,781.7 MB
I have seen it using 20GB RAM before.
I have tried to follow various optimisation guides but these seem tailored to less powerful servers.
The web application I run on this server is almost entirely MYSQL based, with thousands of DB calls a day. Across the entire system I probably get 200,000 bot hits per day or even more. At peak times search engine bots are literally hammering the server.
My server specs are:
# Processor: Quad-Core Xeon Processor * 8
# CPU Speed: 3Ghz
# RAM: 32 GB DDR2
# HDD: 36 GB 15k Raptor X2 + 1TB Sata
# OS: 32Bit CentOS
Changes / optimisation of the actual application is not possible.
My configuration files are:
My.cnf ....
View 11 Replies
View Related
Jun 18, 2009
Can anyone recommend a Apached and MYSQL optimization expert? I just some optimization done on mysql and apache upgraded... but pache wouldn't restart with the following entries:
MinSpareThreads 25
MaxSpareThreads 35
ThreadsPerChild 25
if i remove above then it starts... but my CPU spikes and I am getting a ton of errors emailed to me:
Invalid SQL:
SELECT COUNT(DISTINCT(userid)) AS count FROM vb_session WHERE vb_session.userid>0 AND vb_session.lastactivity>1245294346;
MySQL Error : Out of memory (Needed 8388580 bytes)
Error Number : 5
Request Date : Wednesday, June 17th 2009 @ 09:39:06 PM
Error Date : Wednesday, June 17th 2009 @ 09:39:06 PM
Its constantly complaining about this same query and i thought i removed all of the vbulletin options...
i am pulling my hair out... cuz my server isn't swapping, ther are 0ver 8GBs in cache.... but my CPU goes up
Here is original problem thread...
[url]
2nd post has recommended config... and my subsequent probs
View 6 Replies
View Related
Apr 11, 2009
I'm running a Xeon Quad Core 2.5GHz with 2GB RAM. Comments on my setup for shared hosting? (will upgrade ram if people believe that will help with increase page loadtimes.)
XCACHE:
xcache.shm_scheme = "mmap"
xcache.size = 64M
xcache.count = 5
xcache.slots = 16K
xcache.ttl = 3600
xcache.gc_interval = 7200
xcache.var_size = 64M
xcache.var_count = 5
xcache.var_slots = 16K
xcache.var_ttl = 3600
xcache.var_maxttl = 7200
xcache.var_gc_interval = 300
xcache.test = Off
xcache.readonly_protection = Off
xcache.mmap_path = "/dev/zero"
xcache.coredump_directory = ""
xcache.cacher = On
xcache.stat = On
xcache.optimizer = On
MySQL:
[mysqld]
set-variable = max_connections=500
safe-show-database
old-passwords = 1
skip-networking
skip-name-resolve
skip-external-locking
socket = /var/lib/mysql/mysql.sock
skip-locking
key_buffer = 384M
max_allowed_packet = 1M
table_cache = 5120
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
read_rnd_buffer_size=8M
myisam_sort_buffer_size=64M
thread_cache_size=8
thread_cache=256
query_cache_limit=12M
query_cache_size=512M
thread_concurrency = 8
wait_timeout=120
connect_timeout=30
long_query_time=5
[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
View 3 Replies
View Related
Feb 27, 2008
MySQL keep giving me error like this:
Warning: mysql_connect() [function.mysql-connect]: Can't create a new thread (errno 11); if you are not out of available memory, you can consult the manual for a possible OS-dependent bug in /home/user/public_html/file.php on line 400
Can't create a new thread (errno 11); if you are not out of available memory, you can consult the manual for a possible OS-dependent bug
My server has 5GB SWAP, 2GB RAM
[my.cnf]
key buffer = 512m
read buffer = 4m
sort buffer = 4m
max connection = 500
i learn using a formula key buffer + (read+sort)*max connection
View 4 Replies
View Related
Feb 12, 2008
I hosted users files on a xfs partition.
I use SATA II disks, and sometimes load is increased when ftp activity is intensive.
Drive where users files are hosted returns values like
hdparm -Tt /dev/sdb
/dev/sdb:
Timing cached reads: 5050 MB in 2.00 seconds = 2528.72 MB/sec
Timing buffered disk reads: 4 MB in 3.30 seconds = 1.21 MB/sec
I noticed a load increase specially when I delete a directory with many small files.
ways to optimyze a xfs partition?
View 4 Replies
View Related
Jun 14, 2007
currently im on a VPS server and things are fine normally. However whenever there are like 20 users on my site (browser based text game) things get ugly. The sql queries are all needed and don't think it can get much more effiecent.
Anyways my website is very dynamic all information is changing every second.
For example a user when every page they go to or refresh their account is being set to time() to show that they are online at that second. Processor load gets up to like 6 at times and that is when it is virtually impossible to connect to the site.
Is there anythign you recommend? mysql is the top then when i login via ssh and run 'top'
View 8 Replies
View Related
May 23, 2009
Well, I decided giving it a shot.
I have a large proxy, are getting ~3k pageviews/day.
I only got 128 MB RAM on my VPS, so no panels works on it (not even kloxo/lxa). I would guess apache wont do it in the long run either.
So I decided to go with nginx. But I feel it isn't quite good optimized, as it sometimes takes like 1 minute to load the page.
Here is my current config:
#######################################################################
#
# This is the main Nginx configuration file.
#
# More information about the configuration options is available on
# * the English wiki - http://wiki.codemongers.com/Main
# * the Russian documentation - http://sysoev.ru/nginx/
#
#######################################################################
#----------------------------------------------------------------------
# Main Module - directives that cover basic functionality
#
# http://wiki.codemongers.com/NginxMainModule
#
#----------------------------------------------------------------------
user nginx;
worker_processes 12;
error_log /var/log/nginx/error.log;
#error_log /var/log/nginx/error.log notice;
#error_log /var/log/nginx/error.log info;
pid /var/run/nginx.pid;
#----------------------------------------------------------------------
# Events Module
#
# http://wiki.codemongers.com/NginxEventsModule
#
#----------------------------------------------------------------------
events {
worker_connections 1024;
}
#----------------------------------------------------------------------
# HTTP Core Module
#
# http://wiki.codemongers.com/NginxHttpCoreModule
#
#----------------------------------------------------------------------
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] $request '
'"$status" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
# Load config files from the /etc/nginx/conf.d directory
include /etc/nginx/conf.d/*.conf;
#
# The default server
#
server {
listen 80;
server_name _;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root /usr/share/nginx/html;
index index.php index.html index.htm;
}
error_page 404 /404.html;
location = /404.html {
root /usr/share/nginx/html;
}
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ .php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
location ~ .php$ {
root html;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /usr/share/nginx/html$fastcgi_script_name;
include fastcgi_params;
}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /.ht {
# deny all;
#}
}
}
Any ideas on better optimizing?
View 9 Replies
View Related
Oct 13, 2009
I am facing problem regarding my site. I have dedicated server with WHM/cPanel, i have integrated tomcat with apache , the apache processes are too high aroung 1200-1400, So please guide me howto optimize my apache for better performance.
root@trick [/]# ps aux | grep -c httpd
1403
root@trick [/]# ps aux | grep -c mysql
3
root@trick [/l]# free -m
total used free shared buffers cached
Mem: 4053 3408 645 0 220 1890
-/+ buffers/cache: 1297 2756
Swap: 4094 0 4094
---------------------------------------------
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
<IfModule prefork.c>
StartServers 10
MinSpareServers 10
MaxSpareServers 20
ServerLimit 1000
MaxClients 600
MaxRequestsPerChild 0
</IfModule>
View 7 Replies
View Related
May 25, 2009
I'm looking to get some tips to optimize my Apache webservers. Now i have 2 apache servers with load balancer. If someone can guide me and bring some tips for this purpose is highly appreciated.
Also, i would like to know about disadvantages of enabling KeepAlive. Recently i have enabled it and found my site load much better. But the disadvantage (that i experienced personally) is after stop browsing for a while (about 40-60 minutes) i need to keep triggering the URL of site several times to get enter the website. If not i can't get into it and it likes the site giving no response at all.
View 4 Replies
View Related
Apr 22, 2009
right ive got a server thats getting battered at the moment by lots of apache processes, ive moved sql off to a seperate box (with private lan so thats fine) but now i need to optimize/replace apache with something less intensive on resources, only problem is im using plesk on this server so i am not sure if i can change the webserver app?
using apache 2.2 with prefork mpm
server specs
Athlon 64 3200+
4gb DDR2
120Gb HD
View 13 Replies
View Related
Jul 25, 2009
I am switching from shared hosting to managed VPS. I have minimal knowledge of UNIX/Apache, pretty good with PHP and programming in general.
So, what I have now is my VPS, with 512mb of ram, CentOS, cpanel and my two sites, not live yet, that is, I haven't changed dns records in the registrar.
I want to try to optimize the box or whatever is the correct name for it and understand the whole thing better. Mainly to learn more, I dot' think I'll be pressed for memory or bandwidth for now. Still, at least knowing things I can optimize for speed of space.
Few things that I assume should allow for some simplification and RAM savings(hopefully) is that:
a) Both sites are hobby sites, not very big load. Few thousand visitors a day. Traffic is around 10gb monthly combined.
b) Both sites are mine, nobody else on that box, so I don't have to worry about customers screwing things up and caging things internally.
Based on that, I don't really need email for the second site domain.
When I run top command I see 2 exim processes running. Is that normal, or is it because I have 2 domains registered there? And if yes can one be disabled?
Same applies to pop3-login and imap-login processes, 2 copies of each. Can I have just one of each, i.e. exim, pop3-login and imap-login?
Next, I selected not to run log analyzer cpanellogd for 21 hours, i.e. it'll run for 3 hours a day. I assume that should be sufficient for the site like mine?
I did read sticky about securing the site and quite a bit about Apache/mysql optimizations.
View 11 Replies
View Related
Jun 15, 2008
I've got a VPS server with a total process limit of 100 process.
I know this sounds like a newbie question, what numbers would you recommend for the Apache 2 Conf file to get it running as best as possible without it taking the server down?
View 4 Replies
View Related
Apr 18, 2007
i am having quite problem with my server i cant get it to optimize it. During peak times the load goes beyond 10 sometimes 20 to 50 sometimes its just under 1. I have my webserver running on P4 Core 2 6300, 1 GB ram, 250 HDD, on CentOS 4.4 (kernel 2.6.9-42.0.10) etc. It has cpanel/whm running Apache 1.3.7, php 4.4, mysql 4.1, mailscanner, eaccelerator, etc i dont know really what other info to show you.
I host about 200 accounts and there are 4 large php forums (not like WHT) running on it. Sometimes pages stop loading especially these forums have to refresh to get page load.
During day there are approx 700-900 connections.
Previously i had phpsuexec and the load never went beyond 4-5, i have removed it to install eaccelerator.
Here are my configurations
httpd.conf
Code:
RLimitMEM 200990378
RLimitCPU 240
Timeout 45
KeepAlive On
MaxKeepAliveRequests 512
KeepAliveTimeout 2
MinSpareServers 25
MaxSpareServers 35
StartServers 15
MaxClients 512
MaxRequestsPerChild 0
my.cnf
Code:
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-locking
skip-innodb
skip-networking
skip-name-resolve
query_cache_limit = 1M
query_cache_size = 64M
query_cache_type = 1
max_connections = 200
max_user_connections = 100
interactive_timeout = 30
wait_timeout = 30
connect_timeout = 10
thread_cache_size = 80
key_buffer = 256M
join_buffer = 1M
max_allowed_packet = 8M
table_cache = 1500
record_buffer = 1M
sort_buffer_size = 1M
read_buffer_size = 1M
max_connect_errors = 100
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 4
myisam_sort_buffer_size = 32M
#log-bin
server-id = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 1M
write_buffer = 1M
[myisamchk]
key_buffer =128M
sort_buffer_size = 128M
read_buffer = 1M
write_buffer = 1M
[mysqlhotcopy]
interactive-timeout
top
Code:
top - 13:24:33 up 1 day, 23:24, 1 user, load average: 1.99, 2.26, 3.45
Tasks: 174 total, 1 running, 173 sleeping, 0 stopped, 0 zombie
Cpu(s): 13.3% us, 12.1% sy, 0.0% ni, 13.6% id, 31.8% wa, 29.1% hi, 0.0% si
Mem: 1000952k total, 974492k used, 26460k free, 49024k buffers
Swap: 2031608k total, 171580k used, 1860028k free, 288344k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10804 mysql 15 0 354m 80m 2744 S 35 8.2 4:00.68 mysqld
12849 nobody 15 0 65816 33m 14m S 10 3.4 0:04.56 httpd
2956 named 18 0 66944 18m 1152 S 3 1.9 40:53.96 named
12800 nobody 15 0 57260 28m 17m S 3 2.9 0:06.95 httpd
336 root 15 0 0 0 0 D 3 0.0 112:24.57 kjournald
12854 nobody 15 0 55008 22m 14m S 2 2.3 0:03.68 httpd
12870 nobody 16 0 55108 23m 14m S 1 2.4 0:04.49 httpd
6373 root 0 -20 0 0 0 S 1 0.0 1:13.66 loop0
32021 mailnull 17 0 42500 13m 2480 S 1 1.4 0:09.05 MailScanner
12720 nobody 16 0 56088 22m 12m S 1 2.3 0:04.67 httpd
12877 nobody 15 0 55068 23m 15m S 1 2.4 0:04.05 httpd
12725 nobody 15 0 54836 19m 11m S 1 2.0 0:04.01 httpd
12831 nobody 15 0 55468 25m 16m S 1 2.6 0:03.80 httpd
12859 nobody 15 0 54932 22m 14m S 1 2.3 0:03.75 httpd
7 root 5 -10 0 0 0 S 0 0.0 0:28.61 events/1
30091 root 15 0 2828 520 452 D 0 0.1 0:45.08 syslogd
31213 mailnull 16 0 44896 14m 2472 S 0 1.5 0:13.50 MailScanner
12716 nobody 15 0 56016 24m 14m S 0 2.5 0:04.20 httpd
12774 nobody 15 0 56272 25m 16m S 0 2.6 0:04.76 httpd
12843 nobody 15 0 55156 23m 15m S 0 2.4 0:04.38 httpd
12851 nobody 15 0 56184 21m 11m S 0 2.2 0:03.58 httpd
12853 nobody 15 0 54888 22m 14m S 0 2.3 0:03.39 httpd
12864 nobody 16 0 54920 23m 14m S 0 2.4 0:04.43 httpd
13555 nobody 15 0 54492 21m 13m S 0 2.2 0:02.39 httpd
15385 nobody 16 0 52564 8388 2052 S 0 0.8 0:00.08 httpd
1 root 16 0 2260 396 368 S 0 0.0 0:06.43 init
View 6 Replies
View Related
Jun 9, 2007
My troubles started about a month ago, on a Windows 2003 VPS (clean, no plesk/cpanel):
Every ting worked fine until, for no obvious reason, IIS started to consume too much memory, I tried separating sites (two) in separate application pools, adjusting recycle triggers - nothing worked, sooner or later worker processes would use too much memory and fail, only way to fix the problem was to restart it.
At this point I would like to mention that:
- MySQL is optimized for low memory usage (and task manager never said otherwise)
- PHP is limited to 20MB RAM, only few modules were enabled
- the only other application running is WinDNS, and it doesn't seam RAM hungry
So a few days ago I decided to kill IIS/app server role altogether and install apache, something I should have done long time ago.
Everything went smooth, VPS performance dramatically increased, memory usage was negotiable... at first.
As time goes by, apache consumed more memory (httpd.exe comes to about 80MB in 12 hours), it doesn't seam to be dependant on site traffic, and it doesn't drop, server didn't fail once yet and I have apache on scheduled graceful restarts every 24hrs.
I didn't try setting a MaxRequestsPerChild value, I'll keep that as a last option because I really want to get to the bottom of this
So... I suspect it could be PHP, the only thing that's common in both scenarios. I did upgrade to the latest version during Apache installation though.
VPS:
256MB RAM
Win 2003
Apache 2.2
PHP 5.2.3
MySQL 5.0
What could be causing this "leak"?
View 3 Replies
View Related
Dec 26, 2008
I just got new server:
2.0 GHz, Xeon 5405 (Harpertown) (Quad Core) with CentOS 5 installed. It has 4 gigs of RAM.
I would like to optimize it for php/mysql based forum.
Any suggestions on how to tweak my.cnf or httpd.conf or anything else. My main goal is to optimize it for php/mysql.
I understand I can leave everything the way it is, but I would rather optimize my server, if I can.
Like should I install some kind of memory cache and if yes, which one? Should I edit default settings of PHP?
View 5 Replies
View Related
Jun 24, 2007
if using the Internap FCP technology to optimize the routing. I want a feedback on it, since I want to deploy this solution for have a better traffict routing.
Also, is anyone using avaya? i have looked in their website, but I have no information about their routing optimizer. Basically I want to go beyond of normal BGP since I will be deploying VoIP services soon.
View 14 Replies
View Related
Feb 20, 2007
I been logging queries for a whole 3 days and my logs is huge and the problem seems to be my phpBB search table thats causing such problems for me heres what I logged
Code:
# Time: 070219 19:41:53
# User@Host: admin_db[admin_db] @ localhost []
# Query_time: 11 Lock_time: 0 Rows_sent: 0 Rows_examined: 13274745
SELECT word_id
FROM phpbb_search_wordmatch
GROUP BY word_id
HAVING COUNT(word_id) > 259756;
Everything from phpbb_search_wordmatch takes more then 11 sec Query time.
How can I optimze this and lower the Query time to something like 1 or 2.
Some adivce would be helpful me not that good with mysql.
View 1 Replies
View Related
Jul 11, 2007
anyone have any tips on how to optimize WHM / cpanel to run at top performance on 256-512mb guaranteed VPS accounts?
I've had several VPS providers in the past, and each one of them run webpages and LAMP stuff fine, but as soon as I bring up cpanel /whm, it either crashes the VPS altogether, or cpanel and whm run very slow, choppy, pretty much bogging the whole VPS down, doesn't seem to matter if the VPS is running on 256 or 512, still same issues. I run on Virtuozzo, which I don't know if thats part of the problem, or not,
really all I use out of cpanel/ WHM is the standard things like
creating/deleting accounts
RVSkin, Fantastico
basically what a reseller would use. the rest of the stuff I do not use, so I was just wondering if there was a tutorial on how to strip down WHM / cpanel somehow so it would run better on VPS accounts? or if someone could write one up perhaps?
View 14 Replies
View Related
