Lag/Choke Between MySQL Server And Frontend

I run a rather busy forum. Recently we've finally moved onto a dual server setup. Our setup consists of this:

Files/Http:
C2D 6400
2GB RAM
2x 80GB HDD's in RAID 1
Lighttpd/Xcache/php 5.2.5

DB Server:
Xeon 5360 (2.4Ghz 4MB Cache)
2GB RAM (Upgrade to 4GB coming this week... if we can get these issues sorted anyway).
2x 36GB Raptor's in RAID.

MySQL 5.0.x

=====

Now, when theres less than 1,000 users online my board the server runs fine. The front end server has a load of <2.0 and the DB server goes no higher than 4-5. As soon as there's slightly more load placed on the server it's like they seem to choke. It appears that there's no MySQL data transferred between the servers for a couple of seconds before it goes back down. It's almost like a choking effect. The servers are connected directly to each other and have 100Mbps bandwidth between the two over internal IP's. At most there's probably 60 pages being served a second. On average perhaps 25/second.

Could it be a problem with my MySQL settings? Here's my.cnf

Code:
[mysqld]
safe-show-database
old_passwords
back_log = 75
max_connections = 650
key_buffer = 256M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 4000
thread_cache_size = 384
wait_timeout = 20
connect_timeout = 10
tmp_table_size = 64M
max_heap_table_size = 64M
max_allowed_packet = 64M
net_buffer_length = 16384
max_connect_errors = 10
thread_concurrency = 16
read_rnd_buffer_size = 786432
bulk_insert_buffer_size = 8M
query_cache_limit = 4M
query_cache_size = 64M
query_cache_type = 1
query_prealloc_size = 262144
query_alloc_block_size = 65536
transaction_alloc_block_size = 8192
transaction_prealloc_size = 4096
default-storage-engine = MyISAM

innodb_data_file_path = ibdata1:20M:autoextend
innodb_buffer_pool_size=256M
innodb_additional_mem_pool_size=20M

[mysqld_safe]
nice = -10
open_files_limit = 8192

[mysqldump]
quick
max_allowed_packet = 16M

[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M

[mysqlhotcopy]
interactive-timeout

I am running out of ideas and this is very frustrating. At the moment the site is more speedy on a single server setup than spread over the two...