MySQL Performance

I have a site that runs from a MySQL database. The database isn't big, it has only 16.1 MB on 17 tables with 103,978 records, and all are properly indexed.

My problem is that my MySQL has a particular problem with this database because I'll have to wait around 5-10 minutes to receive the query results.

I've tuned MySQL, I've tuned Apache and in the daily usage i have usually low Load Averages of 0.19 0.22 0.40. My server specs are Intel P4 at 3GHz with HT, 2Gb of RAM (in dual Chanel), 2 Hdd of 250Gb (backup) and 300Gb (main hdd), both with 16 mb cache. I'm running Debian 3.1 with 2.6 kernel and there is no swapping to disk as i had previous on the 2.4 kernel.

I will post my.conf below and maybe you'll give me a suggestion.

Code:
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "/var/lib/mysql/my.cnf" to set server-specific options or
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# [url]

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port= 3306
socket= /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket= /var/run/mysqld/mysqld.sock
nice= 0

[mysqld]
#
# * Basic Settings
#
user= mysql
pid-file= /var/run/mysqld/mysqld.pid
socket= /var/run/mysqld/mysqld.sock
port= 3306
basedir= /usr
datadir= /var/lib/mysql
tmpdir= /tmp
language= /usr/share/mysql/english
skip-external-locking
#
# For compatibility to other Debian packages that still use
# libmysqlclient10 and libmysqlclient12.
old_passwords= 1
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address= 127.0.0.1
#
# * Fine Tuning
#
key_buffer= 32M
key_buffer_size= 32M
max_allowed_packet= 16M
myisam_sort_buffer_size = 32M
table_cache= 3072
sort_buffer_size= 4M
read_buffer_size= 4M
read_rnd_buffer_size= 4M
join_buffer_size= 2M
thread_stack= 512K
wait_timeout= 300
max_connection = 60
max_connect_errors= 10
thread_cache_size= 100
long_query_time= 2
max_user_connections= 50
interactive_timeout= 100
connect_timeout= 15
tmp_table_size= 64M
open_files_limit= 3072
max_heap_table_size = 64M
#
# * Query Cache Configuration
#
query_cache_limit= 2M
query_cache_size = 64M
query_cache_type = 1
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
#log= /var/log/mysql.log
#log= /var/log/mysql/mysql.log
#
# Error logging goes to syslog. This is a Debian improvement :)
#
# Here you can see queries with especially long duration
log-slow-queries= /var/log/mysql/mysql-slow.log
log-queries-not-using-indexes = /var/log/mysql/mysql-index.log
# The following can be used as easy to replay backup logs or for replication.
#server-id= 1
log-bin= /var/log/mysql/mysql-bin.log
# See /etc/mysql/debian-log-rotate.conf for the number of files kept.
max_binlog_size = 104857600
#binlog-do-db= include_database_name
#binlog-ignore-db= include_database_name
#
# * BerkeleyDB
#
# According to an MySQL employee the use of BerkeleyDB is now discouraged
# and support for it will probably cease in the next versions.
skip-bdb
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# If you want to enable SSL support (recommended) read the manual or my
# HOWTO in /usr/share/doc/mysql-server/SSL-MINI-HOWTO.txt.gz
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem

[mysqldump]
quick
quote-names
max_allowed_packet= 64M

[mysql]
#no-auto-rehash# faster start of mysql but no tab completition

[isamchk]
key_buffer= 32M
sort_buffer_size= 32M
read_buffer= 16M
write_buffer= 16M

[myisamchk]
key_buffer = 32M
sort_buffer_size = 32M
read_buffer = 16M
write_buffer = 16M

I thought that i might need to start rewriting my code in order to be able to fix things but i need at least another opinion from someone who knows more.