Long Curl Downloads Delaying Other Connections

I've tried Lighttpd and Apache. Server is QuadCore 1 cpu.

Either one when I have one long curl download occurring in Firefox, and open up another window to connect to the site it waits until the curl process completes.

In Lighttpd I have php 5.2.5 fastcgi and xcache.

There is no errors. It simply just waits until the process is completed and 1 second later the second request (2nd browser) starts.

Code:
fastcgi.server = ( ".php" =>
( "localhost" =>
(
"socket" => "/tmp/php-fastcgi.socket",
"bin-path" => "/usr/local/php5/bin/php-cgi",
"max-procs" => 1,
"bin-environment" => (
"PHP_FCGI_CHILDREN" => "16",
"PHP_FCGI_MAX_REQUESTS" => "10000"
)

)
)
)