Speed Up Serving Large Files
I'm working on a web site which will basically be a flash games portal. I have a dedicated server running Apache 2 on a 100mbit dedicated line but my download speed for large files (flash files of over 5mbs) is really slow. I am thinking this is because of Apache but I don't know much about this. I've read that I should change for a lighter http server for serving static files. The way my server is set up is I have 2 virtual machines running, one doing the PHP processing and the other serving static files, both running Apache, so if I have to change HTTP server for the static files it would be very easy. Although I am not sure if this is necessary or if I can tune Apache to push files faster than this.
View Complete Thread with Replies
Sponsored Links:
Related Forum Messages:
Serving & Playing Breeze FIles
Having trouble hosting Macromedia Breeze files on a brand new dedicated 2003 NT box. Anything I need to adjust? When someone visits the URL, it loads the presentation main screen, but it's supposed to auto-play several movies and it won't. It works fine locally and on 2 unix servers I tested it on, but not the new NT box. The weird thing is, if I remote desktop to that server, I can't play that presentation locally or server thru the linux boxes. I figured it was an extensions thing, but like I said the intro still does appear, and if I go to Adobe.com and play a new "Adobe Presenter" presentation - it works fine.
View Replies!
View Related
What Type Of Server And Os, Etc For Just File Serving- Small Files Like Under 10kb
I have a website that just serves small files, under 10kb most of them. I just need a server that lets me ftp the file to it, set up subdomains and domains for one website. Don't need to manage mysql or anything. Not even php. Just serve files. A good fast OS? Something like lighttpd? Ioono? I'm currently doing 600gb of bandwidth per month. I'm expecting to do about 1000gb by the end of the year. Would a small server like a pentium 4 be able to handle just serving files?
View Replies!
View Related
Mod_Security Log Files Too Large
Linux Fedora 6, Apache 2 with Mod Security, MySQL. Our mod_sec logs get incredibly large very quickly. In the configuration for mod_security, we have specified logging options as SecAuditEngine RelevantOnly SecAuditLogRelevantStatus "^[45]" but the mod_sec.log gets to almost 10 GB (in a matter of 5-6 days) before it is truncated to mod_sec.log.1 and a new one is created. Is there a way we can specify that a max size of one log file is 1 GB, for example? Or another question, how come it gets so huge so quickly? We thought that logging "RelevantOnly" will only display errors / requests that are deemed security risks.
View Replies!
View Related
Hosting Large Flv Files
I have a customer who wants to sell access to videos of conferences he runs. Each flv vid is approx 1 - 1 1/2 hors long approx 380MB each and there will be about 12 videos per conference. approx 4 - 8 conferences per year. My customer suggests 10 - 20 people will buy access to watch each video. Access to watch the videos will be through a password protected webpage. issue - the current site hosting company only allow uploads up to 150MB per file. Can I host the flash videos elsewhere and deliver them through the password protected web page without anyone else being able to see them via server they are hosted on? This would also reduce the bandwidth going through his current site server.
View Replies!
View Related
Transfer Large Data Files
I have a debian box, and have archived a gallery in to a .tar file, 5.77gb. I have a centOS box, and have used wget to bring the data file over to the new server. However upon doing so it only detects it as 1.8gb when it starts downloading. I have terminal access to both servers, just trying to bring my files over from one server to another.
View Replies!
View Related
Large Video Files Need To Be Streamed
I am developing a web application for a private investigative firm. They do surveillance work and therefore have surveillance videos. I would like the capabilities of uploading the videos online and allowing the client to login and view their surveillance video online. Currently, we get the video from the PI, put it on a DVD and then mail it to the client. This takes too long. We want the client to be able to view the video online. Some of these videos can be up to 2 hours long. First, is this even possible? Second, - how much bandwidth would a website like this take? - Is there a host that can hold hundreds of GB of video? I want to convert it to flash to save file size and also so I can stream it.
View Replies!
View Related
FTP :: Stop Uploading Large Files
I'm facing a very strange FTP issue with one of my shared-hosting accounts, while all of my other servers are having no problems but only this one, when I try to upload a file (whatever file) larger than 500kb from my local PCs, in most cases, the file would stop uploading during the process and hang there until it times out. There are 2 interesting things though: The file transmission typically hangs when approximately 248kb of the file have been transferred, pls see the attached screenshot for example. If you look at the attached screenshot, you will notice that the uploading transmission hangs when 248kb of the file have been transferred. This is very strange and what I mean is that for example, I randomly pick up a file, and attempt to upload it onto my host for 10 times, now see, 5 times it will hang when 248kb of the total size have been transferred, 3 times it will hang at other points *near* 248kb (224kb or 280kb typically), 1 time it will hang at another random point, and 1 time it might be uploaded successfully (yes, there is still a tiny chance for the file to be uploaded successfully). My default internet uploading speed is 80kb/s-100kb/s, lately I found that, when I limit the uploading speed on my FTP client (e.g. max. 30kb/s), everything WILL WORK without any problem! No hangs, no interrupt.. Whereas when I free up the uploading speed limitation and let it upload with my regular speed, the problem appears again. It seems to me that the FTP hangs only when the uploading speed is higher than 60kb/s. However my host provider told me that they have customers uploading without any problem at over 400kb/s, and they said "there's no problem or limitations on the server at all". Up until now, I have done following things to troubleshoot the issue but with no luck: Contacted my host. Disabled/Enabled the PASV mode on my FTP client. Tried different FTP clients on different computers (FlashFXP and Filezilla). Rebooted my router and reseted everything with the factory default settings. Contacted my ISP for the issue, they "did something" but nothing were helpful. Rebooted all my PCs. Disabled both firewalls on my PC and on the router. Furthermore, I have asked another friend of mine in another city with another ISP to test the FTP uploading, but unfortunately he got the exact same problem. And I've done some search on the internet for hours but no one seemed to have the same problem..
View Replies!
View Related
Removing Large Files Or Folders CentOS
Something weird happening here. I have tried every string possible... There are a number of folders I want to remove off my server, tried the good old and simple... rm -r /folder/ And then went and ended up with a string as long as my screen. No matter what I do, as it goes recursive in to the directory it asks me if I want to remove each file individually. No matter what string or action I take it insists on asking me as it goes to delete each file. Could this be a configuration option in CentOS?
View Replies!
View Related
Large Media Files & Backups
I've been using Lypha for the past 4 years, but they've taken the last straw (gigabytes of backups went missing and they wont reply to emails as to why). Looking for a web hosting package for under $10/month that has large enough disk-space/bandwidth to allow me to backup large audio / video files to it, as well as the normal site operation (I use it for portfolio website, as well as hosting additional domains)
View Replies!
View Related
Optimizing Lighttpd For Large Files (180mb Avg)
I have four servers with a quad xeon, 4gb ram, and 2x300GB SAS 15K RAID0 harddrives, pushing a total of 1.6gbits. It serves a lot of zip files with an average flesize of 180mb. My question is, how can I optimize lighttpd 1.4.19 to push its max with very low IO-wait. I've looked up some stuff and only found options that deal with lighttpd 1.5 and use Linux-AIO for the backend network. Currently I use writev with 16 workers and an read/write idle timeout of 10s. Logging is off, too.
View Replies!
View Related
How Do You Host Large Video Files And Support The Bandwidth
I've got a client who wants to host audio files... Here are the sizes: 50 x 75MBs 300 x 10MBs 400 x 5MBs That totals 8750MBs or 8.75GBs... If he gets hundreds of visitors, it could end up being 1000's of GBBs or bandwidth. I don't know what to look for to support so much bandwidth... Do you buy bandwidth? Are their special companies out their that host it for you?
View Replies!
View Related
Daily Backup + Rsync Ssh + Large Number Of Files
i just wana know is it safe to do remote daily backup for about 70,000 files? file sizes is about 200kb and every day i have about 1000 new file, so rsync first should check old files becouse i am deleting about 30-50 of them daily and them backup new 1000 files , so how much it will take every time to compare that 70,000 files? i have 2 option now: 1-using second hdd and raid 1 2-using rsync and backuping to my second server , so i can save about $70 each month.
View Replies!
View Related
Does These Speed Test Files From Providers Help You Decide
I am in Europe, and during this week I´ve been doing some speed tests with the files most providers have on their website or they gave me the link here. I am using a DSL 25Mpbs net connection for the tests. Here some results from just 3 providers: FDC (Chicago) - 250k iWEB (Canada) - 1.2MB Take2Hosting (San Jose, California) - 1.4MB My point is, does these speed test file are of any use to have an idea what speed you can expect with the servers? The way I see it ,there´s so many variables (shared and how many shared servers for port, your location, peak hours traffic, etc). So how does this tests influence you to go for one provide, instead of another?
View Replies!
View Related
NFS Slow Speed When Moving Many Small Files
I have mounted NFS part. but when copying a big file the speed is OK like 5-6MB/s but when starting to copying all other (small files) speed is like 20-200KB.s What is the reason and is it way to improve the speed or use other way to mount drive remotely and preserve the same permissions after backup?
View Replies!
View Related
Transfer Large Many Files From Server To Server
In reference to my previous post, i want to tranfer accross 7GB of data, approximatly 80,000 files i believe it is (due to a gallery script). It's currently on another host (on a webhosting account) which uses their own control panel which has no options but to manage databases, the only way i can see to do this is via FTP but it'll take me days. I've tried using compressing and backup scripts, but the damn execution time on the hosts server is too low to allow the files to be zipped. Are there any ways? Can i login to my VPS via SSH and anyhow pull off the files from the other hosts server?
View Replies!
View Related
IIS Just Stopped Serving..
I have been hosting a site for the past number of weeks on IIS on Server 2003..I have a no-ip a/c which is working fine along with port forwarding on my router.. The other day I restarted my server for something but now I cant seem to access my site from across the internet and according down for everyone or just me (website) it is down. I can, however, access the site using domain name from within my lan. I'm not too hot on server 2003 or IIS. What can I do to debug/diagnose the problem?
View Replies!
View Related
Servint - Serving Porn On My Sites
What is up with servint? I have domains and a vps through them. The vps is now serving up porn on all the sites. When I call servint they say I don't own the name but a whois shows i DO INDEED own the name for another year and it is registered to me and they are the registrar!!! The portal user/pass suddenly stopped working so I can't even put in a support ticket All I get is a message machine ... does anyone know if servint staff ever come on this board. It has probably been hacked but it is pretty disturbing when they say I don't own the domain but it is CLEAR THAT I do own it.
View Replies!
View Related
Serving Data From 2 Hard Disks
I have a server with 2 hard drives, say drive A and drive B. Right now all my files, database and data is on drive A, and drive B is empty. Since I have another drive available, I want to split the load between the two drives. I'm ok with having the web pages and the database on one drive. I mostly want to just have the data (I have about 500GB of data) split between the two drives. Note that I want to avoid duplicating the data. I want to have each file on either drive A XOR drive B. Should I map a separate subdomain to drive B and then use that subdomain to serve the half of the data thats there? Is there something I can do with hard/soft links on the server so that even though the data is on 2 drives, users still use the same url to access data on either drive? Any other options?
View Replies!
View Related
Media Serving And Cache Daemon
what I want to do, have a "node" somewhere serve media (static) files from a central server, but cache the static files the first time they are hit, so subsequent requests to the "node" don't require getting the file from the central server. Is there readily available solution to this?
View Replies!
View Related
How To Tweak Linux For File Serving
i have few servers that just serve files (1MB-100MB) Suse Linux on all the servers have minimal apps installed and i already got a llarge performance increase by dumping apache now im look at tweaking at the OS level any settings in Linux itself to speed up sending files down the pipe? net.ipv4.tcp... in /etc/sysctl.conf?
View Replies!
View Related
Static File Serving - Performance
I'm planning to setup a server ONLY for hosting of static binary files varying from few KB to few MB in size. I've seen some of the litespeedtech performance benchmarks, which you can find here: [url] From the "small static file" benchmark chart, i can see that IIS6 beats lighttpd in this test. So i'm wondering does the IIS6 really have better performance at file hosting than lighttpd. Actually it does not matter which operating system i will be using at this server, since i will use it only for file serving. With lots of concurrent connections. Possibly thousands of connections. I need some feedbacks on this, so i can decide, IIS or lighttpd. Few more bucks for win2k3 won't be an issue here, if it's performance is better than lighttpd for this kind of use.
View Replies!
View Related
10 Second Lag Serving Static HTML
this isn't my server, so I don't have a lot of information about it, other than it's a Linux/Apache Dedicated server at EV1. (cPanel/WHM) There are other sites on the server, and they are running fine. One site has a terrible lag. It takes about 10 seconds to serve up a static HTML file.. Now, it's not like the server is slow. It's like this: You request a small HTML file. The site sits and thinks about it for about 10 seconds, and then after that everything processes quickly. The forum on the site is the same way. Everything you click on works fine and loads quickly after that initial delay passes. Is there a common configuration problem that might be causing this? If you want to see this phenomenon, here are a few test files: [url] [url] [url] [url]
View Replies!
View Related
Apache Serving Blank Pages
I am having this issue with a new box where Apache at random serves blank pages and won't serve anything but until it's restarted. I see no errors in /var/log/messages either to show why this is happening. It's not all pages either just several various ones. I know it's not a programming issue either because the only thing the server is running is one instance of vBulletin. I look at top during this and CPU and RAM usage are very low. The only fix to get Apache to serve the pages is to restart it. Anyone have any clues on what it could be? I set my php.ini memory limit to a gig (I have 16 gigs in the sever)
View Replies!
View Related
Preferred Operating System For Web Serving
I have been online since 95 and I've got a lot of information I could help others with regarding web development and interface design. One thing I'm terrible at is System Administration, but I'm getting better and my first step is to pick a reliable operating system for both the webservers and the database servers. Would anyone be willing to answer the following questions or point me to a thread that already discusses these? A little background, our site has 200k members, 30k active and is growing by about 5k a week. We still run MySQL 4.0.27 on the DB servers and they are running FC2. The webservers are Apache 1.3.37 and PHP 4.4.2 running FC2 and FC4 as well. My questions 1. What OS would you prefer to upgrade to for the web servers? and a "few" reasons why? 2. What OS would you prefer to upgrade to for the database servers? and a "few" reasons why?
View Replies!
View Related
Problem Serving Gzipped Static Content
I'm trying to serve as much gzipped content as possible on my web site. I will gzip dynamic pages on the fly using PHP, and I plan to gzip static content just one time, not to use much CPU, and serve css.gz, .js.gz files, etc... So, I uploaded a test.html file and a .css file that changes the color of the text to red. I gzipped the css file: %gzip style.css I also uploaded an .htaccess file with this content: RewriteEngine on RewriteBase /gzip/ RewriteCond %{HTTP_ACCEPT_ENCODING} gzip RewriteRule ^(([^.]+.)+)css$ /$1css.gz For now I'm not going to include any gunzipped style sheet, just to make sure my browser picks the gzipped style sheet. I also uploaded a phpinfo.php file to make sure I have mod_gzip. So, here's the list of files I have: - .htaccess - style.css.gz - test.html - phpinfo.php But when I open the test.html file: [url] the style sheet isn't applied. If I upload a regular style.css file, it works fine. It seems the Rewrite rules aren't taken into account. Do you know how I could server the gzipped style sheet instead?
View Replies!
View Related
Speed- At FDC ( Speed Tests )
I am having some serious speed issues with my 1Gbit server at FDC. After opening a ticket, they've simply dismissed it as a server configuration problem. However I am convinced it isn't because certain ISP's (usually universities) get good speeds, usually 700kb/sec but the vast majority of my users get between 20-50 kb/sec and it's causing a lot of complaints. Furthermore I have other servers with FDC which are 100mbit which perform better than my 1 Gbit one. There are no server bottlenecks (CPU/RAM/HDD), since I've closely monitored them (PRTG) and they aren't even heavily utilised. So the problem is with the network at some point. Speed Test : [url] where abouts you are downloading from, your ISP and net connection. Wget's from servers are also welcome as are traceroutes.
View Replies!
View Related
An Otherwise Great Performing Server Now Delays Prior To Serving PHP/mysql
One of my dedicateds (Fedora 7 / apache 1.39 / php5 mysql5) has been running flawlessly but now has a strange ~4-5 second delay when serving up PHP/mysql pages such as Wordpress. I say delay because it appears to think about it for several seconds, then when the page finally begins the refresh process, everything serves up lightning fast. Latency is low, direct file downloads are faster than ever. ModSecurity is running, and accounts for about 1 second of that delay (tested with it on and off), but the other 3-4 seconds is a mystery.
View Replies!
View Related
Transferring 200 Files : Total Files Size Is 25 GB!
I couldn't keep my mouth shut (technically fingers). A customer wanted to upgrade servers and he needed a way to move the data across. Since I don't allow hard drives to be swapped, they have to do it manually all by themselves. I generally allow up-to 4 days for them to transfer data and make DNS changes, etc. But this time, I offered help! I agreed to move the data (darn me) and it just came out of me, involuntarily. God knows what just happened... but in a positive way, customer is extremely happy! So... Both servers are on cPanel - with root access (duh) 200 odd files which total to 25 GB 1 database about 100 MB in size (no biggie) I was planning on using one of my Windows 2003 servers (via remote desktop) to download the 25 GB and upload the 25 GB, but that sounds like a waste of resources and time.
View Replies!
View Related
Backing Up Large DB
Just had a quick question about backing up a large MySQL DB. I have a database that is 50gb with about half a billion entries in it. One table itself is about 40gb, the other 10gb consists of smaller tables. The problem is, I want to back the database up and be able to keep it LIVE at the same time (as it will fall behind quickly if it's pulled for more than a few hours, as there are somewhere in the area of a million entries an hour, plus other deletions and queries).
View Replies!
View Related
SQL Too Large
I'm having a lengthy issue where my databases are to large to import in phpmyadmin using plesk. Unfortunately I dont have direct access to phpmyadmin and can only access it by DB user through plesk. I have tried to edit php.ini in the following locations: upload_max_filesize = changed this to 64M post_max_size = changed this to 32M maximum_execution_time = changed this to 300 maximum_input_time = changed this to 300 Why am I still not able to import my DB's which are about 8MB each?
View Replies!
View Related
Large MySQL Migration
I wasn't sure where to post this so here goes, I need to migrate a MySQL DB, in the past I have just created an SQL file and used that method (sometimes having to split the SQL file up) but now the DB is about 50 meg and 733,233 records. Is there an easier way to migrate the Database from one server to another?
View Replies!
View Related
Allowing Large Downloads
I'm selling downloads of music files. The zip files are quite large. I've had several people complain that they get a message that the server resets their connection before the download finishes. What can I do to allow these large downloads? I'm on IIS.
View Replies!
View Related
Large MSSQL DB Hosting
Does anyone know any shared hosting or semi-private VPS hosting for a site with large MS SQL2005 DB? Our DB is currently 1GB already but could get up to 5-10GB when all is said and done. The site is a B2B networking platform with many users, company profiles, products, etc.
View Replies!
View Related
Large Backup Storage
I have a website that is approx 50GB, that I essentially would like to take offline for a while. Obviously, while the site is offline, I don't want to be paying for my server. Can anyone let me know of some options to "store" this mammoth of a site. Downloading it locally is not an option, so it needs to stay in the cloud for the lowest possible price.
View Replies!
View Related
Large Apache Domlogs
to delete and reset the domlogs files from Apache server, as they have become too large, but I want to keep a record of the past few years website stats. I could download the domlogs files (they look like MS-DOS applications) but once I have I am not sure how to read them, is there some off server application that could open these files? I currently view stats in awstats from cPanel, but the size of the files is causing problems with updating now. The only thing I can think of doing is taking screenshots of the relevant stats that I want to keep, before deleting the files, but was thinking there must be a better solution than that.
View Replies!
View Related
Large Scale GB Hosting And Other
I would like to have around 100 Gb with room to expand if possible. This will be hosting for a radio station, archival material. I am unsure of where to look, the sites I have seen are pretty small in scale as far as capacity. Further, I would also like to be able to re-sell this, instead of just setting up a straight account. Further if at all possible I would like to bundled with some other small hosting packages (wordpress sites) on a reselling basis.
View Replies!
View Related
Large File Sizes
My client's website needs to hold files that are around 60 or 70 MB. The host only allows files up to 10 MB. Is that typical? Right now I'm linking to a file storage but would rather make the files available from my site without going to a 3rd Party Site. He doesn't want to zip his files either - just to be a straight download.
View Replies!
View Related
Moving A Large Website
I couldn't find a suitable category for this topic, please feel free to move if I made a mistake. I have a big website. MySQL Size: 1 GB Diskspace: 2 GB It's over loading the server, so I am moving it to a dedicated server. In the past, basically I would pause the live website (e.g. turn off forum). Move the data, turn everything back on in the new server and then change the nameservers. But for this website, I don't want to pause the website nor I wish to loose any posts because of DNS propagation. (e.g. user might post data, which is stored in the current server. But then after propagation, the user won't see his post)
View Replies!
View Related
Cannot Receive Large Attachments
My server is running WHM 11.15.0 cPanel 11.18.6-S24739 / REDHAT Enterprise 4 i686 on standard. I am using EXIM. Lately, users on my server have not been able to receive most attachments. E-mails with small attachments, such as an 11,000 byte file, show up just fine. An e-mail from the same person, but with a 560,000 byte attachment will simply never show up, and there is no returned "bounce" message to the sender. It is like the e-mail was never sent. The odd thing is, e-mails sent to and from e-mail addresses hosted on my server have no trouble sending/receiving large attachments of any size (I have tested up to several megs in size). For example, tom@example.com and joe@another_example.com both have their domains hosted on my server. They can send each other large attachments with no trouble. But, if tom@yahoo.com or tom@gmail.com tries to send a larger attachment to tom@example.com, the e-mail and the attachment never show up. Things to note: 1) The e-mail accounts are no-where near their storage limits. 2) message_size_limit is not set, so I assume the max attachment limit is the default 50mb.
View Replies!
View Related
How To Transfer A Large File
I have a situation here. I wanted to transfer a file (about 10GB) from one server to another server. Both server are on the same LAN. Inside server1, I zipped a folder as file.tar.gz From server2, I uses wget http://server1.com/file.tar.gz and it says "404 Not Found" I suspect it won't allow me to download because the file is too big.
View Replies!
View Related
To Transfer A Large DB To Another Server
to transfer a MySQL database with way over 1 million rows to a different server. What is the best way to do this without losing any data? I've tried using the export feature in phpMyAdmin but it doesn't seem to handle it very well. There has to be a feature in WHM or cPanel,
View Replies!
View Related
Ext3 On A Very Large Filesystem
We're considering deploying a large server that will have 8x 500GB drives in a RAID-10 config. I intend to use a 3ware 9650SE w/ BBU along with A/B power to each of the PSU's. My question is... since this will return into a 2TB array/partition, in event of a crash (kernel panic, etc -- I expect a power outage will be very, very rare if at all) what do you guys think the fsck time would be? In my experience a RAID BBU significantly drops it, sometimes to the point of no manual fsck required, but in event of a manual fsck shouldn't the BBU be able to provide more consistent data (less errors) and therefore a much shorter fsck? Maybe just recovering the journal?
View Replies!
View Related
Large Amount Of INSERTs Into DB
I started tracking and collecting information about every page load on my site, which made the DB grow very fast. In 24 hours I had over 1.4 million INSERTs, so the DB will grow by over 170MB/day. The inserts so far didnt affect the performance of the server (dual clovertown, 4gb ram, SCSI HD), but at this rate the DB will grow by over 5GB per month. Is it a good idea to let the DB grow to those sizes? Would it affect performance if a table had several 10s of millions of rows? Public doesnt run any SELECT/UPDATE queries on that table, only admin does (me).
View Replies!
View Related
Moving Large Site
to move a site to another server at another datacentre that have approx 100 GB of media files. Typically for small sites i am able to use server to server ftp to move files. However, when moving large site like this, ftp might be disconnected and lost track of which files are not transferred. How can i transfer/move the files (server to server) and resume if disconnected?
View Replies!
View Related
|
TRACKER
