Moving Large File Storage, Upload/download To Amazon S3
Jul 24, 2007
I'm currently running on a VPS. My site allows for large file uploads and downloads, with files over 600mb in size.
The server has issues when the site gets three or more requests for large file downloads. I'm trying to grow this site to thousands of users and it is hard to do when the site can't handle even three.
I've been told by my host that I need to upgrade to dedicated. My VPS only has 512mb RAM and one large file download is eating up that RAM. This is causing the issue.
I'm a newbie and while I knew I was risking a bit by going with VPS I do find it a bit annoying that these guys advertise 1TB of bandwidth per month but I can't even support downloading 1GB at the same time....maybe it's just me...
Anyway, I am now looking into moving the large files and the upload/download over to Amazon S3. If I do this I am expecting my RAM usage on the VPS to greatly decrease. Is this correct? If my PHP code is running on the VPS, but the actual file download via HTTP is coming from S3, that should not be a heavy load on my box, correct?
on good hosting setups for getting large amounts of disk space.
I would like to be able to offer up to 2Gb storage space for 100s, maybe up to a few 1000 users - any solution should scale well. The files would be static files that might be up to 400Mb in size.
It would be nice to be able to give users FTP access to their disk space, although it's not a core requirement.
I have a website that is approx 50GB, that I essentially would like to take offline for a while. Obviously, while the site is offline, I don't want to be paying for my server.
Can anyone let me know of some options to "store" this mammoth of a site. Downloading it locally is not an option, so it needs to stay in the cloud for the lowest possible price.
Our congregation needs webspace to archive mp3 and mp4 files of the weekly messages. Low mp3 is about 3 MB size. High mp3 is about 20 MB size. mp4 video is about 325 MB size. We would provide links to the files so they could be downloaded - no online streaming is needed. Also no normal website stuff - just file storage space and ability to download by anyone who has the URL links to the files.
1and1.com has 250 GB storage with 2.5 TB/mo bandwidth usage for $10/mo. Their phone rep said this type of use of their web space is OK. I wouldn't even look any further except I've read a lot of bad reviews about them.
I've two connections: ADSL (2.5mbps DOWN, 512 UP) and Cable (256 Down, 56 UP). Uploads are free on ADSL, Cable however is truly unlimited.
I'm wondering if there is any way to upload using ADSL and download using Cable? since there seems to be some amount of inward data transfer even while uploading (p2p) and the ADSL is highly expensive so I wanted to use Cable for that.
I have found SL can offer you 12*1 TB drive based systems, after RAID-5 and Win 2003 install you get just over 10 TB of storage. The monthly price works out to $1000/Month.
I know some time ago LeaseWeb offered these type of storages....any one else know of any others ?
I have just moved hosting script account from one server to another server and now when I try uploading an image, I am getting the following error
Warning: move_uploaded_file(./uploads/8e90ced776.jpg) [function.move-uploaded-file]: failed to open stream: Permission denied in /home/imgrip/public_html/upload.php on line 29
Warning: move_uploaded_file() [function.move-uploaded-file]: Unable to move '/tmp/phpDmsMkq' to './uploads/8e90ced776.jpg' in /home/imgrip/public_html/upload.php on line 29
Anyone would be kind enough to let us know how to fix the error?
I couldn't find a suitable category for this topic, please feel free to move if I made a mistake.
I have a big website.
MySQL Size: 1 GB Diskspace: 2 GB
It's over loading the server, so I am moving it to a dedicated server.
In the past, basically I would pause the live website (e.g. turn off forum). Move the data, turn everything back on in the new server and then change the nameservers.
But for this website, I don't want to pause the website nor I wish to loose any posts because of DNS propagation. (e.g. user might post data, which is stored in the current server. But then after propagation, the user won't see his post)
I have to move some large websites from one host to another. The websites contain data about 1 - 3 GB, and my internet connection here is not that fast, so that downloading and uploading would take many hours. Unfortunately I have no SSH-Access to both accounts (old and new one), so I have to make it by FTP.
So I tried to make a tar file and moved it from the old to the new host. I made a tar archive of the whole webfolder (I used php to execute the shell commmands) and moved it by ftp to the new host. But now I´ve got troubles when extracting the archive on the new host: the extracted files are obviously created by the wrong user, so I can´t delete or access them by ftp.
So I´m trying to find other solutions to fix these problems. Maybe someone else had the same problems and could give me advice how to move large sites? Or do you know php-applications that can directly connect one host accout to another via ftp?
i manage linux apache webserver with a few wordpress blogs and from time to time i see someone inject a malicious .php file into wp-content/uploads/2014/10/ directory.
i think its some bad plugin or theme, but these is more blogs, i ugrade, update, WP, but
how can i setup some monitor to tell me which php file (or even line in php file) injected that malicious .php ? I have linux root access so i can setup anythingÂ
I have been reading about Amazon S3's service at [url] and I was wondering if it would be wise to use their service for a file host (like rapidshare..) I may be starting?
How good is their bandwidth and what is their policy on copyrighted material?
Or should I buy my own servers from an unmetered provider, such as alphared?
I'm moving a large, 4 gig site between 2 servers (both have cpanel... if that's an option). Is there a way to do it quickly and correctly? How would I transfer such a large site?
What's the best way to set up an automatic file backup to Amazon's S3 service? I have a Linux cpanel VPS. I have minimal VPS administration knowledge but at least the VPS is managed by a good team
Is there any way to do incremental backups to save on the bandwidth charges?
(a full backup for me would be about 15 gigs each time)
I'm building a kind of a file sharing site also with video conversions ( convert a full DVD to flash and stream) and if i can find more ideas, even more features. It won't be free (i'm not sure which way to go though, subscription-based or pay per bandwidth + storage) and it will support download managers. I'm not sure which way to go : Joyent or dedicated servers with RAIDs. If i distribute files over servers, and a users wants the files to be converted to flv etc., it will increase the server load, ( and if i put a lot of users in 1 server it will increase the load even more) so it won't be very good.
So i thought SAN for this, but couldn't find any hosts serving it ( especially for a reasonable price ). Joyent is close enough (Amazon S3 is very very expensive), but although bandwidth and storage prices seem good, server power is low ( and very expensive if higher ) Which way should i go?
I have a 6GB backup file created with another Plesk Backup Manager, now I trying to upload this backup file to my Plesk Backup Manager but after upload 3% I am getting "413 Request Entity Too Large" error, I tried with disable NGINX but still getting this error.
how can I resolve this error or is their any other way to upload my file on backup manager?
I see that Backup Manager have a file size restriction of 2GB how can I increase this?
Network File Storage (NFS), does anyone use it or find it useful?
From what I see are advantages:
- Raid redundant
- Good backup
- Cheap extra space
Cons:
- slower speeds read/write speeds?
Looking for someone to add on and change my point of view on NFS. Also if you like NFS what is a good model for us to get. We are currently looking at the the Dell PowerVaults...
I run several sites and all of them are hosted at invision. The main reason for having my sites hosted there is that my sites are "forum centered" and I'm very happy with the service that I have got over the last few years from them, so I don't want to change that.
However, I now wish to expand and provide my users with a file repository. The problem is, whilst hosting my sites at invision is fine, hosting my files there would be quite expensive...
Thus, I'm now looking for a host to host my files and nothing else. (I run chess sites, so I'll be providing my users with files and possibly a gallery. All legal material, of course).
I don't need any download manager or anything of the kind. Invision forums actually has its own download manager, and I can have my files hosted externally (i.e. other than on my site).
Thus, I'm looking for a host that offers specific packages for what I am looking for - I would not need scripting or any 'fancy' features, just file storage with FTP access.
How much space? around 1 GB, possibly 2 in the future, maybe 3 or 4 if I add the gallery one day. And bandwidth, as people will be downloading files from my site.
I've been looking around, but it's just so difficult: they all offer webhosting services for people who need to have their sites hosted, etc. and that's not what I need.
Any recommendations?
I don't have any fixed budgets, my focus will be on price, speed and reliability. Preferably a hosting company which has been around for a while and has good reviews.
is there any web hosting that can be used as file storage (>10G) and cheap? I have some huge files, but many web hostings can not accept non-web-content files.
I think I messed php config and I can't upload anything with php now Dir is chmoded on 777 and File_Uploads = On in php.ini
I'm running lsphp5 with suhosin, when I try to import db via phpmyadmin I get error: Uploading is not allowed and when I try to upload some file via php script I can't
I have a forum ( VBulletin ) in admincp Upload file is ok and high, For example .Zip file are max 3 Meg upload, but i want upload .Zip in thread, i can not upload over 1 Mb, and i view database error!
When I try to install ffmpeg, but it fail. The server cannot upload 1KB file from php. $_FILES['xxx']['size'] return to 0 $_FILES['xxx']['tmp_name'] return to ''
