Scenario: Our website has 3 terabytes of data (photos, database files, etc).
1. How does one move all of this data from one hosting company to another?
2. How does this typically work? Does it involve buying the actual disks?
3. Does this need to be specified in a contract that's signed with the hosting company?
Copying it over the Internet (ftp, sftp, or http) is not an option.
Recently I changed server providers, so now I'm looking for a way to transfer all the data to my new server. I have a total of 420GBs of files in my secondary HDD that need to be transferred.
The old server is at a 10Mbps line, the new one is at a 100Mbps one. From old server, less than half the pipe is being actively used. So theoretically, I should be able to transfer it all in about a week.
I tried
1) SCP. That was waaay too unreliable. And I couldn't get it to restart from the point left on whenever the transfer stopped (like when the servers were restarted).
2) Transfer using a web script. Way too slow, got to about 35GBs, total would take like 2 months.
Is there any other, reliable way of transferring data from server to server?
Everything on our Qmail installation is working fantastic except for one thing (most of the qmail patches are installed from [url]:
If a user sends an email to several recipients, often times, all users of the list will receive several copies (10 times or more) of the email.
So far I think the situation is that a user sends a weekly update (sort of mini mailing list to 20 users or so) by just adding them as BCC in their outlook.
Then I think one of the 20 emails bounces or throws back a deferral, and then qmail resends the email many many more times not only to the bounced recipient (but also to the entire list of recipients in the BCC)
I'n my email today...
Quote:
My name is Todd Mitchell, I am Chief Operating Officer at Site5. We rarely send out company wide emails, so you can be sure that a very special announcement is necessitating this type of mass distribution! Today I have some remarkable news to share with you regarding Site5, your web hosting account(s) and a new partnership we've solidified over the last several months. We are proud to announce a new strategic partnership with The Planet (www.theplanet.com), a very well-respected infrastructure provider, which will soon be housing our server fleet and network in their state-of the-art, enterprise-level data center.
As I'm sure you're aware, Site5 is well known in the web hosting industry for a number of reasons:
- Site5 is innovative: Demonstrated in the research and development of Flashback, NetAdmin/SiteAdmin, Backstage and our stable web hosting platform.
- Site5 is transparent: Our weblog [url], forums [url] and corporate web site [url] are prime examples of our transparent nature and operation. We are upfront and honest about everything we do at Site5. Clients are family.
- Site5 has award winning support: Whether you contact our billing, sales or support groups you are guaranteed a high level of customer service. We pride ourselves on our 24 x 7 in-house support group. Site5 does not outsource and that makes a difference.
We are building upon those core competencies by focusing our efforts on improving our server hardware, data center and network via this new business relationship. Throughout our continued growth we have added value to every aspect of our hosting platform without increasing prices to clients, and this announcement is certainly not an exception.
Site5 Internet Solutions, Inc. (one of the largest and fastest growing web hosting providers in North America) and The Planet.com Internet Services, Inc. (the largest dedicated hosting / infrastructure provider in North America) have come together to form a mutually-beneficial partnership. We have selected The Planet to be Site5's sole data center, hardware and network provider after evaluating several other data center providers relative to our current hardware infrastructure. Following months of meetings, several on-site data center/ hardware evaluations, and a significant amount of consideration for our clients, we selected The Planet because they maintain a great support structure, the best hardware quality, and a shared desire to help Site5 provide the best hosting experience in the industry.
How will this affect you?
- Name brand servers: Our entire infrastructure will be migrated to enterprise level Dell servers custom made and tested specifically for Site5.
- Hardware upgrades: Every single server in our existing fleet will be upgraded to new hardware! We are standardizing our fleet on dual core Intel Xeon processors and large redundant and fast RAID arrays.
- Network upgrades: The Planet over the last several years has built out a substantial network. Rated #2 in North America, their network is resilient, reliable and redundant at all points. This network upgrade will ensure that your visitors and clients can access your web site(s) 24 x 7 without interruption and we will be able to guarantee that your pages will load exceptionally fast.
This new partnership is a substantial commitment for The Planet and Site5. Both companies employ exceptional people and this partnership is just a glimpse of great things to come from Site5 over the next 12 months. We are committed to keeping you informed and we will continue to send emails with pertinent information regarding this new partnership. Please keep an eye out for our emails and remember if you think you've missed an email, all notices and information will be reposted on our official forums - [url]. The official press release regarding our new partnership will be released to financial analysts and news wire services on Tuesday April 3rd, 2007.
We hope this news is as exciting for you as it is for us. As we finalize the migration details for each server, we will post forum updates with all of the information you will need about your server's move and how/when your account will be upgraded. Thousands of people have been a part of the Site5 community for years, and we continue to see record growth, so we wanted to be sure that this move would help express our gratitude for your business.
Does anyone have any suggestions on how you would physically move servers from one datacenter to another with minimal downtime?
The IP address is going to change, so the server will go off and then come back online with a new ip.
Just how much downtime is acceptable before customers get annoyed?
When is the best time to do it?
Move them yourself or courier or any other ways?
I couldn't find a suitable category for this topic, please feel free to move if I made a mistake.
I have a big website.
MySQL Size: 1 GB
Diskspace: 2 GB
It's over loading the server, so I am moving it to a dedicated server.
In the past, basically I would pause the live website (e.g. turn off forum). Move the data, turn everything back on in the new server and then change the nameservers.
But for this website, I don't want to pause the website nor I wish to loose any posts because of DNS propagation. (e.g. user might post data, which is stored in the current server. But then after propagation, the user won't see his post)
to move a site to another server at another datacentre that have approx 100 GB of media files.
Typically for small sites i am able to use server to server ftp to move files.
However, when moving large site like this, ftp might be disconnected and lost track of which files are not transferred.
How can i transfer/move the files (server to server) and resume if disconnected?
I have to move some large websites from one host to another. The websites contain data about 1 - 3 GB, and my internet connection here is not that fast, so that downloading and uploading would take many hours. Unfortunately I have no SSH-Access to both accounts (old and new one), so I have to make it by FTP.
So I tried to make a tar file and moved it from the old to the new host. I made a tar archive of the whole webfolder (I used php to execute the shell commmands) and moved it by ftp to the new host. But now I´ve got troubles when extracting the archive on the new host: the extracted files are obviously created by the wrong user, so I can´t delete or access them by ftp.
So I´m trying to find other solutions to fix these problems. Maybe someone else had the same problems and could give me advice how to move large sites? Or do you know php-applications that can directly connect one host accout to another via ftp?
I'm moving a large, 4 gig site between 2 servers (both have cpanel... if that's an option). Is there a way to do it quickly and correctly? How would I transfer such a large site?
View 3 Replies View RelatedI'm currently running on a VPS. My site allows for large file uploads and downloads, with files over 600mb in size.
The server has issues when the site gets three or more requests for large file downloads. I'm trying to grow this site to thousands of users and it is hard to do when the site can't handle even three.
I've been told by my host that I need to upgrade to dedicated. My VPS only has 512mb RAM and one large file download is eating up that RAM. This is causing the issue.
I'm a newbie and while I knew I was risking a bit by going with VPS I do find it a bit annoying that these guys advertise 1TB of bandwidth per month but I can't even support downloading 1GB at the same time....maybe it's just me...
Anyway, I am now looking into moving the large files and the upload/download over to Amazon S3. If I do this I am expecting my RAM usage on the VPS to greatly decrease. Is this correct? If my PHP code is running on the VPS, but the actual file download via HTTP is coming from S3, that should not be a heavy load on my box, correct?
any opinions on S3?
Our congregation needs webspace to archive mp3 and mp4 files of the weekly messages. Low mp3 is about 3 MB size. High mp3 is about 20 MB size. mp4 video is about 325 MB size. We would provide links to the files so they could be downloaded - no online streaming is needed. Also no normal website stuff - just file storage space and ability to download by anyone who has the URL links to the files.
1and1.com has 250 GB storage with 2.5 TB/mo bandwidth usage for $10/mo. Their phone rep said this type of use of their web space is OK. I wouldn't even look any further except I've read a lot of bad reviews about them.
I have a debian box, and have archived a gallery in to a .tar file, 5.77gb.
I have a centOS box, and have used wget to bring the data file over to the new server.
However upon doing so it only detects it as 1.8gb when it starts downloading.
I have terminal access to both servers, just trying to bring my files over from one server to another.
I'm trying to move data from my old server to new server, using WHM, now thing is while moving it just uses all IP on the system and when there are 0 IP's free, it says copying failed, How to make all accounts goon just 1 IP?
View 1 Replies View RelatedI made a backup of my ancient computer and put the whole thing in an external hard disk.
Now, when I opened my external hard disk, I found a lot of .dat files. I am wondering how can I convert them to something human readable, or how can I move the information to my new computer!
I am using Wordpress on subdomain.example.com
I want to move all the posts I have on that Wordpress site to:
somewhereelse.example.com
So basically changing a whole Wordpress blog from one subdomain to another subdomain on the same domain.
I want to keep all posts I have.
Could I simply back up the mySQL database and then upload it to the new subdomain? Will this work? I have some doubts as I don't know if the database it is linked to the domain name or not.
to move data (a lot) from one server to another. The thing is that the old server's host will not allow SSH access, not even just for a few hours. The new server is a dedicated, so I will of course have SSH there, but how should I handle this situation?
The data in question is massive..Much too much to download to the PC via FTP and upload to the new server. I'm not too familiar with FTP on linux. Could I use SSH on the new machine to FTP into the old machine and recursively grab everything (IIRC, the FTP protocol doesn't allow recursive gets...although it's been awhile since I've used CLI FTP)
My server provider is moving my server to a new data center, I am going to keep my data on it.But I am going to be assigned two new IP's.I have a Centos server with WHM and CPanel.
tell me exacly on which file on my server i need to configure my two new IP's, is changing them on WHM enough? Do I have to edit the DNS zone for each domain?.
I just took a server at a hosting service where I already have several Plesk servers running. The servers I already have running are equipped with 2 conventional disks in RAID1
This new server is equipped with a 2 x 2 TB and a 128 GB SSD disk. With a setup wizard you can install Plesk which comes with licenses.
This new server has a bit different partition model where /var is mounted on the SSD and there's a /data that's mounted on the 2 TB RAID. This /data will then not be used for Plesk.
I can think of many scenarios to change this, but I don't want to find out in the near future that back-ups aren't working because I'm using a symlink somewhere (just an example). The increased speed of the SSD is of course good for the databases, so maybe it's better not to move everything to that partition.
I'm thinking of creating mount points instead of /var/www , /var/qmail and /var/lib/psa
Copy its content to /data/var/www , /data/var/qmail and /data/var/lib/psa
Then modify /etc/fstab so these will point to the appropriate folder
I think it's more robust and transparent instead of using symlinks.
And what are the folders I chose to move?
Apache didn't want to start due to a permission problem.. I think I solved it by making SELINUX permissive
grep SELINUX= /etc/selinux/config
Code:
# SELINUX= can take one of these three values:
#SELINUX=enforcing
SELINUX=permissive
[Code].....
I have a dedicated windows server. I have it protected to the best of my ability (disabling administrative shares, anti-virus, anti-spyware, disabled remoted desktop, firewall, etc.). The purpose for the server is to host game servers. Over the past month, ive noticed massive amounts of bandwidth being used. For example Windows reports the bandwidth usage at a little over 2 GB in 1 day, yet on my control panel which reads the traffic from the switch, it shows that 9 GB have been transferred. I asked the colo what's happening and they said that traffic that is blocked by the firewall is still counted toward my monthly limit. But how can there be 7 GB of blocked traffic, keeping in mind i never had this problem months ago. As a test, last night i disabled all game servers and applications that use bandwidth. In the past day over 9GB has been transferred, despite the fact that i have nothing running that takes more then 50KB of bandwidth.
Aside from these bandwidth issues, everything is running fine, my passwords are still the same, etc. Is there anyway detect security holes, so that i may patch them when i reinstall the OS? I asked my colo for a security audit.
My question is, do you think my server has been compromised?
This is applies to both Horde and roundcube webmail client software;
Using Plesk 11.5.30 with Horde 5.1.5 or roundcube 0.9.5 on CentOS Linux release 6.5 (Final).
We have seen this behavior occur on multiple servers.
Clients experienced slow to no response after executing a search, which eventually results in a failed to communicate with the server-error in the webmail client.
The Apache server log shows script time-out errors when searching larger mailboxes (i.e. larger than 950 MB), this does not happen on smaller mailboxes.
We have seen errors like the following in the Apache server error log with Horde (personal data like IP-address and domain name are x'ed out):
[Thu Jun 19 14:55:06 2014] [warn] [client xx.xxx.xxx.xxx] mod_fcgid: read data timeout in 45 seconds, referer: http://webmail.xxxxxxx.com/imp/dynamic.php?page=mailbox
[Thu Jun 19 14:55:06 2014] [error] [client xx.xxx.xxx.xxx] Premature end of script headers: ajax.php, referer: http://webmail.xxxxxxx.com/imp/dynamic.php?page=mailbox
And with Roundcube:
[Tue Jun 17 13:02:04 2014] [warn] [client xx.xxx.xxx.xxx] mod_fcgid: read data timeout in 45 seconds, referer: https://webmail.xxxxxxxxxxx.com/?_t...d=19445&_mbox=INBOX&_caps=pdf=0,flash=1,tif=0
[Tue Jun 17 13:02:04 2014] [error] [client xx.xxx.xxx.xxx] Premature end of script headers: index.php, referer: https://webmail.xxxxxxxxxxx.com/?_t...d=19445&_mbox=INBOX&_caps=pdf=0,flash=1,tif=0
Steps to reproduce:
- use a large mailbox (950 MB or higher)
- login to the webmail (Horde or roundcube)
- do a search in the search field on the top right
- the time-out error should appear in the server Apache error log (after at least 45 seconds)
This seems like an inefficiency or bug in the search query that searches the user's mailbox. Is there any other way we can prevent this issue and the error messages?
how the inside of a DC is but I was looking at NetDepot's.
I thought that they used rackmounts in datacenters?
Like these:[url]
These appear to me as desktops?
[url]
Am I mistaken?