I just had a quick question about backup solutions. What advantage would I have by setting up 2 HD's in a RAID-1 array as opposed to just doing daily automated backups on one of the drives.
The way I see it, if I have automated backups, HD use for that backup drive is limited to say 20 minutes a day. In a RAID-1 array however, both drives are used at the same rate. Wouldn't this provide better life expectancy for the backup drive, granted it is at the expense of having a guaranteed instant replacement for that original drive?
Reason I'm asking is because I'm setting up a Mac Mini for a friend as a web server and he would like to have data backups. The only way to add space is to intall an external hard drive so my options are a bit limited.
I want to set up my server (a linux dedicated server) to automatically create daily backups of the pop3, mysql, & webfiles. I want it to go to a server which i have purchased with the exact same specifications.
I am not very good at unix command line/scripting. So what I need is for someone to help me define the backup strategy, select the scripts, and tell me of how to make sure backup server is secure.
A couple weeks ago, I encountered a big server crash on my VPS that caused me a lot of downtime. I'm currently trying to figure out a solution to keep a current "clone" of all of my server accounts on a second server. That way, if I ever encounter another crash, I'll be able to simply change DNS information to have all accounts "live" using the backup server.
I appreciate any input, advice, suggestions, criticism, etc. Here's what I have in mind...
1. I currently have all of my websites hosted on Server #1. (We'll call it that for the sake of avoiding confusion.)
2. I have an automatic nightly backup setup via cPanel / WHM that backs up all accounts from Server #1 to Server #2 via FTP. (Server #2 is in a totally different data center, with a different provider.)
3. The nightly backup packages all of the accounts as "cPanel Full Backups." So, they're compressed, and as such, they don't work as "live, functioning websites" on Server #2.
The only way to make them "live and functional" on Server #2 would be to use cPanel to "restore" the backups.
4. So, what I'd like to do is setup a CRON job that would automatically "Restore" the backups each morning on Server #2. That way, Server #2 would always have a functional version of all my accounts, that is less than a day old. Then, if Server #1 ever crashed, I'd just have to change DNS information to point to Server #2, and all of the websites would be live again, without having to physically restore all of the backups using cPanel.
I don't know a ton about CRON. However, as I understand it, CRON couldn't actually make cPanel restore the backups. However, I'm assuming that when you use cPanel's "Restore" function, it just goes through a series of processes. So, it seems logical to me that, if you knew what those processes were, you could write a CRON job to automate the process every morning.
Did that make sense?
If so, is it possible?
Do you guys have any input, criticism, etc?
If it's doable, can you make any suggestions that would help me make this happen?
Finally, if you think you have the expertise to make this happen, I'd be interested in chatting with you via Private Message. I'd be willing to pay to have this done.(Note to Moderators: I'm not sure if my last comment is allowed or not ... if not, please feel free to remove it. I'm far more interested in the discussion of this process than trying to solicit help in making it happen.)
I encountered a big server crash on my VPS that caused me a lot of downtime. I'm currently trying to figure out a solution to keep a current "clone" of all of my server accounts on a second server. That way, if I ever encounter another crash, I'll be able to simply change DNS information to have all accounts "live" using the backup server.
I appreciate any input, advice, suggestions, criticism, etc. Here's what I have in mind...
1. I currently have all of my websites hosted on Server #1. (We'll call it that for the sake of avoiding confusion.)
2. I have an automatic nightly backup setup via cPanel / WHM that backs up all accounts from Server #1 to Server #2 via FTP. (Server #2 is in a totally different data center, with a different provider.)
3. The nightly backup packages all of the accounts as "cPanel Full Backups." So, they're compressed, and as such, they don't work as "live, functioning websites" on Server #2. The only way to make them "live and functional" on Server #2 would be to use cPanel to "restore" the backups.
4. So, what I'd like to do is setup a CRON job that would automatically "Restore" the backups each morning on Server #2. That way, Server #2 would always have a functional version of all my accounts, that is less than a day old. Then, if Server #1 ever crashed, I'd just have to change DNS information to point to Server #2, and all of the websites would be live again, without having to physically restore all of the backups using cPanel.
I don't know a ton about CRON. However, as I understand it, CRON couldn't actually make cPanel restore the backups. However, I'm assuming that when you use cPanel's "Restore" function, it just goes through a series of processes. So, it seems logical to me that, if you knew what those processes were, you could write a CRON job to automate the process every morning.
Did that make sense?
If so, is it possible?
Do you guys have any input, criticism, etc?
If it's doable, can you make any suggestions that would help me make this happen?
Finally, if you think you have the expertise to make this happen, I'd be interested in chatting with you via Private Message. I'd be willing to pay a reasonable sum for some help with this.
Today we are going to conduct a detailed study of RAIDability of contemporary 400GB hard drives on a new level. We will take two "professional" drives from Seagate and Western Digital and four ordinary "desktop" drives for our investigation. The detailed performance analysis and some useful hints on building RAID arrays are in our new detailed article.
Is Motherboard RAID as good as a dedicated PCI-E card? I am guessing a dedicated card is the best option, though costs more.
We are looking at buying a barebones server from Supermicro. It features an onboard RAID controller which supports RAID 0, 1, 5 & 10 - but for some strange reason it will only support RAID 5 if you use Windows. Here is a link to the page detailing the RAID features.
[url]
We are going to be running Linux, CentOS 5.1, so we will only have the choice of RAID 0, 1 or 10. This isn't an issue, as having RAID 10 on 4x SAS (15k) drives will be fine for speed and stability. What is an issue is would this RAID controller be as fast or reliable compared to a dedicated PCI-E card? If it can only use RAID 5 in windows, does that suggest this controller is too reliant on software? It would be a nightmare to suffer downtime and data loss because the controller couldn't hack it during a drive failure, or one day it decided to bugger up the array when rebooting.
So that leads me to looking at this card, this looks very good for what we need. Are adaptec a reliable brand? I've seen it advertised for £200, which is a good price.
[url]
This card features RAID 5 and 6, would RAID 6 be better than RAID 10 for redundancy, or is it too slow to bother with? Also it seems to have a battery module available for it, what does this achieve? Cos surely if the power dies the hard drives and motherboard can't run off this little battery, or does it just help the controller stay alive long enough with some hard drive information in its memory if the power goes out during a rebuild?
I am in a somewhat complicated situation... I wanted to order a custom server with hardware 3Ware RAID controller but after over a month of waiting I was told the HW RAID controller, as well as any other 3Ware controller they tried, does not work with the motherboard used in the server from Fujitsu-Siemens and that they simply got a reply from FS that the controller is not certified to work with their motherboard.
So although I'd prefer a HW raid, I am forced to either choose a different webhost or setup a software RAID. The problem is, I haven't done that before and am somewhat moderately...scared
I have read a lot of the info about SW RAID on Linux that I could find through Google but there are some questions unanswered still. So I thought that perhaps some of the more knowledgeable WHT members could help me with this problem...
The server specs will be:
Core2Duo E6600 (2.4Ghz), 2GB RAM, 6-8x* 250GB SATA II HDDs, CentOS 4.4 or SuSe, DirectAdmin
* I prefer 8 HDDs (or actually 9) over 6 but I am not sure if their server chassis can hold that many HDDs, I am awaiting answer from them. They don't have any other drives beside the 250GB ones so I am limited to those.
The preferred SW RAID setup is to have everything in RAID 10, except for the /boot partition which has to be on RAID-1 or no RAID I believe, plus one drive as hot spare (that would be the 9th drive). I am quite sure they will not do the setup for me but will give me access to KVM over IP and a Linux image preinstalled on the first HDD so that I'll have a functional system that needs to be upgraded to RAID-10.
How do I do that? The big problem I see is that LILO or GRUB can't boot from a software RAID-5/10 so I will have to mount the /boot partition elsewhere. It's probably terribly simple...if you have done it before which I have not. I have read some articles on how to setup a RAID-5/10 with mdadm (e.g. [url] ) but they usually do not talk about how to setup the boot partition. Should it be setup as a small sized (100-200MB) RAID-1 partition spread over all of the drives in the otherwise RAID-10 array?
What about swap? Should I create a 4-8GB (I plan to upgrade the server RAM to 4GB in near future) RAID-1 swap partition on each of the disks or swap to a file on the main RAID-10 partitions. The second sounds simpler but what about performance? Is swapping to a file on RAID-10 array a bad idea, performance wise?
Is it possible to grow a RAID-10 array in a way similar to growing a RAID-5 array with mdadm (using two extra drives instead of one of course)? mdadm doesn't actually even mention RAID-10 despite it does support it without having to create RAID-0 on top of RAID-1 pairs if the support is in kernel, from what I know.
How often do RAID arrays break? Is it worth having RAID if a servers hard drive goes down? I was thinking it may just be a better option to just have a backup drive mounted to my system and in the even of a system failure just pop in a new hard drive, reload the OS, and then reload all my backups?
I am in the process of restructuring the infrastructure on our servers. I am thinking of using either RAID 5 (1 hot spare) vs RAID 10 as my 1U server has 4 HDD tray.
RAID 5 would have better capacity but RAID 10 has better overall performance. Which one do you guys go for a shared hosting server?
Is it possible to turn a non raided setup into Linux software raid, while it is live, and if it's the OS drive? Can you even software raid the OS drive remotely? I've been thinking about doing it for the redundancy (and possible slight performance boost for reads, but doing it more for redundancy). I'm using CentOS.
I've been talking to the Planet about trading in my four and a half year old "SuperCeleron" (from the old ServerMatrix days) Celeron 2.4 GHz system for something new. As part of their current promotions, I've configured a system that looks decent:
Xeon 3040, 1 gig of RAM, 2x250GB hard disks, RHEL 5, cPanel+Fantastico, and 10 ips for $162.
Not too bad. I could bump up the ram to 2 gb for, I think, $12 more, which I'm thinking about and wouldn't mind some thoughts on. But, the thing that has me really confused is RAID. I like the idea of doing a RAID 1 setup with those two hard disks. But, the Planet wants $40/month for a RAID controller to do it. I really don't want to go over $200 a month!
Any thoughts on alternative redundancy strategies that might avoid that cost? Software RAID does not seem to be offered by the Planet, unless I can figure out how to do it after installation (is that possible?) Better ideas in general on the server?
Are there any significant difference between 4 15K SAS HD in RAID 10 versus 8 7.2K SATAII HD in RAID 10? I have the same question for 2 15K SAS HD in RAID 1 versus 4 7.2K SATAII HD in RAID 10.
I have room for 4 more hard drives on my home server. My original goal was to go raid 10 but I've been thinking, raid 5 can support 4 drives and give more capacity. Which one would have better performance as software (md) raid? I'm thinking raid 10 might actually have bad performance as software raid, vs hardware, compared to raid 5. Would raid 5 with 4 drives be better for my case?
We are looking to build our first server, and collocate it. It will be a higher investment than just renting the server, but will be worth it in the long term, and we have already decided we are going to support the hosting business for a minimum of 3 years - so we might as well invest in a server from the outset to benefit from lower data center charges and higher redundancy and performance.
We are currently looking at Supermicro for servers as they offer 1U barebones systems with dual hotswappable psus and upto 4 hotswappable drives. This would be ideal for redundancy, and also for taking advantage of the speed and redundancy that a RAID 10 array would give you. These two factors combined are very appealing as it would reduce the possibilities of downtime and data loss. Obviously we will be backing up daily, but its good for piece of mind to know that you could potentially blow a PSU and 2 hard drives, and your server will still be up long enough for a data centre technician to replace the parts.
Now then, my business partner and I are currently deciding what the best all round hard drive configuration would be. He has decided that we should opt for SAS instead of SATA to have lower latency seek times, which would give us better performance. I agree, though this does increase costs considerably.
He is then arguing that we use RAID 5 on cost grounds. He says we should only use 3 of the slots to begin with, save money on one drive by not having a spare, and hope we don't have a drive failure - which sods law will happen. I'm not happy us cutting corners to save money, because if we gamble and lose, that's a hell of a mess we have ourselves in, and will cost us a load more time, reputation and data center charges to get ourselves out of it.
I say we might as well go for RAID 10 for that extra performance, and redundancy, you can potentially lose 2 drives so long as they aren't from the same mirrored pair. With RAID 5 you can only lose a drive, it takes longer to rebuild onto a spare, and during rebuild the performance takes a hit. Also RAID 10 is much faster than RAID 5, and at the expense of the cost of a drive.
Now the question we should be asking is... would a SATA2 RAID 10 array provide better performance than a SAS RAID 5 array?
So I think the choice we have to make is either go for RAID 5 and run with a hot spare, and stock a cold spare, or go with RAID 10 and stock 2 cold spares.
We are considering going with Seagate drives because they are high performance and have 5 year warranties. I have had to RMA two Western Digital drives already in the past 12 months, a raptor and a mybook, both deaths invoked data loss.
The server is going to be a linux web, email, dns and mysql box. It will likely feature a single dual/quad core processor, and 4-8GB of unbuffered ddr2 ram.
Question though on RAID choices... I'm considering getting 3 x 250GB SATA drives. Would it be better to make two of them a RAID-1 mirrored pair for my OS, home directories, and use the 3rd drive seperately for backups, swap, and perhaps some logs.... OR should I put all three drives into a RAID-5 set and treat it as a single logical drive?
my math says usable space would actually be identical... with 465GB usable in either setup. RAID-1 would be faster for I/O with no parity overhead... but one drive would not be redundant. On the other hand, RAID-5 would be fully redundant but have parity overhead for writes.
Our website is receiving a daily attack from a french network called Neuf Cegetel. The IP is different each day but the network is always the same. The attack is daily and during several hours.
The website does not use ajax (the request is an ajax request) and there is no URL /0_0?_=... But the attacker use a random URL similar to this /0_0?_=1238873869634. Since the URL is always different the page is not cached so it is compressed by mod_deflate and therefore the attack is more harmful. The User-Agent and the cookies changes quite a lot but it is always an ajax request. Taking in account that it is the only ajax request in the server that would be the easily way to stop it. But it seems that when we try to stop the attack, the attacker try another way, what makes me think that the attack is voluntary (not a virus nor something like that).
Since it seems that the attacker can be easily found it (we are a Spanish website and the attacker comes always from the same French network), should we report this? If it were a virus in a remote server, the solution maybe is just to contact the abuse department of the network but if it is voluntary I think that we should discover who is behind the attack since it might be a company that want to bother us, a competitor or something like that. What do you think?
This is a very small copy of the logs containing a few examples:
Code: 4087 ReqStart c XX.XXX.42.189 52592 517548693 4087 RxRequest c GET 4087 RxURL c /0_0?_=1238873869634 4087 RxProtocol c HTTP/1.1 4087 RxHeader c x-requested-with: XMLHttpRequest 4087 RxHeader c Accept-Language: fr 4087 RxHeader c Referer: http://thewebsite.com/ 4087 RxHeader c Accept: application/xml, text/xml, */* 4087 RxHeader c x-requested-handler: ajax 4087 RxHeader c UA-CPU: x86 4087 RxHeader c Accept-Encoding: gzip, deflate 4087 RxHeader c User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30618; FDM; OfficeLiveConnector.1.3; OfficeLivePatch.0.0) 4087 RxHeader c Host: thewebsite.com 4087 RxHeader c Connection: Keep-Alive 4087 RxHeader c Cookie: __utma=9819446.1354119376.1238785835.1238785835.1238865537.2; __utmz=9819446.1238865537.2.2.utmccn=(organic)|utmcsr=msn|utmctr=thewebsite|utmcmd=organic; __utmc=9819446; /= 4087 VCL_call c recv lookup 4087 VCL_call c hash hash 4087 VCL_call c miss fetch 4087 Backend c 3052 default default 4087 ObjProtocol c HTTP/1.1 4087 ObjStatus c 404 4087 ObjResponse c Not Found 4087 ObjHeader c Date: Sat, 04 Apr 2009 19:37:47 GMT 4087 ObjHeader c Server: Apache/2.2.3 (CentOS) 4087 ObjHeader c Vary: Accept-Encoding 4087 ObjHeader c Content-Encoding: gzip 4087 ObjHeader c Content-Type: text/html; charset=iso-8859-1 4087 TTL c 517548693 RFC 120 1238873867 0 0 0 0 4087 VCL_call c fetch 4087 TTL c 517548693 VCL 3600 1238873868 4087 VCL_return c deliver 4087 Length c 235 4087 VCL_call c deliver deliver 4087 TxProtocol c HTTP/1.1 4087 TxStatus c 404 4087 TxResponse c Not Found 4087 TxHeader c Server: Apache/2.2.3 (CentOS) 4087 TxHeader c Vary: Accept-Encoding 4087 TxHeader c Content-Encoding: gzip 4087 TxHeader c Content-Type: text/html; charset=iso-8859-1 4087 TxHeader c Content-Length: 235 4087 TxHeader c cache-control: max-age = 300 4087 TxHeader c Date: Sat, 04 Apr 2009 19:37:47 GMT 4087 TxHeader c X-Varnish: 517548693 4087 TxHeader c Via: 1.1 varnish 4087 TxHeader c Connection: keep-alive 4087 TxHeader c age: 0 4087 ReqEnd c 517548693 1238873867.757586718 1238873867.758437872 0.936849117 0.000804424 0.000046730
customer of HostV's VPS hosting, and for the past 3 days, at almost exactly 01:20 GMT, CPU load jumps from an average of about 0.10 to 2.5+, stays there for over an hour, then drops back down.
During this time, there are NO processes on my virtual server using any significant amount of CPU time, memory, or IO. No cron jobs are running on my server, etc.
Note the output from 'uptime' below (I was monitoring it waiting for the problem to occur, which it did at exactly the time I expected):
00:32:05 up 22:01, 2 users, load average: 0.09, 0.11, 0.08 00:32:07 up 22:01, 2 users, load average: 0.08, 0.11, 0.08 01:09:49 up 22:39, 2 users, load average: 0.06, 0.03, 0.00 01:10:03 up 22:39, 2 users, load average: 0.05, 0.03, 0.00 01:19:26 up 22:48, 2 users, load average: 0.46, 0.16, 0.04 01:20:42 up 22:50, 2 users, load average: 1.53, 0.55, 0.18 01:21:39 up 22:51, 2 users, load average: 1.40, 0.67, 0.24 01:46:04 up 23:15, 2 users, load average: 3.06, 2.02, 1.52
Also note output from 'top', taken when load average was at 3.06 shown on the last line above:
My cpu usage is very low (0.1%) but wait time is at 9.0%, and I've seen this go as high as 70% during these times.
So, basically, there is a problem that exists on the host node somewhere that is causing my site to become effectively unresponsive (page load 20 seconds+ - measured), and it happens every single day at the same time.
So, why am I posting it here instead of logging a trouble ticket? I have logged a trouble ticket, but when I encountered the problem yesterday, despite logging it as "CRITICAL", I had to wait nearly 5 hours for a response, which effectively said not much beyond "we noticed the problem and fixed it and we're monitoring it". So I don't have a lot of faith that today's response will be any better.
I moved to HostV because of similar problems I was encountering with shared hosting, and was assured before signing up that the kind of problem I'm seeing doesn't happen. So now I'm outlaying more than 10 times the cost for almost exactly the same problems and a similarly unhelpful response to it.
By publicly posting the problem, I would hope that someone at HostV will ensure the problem is addressed PROPERLY, rather than bandaided again, and that hopefully we will all be able to see just how good HostV's support CAN be (as evidenced in another similar post).
I await HostV/Cirtex's response.
As shown in the uptime information about, server uptime is 23:15, because I rebooted the virtual server yesterday to see if that helped. It didn't. In fact, it took over 20 minutes for the server to come back up, which is why I'm not going to do it again.
