We've come to a crossroad with our site and not sure where to take it from here. It would seem like a cluster option would be the best bet, but wanted to ask the professionals first to see if there were any other way to solve it.
The problem is (as usual) too high load making the servers slow. Actually, only the apache server. We recently seperated mySQL and apache on two servers to help out with the load. The mySQL server load is around 0.25-0.5 throughout the day, but the apache server load is almost always at 6.xx-10.xx.
Both servers has the following stats:
CPU: Intel Core2Quad Q6600-S3 (Quad Core @ 2.4GHz)
RAM: 8GB DDR2
HDD: 750gb SATA2
* Shouldn't an apache server with these stats be capable to handle the current visits?
* Could this perhaps be solved if we swapped the sata drive with a scsi/sas?
If we're moving to a cluster solution, we'd be looking at minimum 5 servers.
2x Load balancing servers
1x SQL server
2x Apache servers
Preferably we should have 2 SQL servers for redudancy, but for a site that is hobby based, it would be insane to pay around $1245-1490 a month just to keep it up.
I would like to know if there's any way we could reduce "conversion times" for videos while converting them with "mencoder" and "x264 codec"?
Is it possible to cluster 2-3 or more servers (Quad core/8GB RAM) so that I can reduce the conversion timings effectively?
The original video sizes are around 500-700MB on an average and I'd like to convert them using mencoder in x264 codec @ 500Kbps bitrate and 2pass settings.
Ofcourse it'll take atleast 1-2hours to encode 1 such video at these settings on a 'single' such server, so is there a way to reduce the conversion times to around 10-15minutes/video by using "parallel encoding" with x264 codec?P.S: I know how to form a cluster, using beowulf/rocks, etc...what I need help with is 'using' the cluster with x264 codec
I can't get access to a certain site. I always get the page with:
network time out - server at *** takes to long to respons. More people have noticed this and apparently it only happens to people with certain specific providers. And not all the time. Some times they DO get access eventy to they belong to the same ISP. So I guess an ISP isn't blocking access to it otherwise it would be permenantly/The site administrator insists that certain ISP's are blocking his site. He's hosting it on his own server. The domain belongs is registered at namecheap.com.
If an ISP is blocking this site (if that's possible?), that would lead to that 'network timeout' page wouldn't it?
What is the most likely reason for getting a timeout page anyway?
I have a dedicated server specs: AMD 3500+ 64 Bit CPU, 1 GB Ram, 160 GB Sata Drive. For 1 month, CPU load average reaches 40-50 value. This happens about 5-6 times in a day. When I stop httpd service for 30 seconds everything goes normal. I think this is not a DoS attack because it comes systematic, I dont believe no one makes this regularly except bots.
Maybe its a system service or a cronjob but it stops when I turn off httpd service? How can I be sure about what's making this regularly load?
I also did set up a script which mail me when load average of system goes crazy and restart httpd service. But instant restart is not working to stop load increase.
The server is going down from time to time, every 12 days or so the site hosted there is no longer accesible, everything starts with the site slowing don and down and then is not longer reachable, what we do is to request a power cycle, and with this we start all over again till next power cycle, so on so on, of course, here are my server details and more info on this:
- MySQL - 5.1.41-3ubuntu12.10 - Apache - 2.2.14-5ubuntu8.4 - PHP - 5.3.2-1ubuntu4.9 - operating system: Ubuntu Server 10.04 LTS
After some time emailing the support guys to barely check about what's going on, we received an email with a few things:
1.- found a few errors that likely would cause issues with Apache. The first error is: [Mon Feb 04 05:03:10 2013] [error] mod_fcgid: fcgid process manager died, restarting the server and the next error is: [Mon Feb 04 14:32:34 2013] [error] server reached MaxClients setting, consider raising the MaxClients setting ...
Both these errors seem to indicate that you have a process that is running out of control on your server. We were unable to determine what script on your site is running caused your connections to be maxed out however it does appear that before these errors were generated there was a WordPress plugin referenced in your access logs...
2.- Additionally during our review we did find that your error log for mercadodedinerousa.com is 45 GB's which is excessively large and can cause problems when Apache is trying to write a such a large file.
3.- The majority of the errors being logged are: [Wed Feb 06 12:12:31 2013] [error] [client 200.76.90.5] Options FollowSymLinks or SymLinksIfOwnerMatch is off which implies that RewriteRule directive is forbidden: /var/www/vhosts/mercadodedinerousa.com/httpdocs/index.pl, referer: [URL]
Anyone had any experience with setting up an ftp cluster? We have tried to search Google and haven't been successful in such a solution so I came here to ask if anyone else has ever thought about doing it. Or done it and how did you do it? I'm looking in to it for windows. 99% of my servers are windows.
I see more and more hosts popping up (and older hosts switching to) offering what they call clustered load balanced plans. These plans are mostly on the low end price wise (sub $10/month), and the marketing speak tells of zero downtime, intelligent clustered, load balanced, failover, will wash your clothes approaches to keeping websites up 110% of the time.
I guess what I fail to grasp is are these plans really, truly load balanced clustered approaches, or is it more of a case of we have 1 DNS server, 1 mail server, 1 web server, 1 mysql server, and a backup server that in an emergency has rsyncs each hour to "failover" to and screw up your dynamic data?
If the webservers are load balanced, what about accounts that use SQLite db's, or berkeley db's, are these maintained 100% in synch, and if so, how? If one server goes down, and the other picks up the slack, won't the db's be out of synch and data contention will occur? Are sessions including SSL truly handled correctly?
Same for MySQL. Is this true two way replication with fault tolerant switchovers between master and slave to preclude data loss and corruption on hot switch overs when a server fails?
I guess I just see everyone slapping a "clustered" label on things, and speaking of load balancers, etc... all for the cup of coffee a month. But I am highly suspect if these plans are truly load balanced, or is it a band-aid approach to load balancing?
Just the ranting of a an older hosting owner who's just not sure what many companies are truly offering when they speak of clustered, load balanced, more uptime than Viagra type of solutions...
We have been running two dedicated servers for a number of years now and we use Cpanel and DirectAdmin for the control panels. Today we had a big problem with our first server which effectively made the 150 sites on it out of action and it exposed a potential issue to us that if the server died that we'd have a big task restoring all the sites from the backup drive. I've heard that some hosting companies strip data across more than one machine and I presume this is a cluster, and I was wondering if this is the best method and what the most cost effective route would be to do it?
I was wondering if there is any way to connect 40-50 Linux VPSes in a cluster. Most of the cluster solutions out there want you to recompile a custom kernel which cannot be done on a vps. The virtual machines are on different hardware nodes and I don't have access to those since I am renting the VPSes from another company.
On setting up a pair of CPanel DNS only servers, I decided to use 2 Virtual Servers from 2 seperate companies - one Xen and the other was Openvz. Reason behind it was in case something happens to the company or the virtualization technology implemented, I'm not screwed.
Guess what! Next day my Openvz node (I will not name the provider) went south and I'm on one leg right now. I'm kinda pleased with myself for having this laid out from the beginning and DNS is still working fine.
Now I'm thinking of having different OS - one linux and the other freebsd. Am I paranoid and over-the-top?
we are planing to build a 5 system mysql cluster for our website. i have few questions regarding this. As per our provider, we need mod_ndb for apache to connect database cluster. (SQL API). we have 5 system apache cluster.
Mod_ndb configuration, we need to add all table structure in http configuration. Is there any other way to connect database? or any other way to configure mod_ndb without adding table structure inside http configuration.
All my sites that depend on MySQL at iMountain have been down all day. I was in the process of converting some of them to Joomla/Drupal but I'm having second thoughts now.
My question is - shouldn't "clustering" prevent this sort of thing happening? I thought it implied some level of redundancy.
I try to keep on the up and up about hosting and constantly looking for a new host to provide better services, but I saw an article in TheWHIR that got me to wonder if people are making stuff up or is it just "marketing" or real?
Ok, so the article:
[url]
I hope adding a link to the article is ok?
First of all I have NEVER heard of cPanel ever offering clustered technology. Can anyone confirm or deny that this is real? I have really only used HSphere which I know has some cluster abilities and even those hosts don't advertise "clustered" services.
Then I see other so called cPanel wannabe clustered services which they call cPanel clustered failover or something like that which to me is ummm..more fault tolerant hosting than actual cluster hosting.
Basically, am I just not understanding the meaning of "cluster"?
I think cluster has to do with the proper allocation of reasources spread across the network of defined processes, though, I have been wrong in the past...
i have a site which involves heavy cpu use but its in a small private network with 3 other boxes which are pretty much idle, so im wondering is their a way to use the idle cpu time /ram possibly on my main server via the network ?
I have a total DB in size of 800MB... Recently I had to make a few configurations in the management tool and had to do an intialize restart... Not sure what's taking so long, but it's been nearly 8 hours and I have not seen any of my tables that used to be there.
When I do du -sh /var/lib/mysql-cluster, it shows that I have 800M /var/lib/mysql-cluster...
And on cluster02 and cluster03, I did something like that: - srv01 (synchronize changes) - cluster01 (synchronize changes) - cluster03 (standalone) And - srv01 (synchronize changes) - cluster01 (synchronize changes) - cluster02 (standalone)
1. So, is that correct?
2. It seems that when I click on 'syncronize all DNS records on all the servers' only cluster01 and srv01 got all the DNS records. So is it normal?
Yet If I add a domain on cluster02 for example, where I need to add the DNS? cluster02 named.conf or on cluster01/srv01?
I'm planning to upgrade my servers to a more "enterprise-class" installation and would like to hear your advice for good managed hosting providers for this.
Currently I use RackSpace which are great but pricey (leased servers); in the past we had a colocation with another provider: it was good cost-wise and generally proffesional but alas the provider's infrastructure was not sufficiently resilient to natural disasters and unlikely but occurring infrastructure accidents.
My feeling is to stay with a more managed service rather than try to colocate again: we need someone making sure the hardware stays around 24/7 and makes sure to have the appropriate spare parts and expertise in the hardware they sell us. We can handle the software parts.
I'll probably stay with RackSpace because they can and have already built stuff like the below. But maybe there's another provider out there which fullfills similar needs for customers.
My needs are: 1.5 TB of shared storage (i.e. SAN or NAS; must be reasonably fast (maybe 10MB/s sustained) and 99.99% up); add .5TB per year 4 application servers in various configuration (8-16G of RAM, quad or dual-quad Xeons) 2 database servers (With fast drives in RAID-10 but not much disk; most of our data is not in an SQL server) Some appropriate load balancing appliance (with at least 99.99% reliability) Red Hat Enterprise Linux 5 Redundancy to a point (i.e. redundant mechnical parts, hot-swap drives) Usual stuff: managed backup, private network, hardware monitoring with fast hardware exchange No Linux system administration beyond a sane installation needed
99.99% uptime on the network and infrastructure (i.e. if an UPS blows up or a router goes down this should not affect me; redundant connections to multiple network providers etc. etc). RackSpace has a 100% network uptime guarantee and in the many years we've been with them in one form of another I haven't been disappointed in their competency on that front.
This would be in some US datacenter. Does someone have experience with such a scale installations with other providers (or RackSpace too) ?
(I wrote "SAN or NAS" because I'm not really yet sure if some NFS server or maybe shared storage via SAN and something like RedHat's GFS will be best for my needs; it would be nice if the provider was capable of advicing on things like that).
I have started to move my websites over to my UK Colocation But would like a little guidence on what the best solution would be.
I would like to build up a Server Cluster that will handle all of my sites really and also have redundancy, so If 1 server goes down the other Web/SQL/Whatever server will be used instead.
I have websites ranging from large forums to streaming & download websites.
Should I got for a setup for example:
X Web Servers + X Database Servers Connected to X Storage (see link below)
Connecting the Web Servers in something like a a round robin config or use a Load Balancer / Other
OR Should I setup multiple: Web Server + Database Server + Media Server
OR another config?
Below are my current setups
Current UK Setup (Colocation)
Web Server Quad Core, 8GB Ram, 250gb HDD Raid 1 Quad Core, 8GB Ram, 250gb HDD Raid 1 (just ordered)
SQL Server Quad Core, 4GB Ram, 250gb HDD Raid 10
Storage Server HP StorageWorks NAS 1200s 1TB (just Ordered) Link: [url]
Current US Setup (Dedicated Servers)
Web Server Quad Core x2, 8GB Ram, 3TB HDD
SQL Server Quad Core x2, 8GB Ram, 1TB HDD
Media Server Quad Core x2, 8GB Ram, 3TB HDD Dual Core, 4GB Ram, 3TB HDD
i am going to use with hsphere preferently i want to use Dell blade server 1955 with exchange and sharepoint, cause the low HD capacity i will like to add and HD array could be NAS or SAN,
This is clearly the next logical step in hosting. Consider what RAID1 did for hard drive availability. Worried about your disk failing? No problem - throw in a duplicate. From a user/admin perspective the mirrored disks appear as one, highly reliable disk. Every modern OS supports this, even cheap motherboards support it, with a minimal amount of configuration.
The perfect high availability cluster should be completely transparent. Think about it... What if clustered hardware allowed you to manage a single instance of your chosen server OS with no worries about hardware/network/power failure?
The closest you can get to this finding a host that will give you a dual power supply server connected to different power feeds, dual network cards connected to different switches, and hot-swappable mirrored hard drives. This solution still doesn't give you any protection from CPU, memory, or motherboard failure.
I want a solution that is as simple to configure and manage as simple RAID1 mirroring.
put together a high availability cluster for a PHP + MySQL based app to run on a LAN. We're going to use Linux, and cost is a major consern. The app itself doesn't use / need too much resources, as it will only be accessed by 2 / 3 people at a time, so I'm using the following:
2 identical PC's with: 3Ghz PIV CPU 1GB RAM 2x SATAII 160GB HDD space setup as RAID 1 10/100 Mbps LAN NIC's, on a 100MB 8 port switch
Up to now I have been running MySQL-Max 5.0.15 to run a MySQL master-master replication server, which works fine, but the setup involved a lot of manual work, and downloading of the right binaries.
Furthermore I used Linux Heartbeat todo auto switchover between the two servers & RSYNC to sync the application files between the two. This has been working fine untill one of the server's HDD's failed recently corrupting both HDD's
So, I need a better way of doing this, and want to meet the following requirements: If 1 HDD fails on either server, the server still needs to be able to run without a problem. Replacing a HDD & rebuilding the RAID array should be easy to manage, preferabbly over the net.
Setting up a cluster should be easy to manage, both for the MySQL DB server & the files that need to be synced between the two machines Re-installing the server should be easy todo as well.
For No.1 I have been thinking of setting up RAID 5 with 4x HDD's - how reliable / safe / redundant is this?
For No. 4 I have been thinking of using something like sysimager to backup the server once setup, but will / can it recreate the RAID array upon restoration? The MySQL DB & PHP files are being backed up to a removable HDD on a daily basis.
The client is 700Km's away, so we can't just drop-in to fix things as often as we like. Thus redundancy is of utter importance. Currently I'm running Suse 9.3, simply due to the fact that it's easy enough to tell the client over the phone howto do things with YaST. Suse 10.1 will be used for the new setup, but I could also use Fedora Core 5, and have also been thinking of using SME server 7.0.
