Recommendations For High-performance Hosting (clustering)
Jun 14, 2008
I'd like to get your recommendations on how I should approach this problem. I posted this in another part of the forum, but I feel that this is a more appropriate place for it.
Problem:
How to handle large amounts of traffic with for a social network website? If a user uploads a photo or video, how does it become accessible on all of the server? If traffic is expected to be about 500,000 visitors a day, how many machines do you think I should use?
Possible Solution:
I've come up with the following possible infrastructure.
One load balancer. The load balancer has 3 PHP/Apache servers behind it. Behind each of the PHP/Apache servers is a (slave) MySQL server, from which data is read. Behind the slave MySQL servers, there is 1 master MySQL server, which handles all of the database writes. The master MySQL and slave MySQL servers are synced up, so data is up to date.
The actual photo and video files are not stored in the database, only the links to them is stored in the database (to keep the database small). The photo and video reside in a central location (like a SAN or NAS), which is accessible by all of the 3 PHP/Apache webservers.
Questions:
1. How many machines do you think will be able to handle photo and video uploads for 500,000 visitors a day?
2. Is having a SAN with Terabytes of RAIDED disk space an available option?
3. If a SAN or NAS is not an option, does anyone have any ideas on how to make sure all of the web servers have access to the same photos and videos? Is rsync a viable solution?
4. Which hosting provider do you think I should go with?
5. Is clustering what I need? What is clustering and how will it address my concerns?
View 1 Replies
ADVERTISEMENT
Jun 14, 2008
I'd like to get your recommendations on how I should approach this problem.
Problem:
How to handle large amounts of traffic with for a social network website? If a user uploads a photo or video, how does it become accessible on all of the server? If traffic is expected to be about 500,000 visitors a day, how many machines do you think I should use?
Possible Solution:
I've come up with the following possible infrastructure.
One load balancer. The load balancer has 3 PHP/Apache servers behind it. Behind each of the PHP/Apache servers is a (slave) MySQL server, from which data is read. Behind the slave MySQL servers, there is 1 master MySQL server, which handles all of the database writes. The master MySQL and slave MySQL servers are synced up, so data is up to date.
The actual photo and video files are not stored in the database, only the links to them is stored in the database (to keep the database small). The photo and video reside in a central location (like a SAN or NAS), which is accessible by all of the 3 PHP/Apache webservers.
Questions:
1. How many machines do you think will be able to handle photo and video uploads for 500,000 visitors a day?
2. Is having a SAN with Terabytes of RAIDED disk space an available option?
3. If a SAN or NAS is not an option, does anyone have any ideas on how to make sure all of the web servers have access to the same photos and videos? Is rsync a viable solution?
4. Which hosting provider do you think I should go with?
5. Is clustering what I need? What is clustering and how will it address my concerns?
View 11 Replies
View Related
Aug 12, 2008
It should support jsp/jdk6.0/tomcat6.0/mysql5.0 or higher, and SSH is prefered.
Your advice will be highly appreciated.
View 0 Replies
View Related
Mar 25, 2009
We are considering a new dedicated server host for a set of 25 domains, about 5 of which are very high traffic (80 million clicks a day each).
I am told that the biggest speed boost and performance comes from memory and fast hard disk. So I'm looking for at least a 16GB RAM and SCSI 10k 300GB hard disks.
I am also told that PHP etc is okay, but MySQL is the one that hogs resources after a while. So for the database server I need a high end server.
I like WHM/Cpanel so that should be alright.
A lot of email alerts may be sent through our system (about 500,000 a day on a good day as there's user subscription to updates functionality) so we may need a separate mail server?
My question: is the following config of 4 x quadcore Dunnington Intels be good enough for the above site? Can I run a database on this config for my kind of traffic or do I need a separate server too?
Code:
Quad Processor Hex Core Intel 7450 - 2.40GHz (Dunnington) - 6 x 9MB (L2) 12MB (L3) cache
Second Processor
Hex Core Intel 7450 - 2.40GHz (Dunnington) - 6 x 9MB (L2) 12MB (L3)
Third Processor
Hex Core Intel 7450 - 2.40GHz (Dunnington) - 6 x 9MB (L2) 12MB (L3) cache
Fourth Processor
Hex Core Intel 7450 - 2.40GHz (Dunnington) - 6 x 9MB (L2) 12MB (L3) cache
16 GB FB-DIMM Registered 533/667
1000 Mbps public uplink
1000 Mbps private uplink
Disk Controller RAID 10
HD1: 300GB SA-SCSI 10K RPM
HD2: 300GB SA-SCSI 10K RPM
HD3: 300GB SA-SCSI 10K RPM
HD4: 300GB SA-SCSI 10K RPM
CentOS 5 (32 bit)
View 7 Replies
View Related
Jul 17, 2009
I don't know where to start. I need advice about how to setup a high performance firewall for data center. What big companies use? I suppose that they use Cisco or Juniper firewalls. But I have specific needs. I want to control that firewall with php program - to pass IPs witch can pass the firewall. I think that I have to store them in .txt file. So the firewall must read that file.
In my case I think that I need iptables + IPset or maybe nf-HiPAC? Or transparent proxy like squid or Haproxy?
I'm very confused about the software? The firewall must forward 10GBit of traffic and 60 000+ connections
View 14 Replies
View Related
Mar 11, 2009
I am currently hosting my website on one server with the specs:
2.8ghz Dual Quad-Core processor + 8 gigs of ram + two 500 hard drives with a 50 mbps unmetered bandwidth package.
My current problem lies in high server loads and very slow server performance throughout the day.
I am considering migrating over to The Planet onto server with the specs:
3.0ghz Dual Quad-Core + 18 gigs of ram + two 50gb hard drives with 2TB of monthly bandwidth transfer.
In an attempt to have great bandwidth pricing and server performance, I plan on downgrading my current server with my current host to a lowe-end server and keeping it only to host my VIDEO and MUSIC files with the 50mbps unmetered package. The Planet will then host my database and all other web related files on their new server.
Is this a good idea as an attempt to save money in bandwidth costs and eliminating my server lag issues?
I was offered a setup of a separate web and database server at my current host but from what I have read, no one touches the performance and reliability The Planet has to offer.
View 6 Replies
View Related
Aug 12, 2008
Can someone recommend a good, fast set up Windows (ASP.NET 2.0) host in Netherlands or Germany or France? Shared hosting with SQL Server 2005.
No UK please. I would prefer someone using the recommended host and has positive feed back.
View 14 Replies
View Related
Dec 31, 2008
For the past 3 weeks I have been trying to set up a joomla web site.
At first, I tried to host a joomla website on liquid web's shared host.
They seemed to be quite good. They had all the feature I needed, but then I couldn't upload files into my webspace via ftp.
They told me they don't know what is the problem, and that I should use what is available.
I have stopped hosting at liquid web, and tried a different web host.
HostNexus.
There I had the same problem with myu ftp account, but they told me the problem is either in my firewall or ISP.
This really allowed me to look into a solution for this problem, and I partially solved it.
I believe this was the same problem I had with liquid web, but liquid web support didn't tell me the information that helped me to solve this problem.
Though my problems are not solved yet.
I need to create a ftp account into the httpdocs or public html folder so joomla could use it to update its own files.
I was able to create such ftp accounts in liquid web, but it is possible that I am unable to create such ftp account via the hostnexus control panel, and I have asked them if its possible for them to create me such an account.(Asked them topday, so I didn't get a reply yet).
Now, what should I do?
I need a ftp account in order for joomla to work properly.
Should I go back to liquid web?
Should I look for a different web host?
Maybe HostNexus will solve my problem, and all this will be irelavent.
But assuming host nexus won't solve my problem, what should I do?
View 14 Replies
View Related
Aug 31, 2006
My boss has recently asked me to source a new host for our servers. At the moment we're running 8 Windows boxes but are unhappy with the service we're receiving from our hosting company. We are looking for a managed hosting company, that has offices in the UK and US to look after the servers.
At the moment we haven't nailed down our exact requirements so I'm just really looking for any recommendations you guys might have. I've done a wee bit of Googling and have found a company called Rackspace but I'm ideally looking to contact around 5 companies to see who can offer us the best deal.
View 14 Replies
View Related
Feb 12, 2009
I am planing on running I guess you would call it a semi tube script. It pulls videos from others sites and thumbs them at your own site in categories. The viewers however do view the videos at my site in a popup video player or another page.
Hostgator looks like a good host however I just talked to them and they don't have ffmpeg....
View 13 Replies
View Related
Apr 21, 2008
Any recommendations for a joomla website hosting?
Multiple domains one one account and such...
View 11 Replies
View Related
Jun 19, 2008
I have accounts on site5 and hostgator (shared plan, not dedicated) and was trying setup knowledgetree a open source document management portal built on Linux/PHP/Apache/MySQL platform.
Everything worked fine except the indexing component as that required Java/JRE. Indexing is very important to me.
So all my hard work for the last 2 days came to a screeching halt.
I badly need to find a hosting service that offers LAMP + Java technologies for a decent price.
View 2 Replies
View Related
Apr 21, 2008
A friend of mine basically just wants to get his business on the web. He wants a website that is entirely informational (e.g. no commerce or anything like that). Basically a home page with a couple of links to download some documents and some contact info.
He does not have a domain name, so he would need that.
And the person wants an easy to use website designer for this simple site.
My initial thoughts were:
Microsoft Office Live - but that doesn't seem to be totally free anymore.
Wordpress.com (I am not sure this would work as well especially if you wanted to post documents or something)
View 11 Replies
View Related
Jun 4, 2008
I have been building a business around blogs and other social media items on mostly OSS technology - but unfortunately, my current reseller account is inadequate for what I need (especially with sites that query a DB) and unable unfortunately to upgrade to what I might need. I don't want to rip on them because they've provided me great service, but they're just not able to ramp up. My thought is there won't be any reseller to meet my needs after learning more info, so I'm going to make the leap up.
I'm not going to go pound the pavement for hosting business; this is only really for my clients I serve as a agency/consultant/designer and my clients need and deserve more.
I have been considering Mosso; but also weighing a dedicated box as it's around the same price and looking around here for the solutions people recommend for the past few hours. But since I sort of have a specialty application, I thought I'd ask for your recommendations.
I want to say that I came here from another forum because my provider was affected by the Planet outage and the conversation and knowledge was great; you are a very smart community. I hadn't heard of this place beforehand but am glad I found it.
View 3 Replies
View Related
Feb 2, 2008
Iīm thinking about using a centralized hosting solution, in order to achieve better redundance and performance while having more room to expand if necessary. In order to achieve this, I was thinking about implementing a storage server, and use a software to provide iSCSI target capabilities.
As storage server, I was thinking about using HP DL320s ( URL ), loaded with 12 147GB SAS HDīs 15K RPM. I will make some tests to understand the real difference between RAID5 and RAID10, concerning write speed. Also, Iīm not sure if the controller provided with this server is good enough to provide a reliable operation.
For switching, I will use HP 2824 or 2848 Gigabit switch, and use port trunk in order to join both NIC controllers of the storage server.
As iSCSI target software, I still donīt know wich one to use. I think FalconStor would be a good bet, however it seems to be a bit expensive. Any good alternative?
This storage server would be used to provide storage for about 10 "regular" hosting servers, that have, at the moment, regular dual 10K SATA Drives (Raptor) in RAID1. I'm afraid the 2x Gigabit ports arenīt enough, even considering that I will not have intensive sequential reads / writes, but random acceses.
View 14 Replies
View Related
Jun 7, 2008
I'm going to be rolling out a php/mysql driven site soon and I'm pretty much resigned to the fact that the mysql performance dreamhost has given me isn't going to cut it, its probably too oversold. Simple one table one column selects can take 30 seconds or time out depending on how badly the server is being hammered. HTTP requests are usually snappy, but the mysql performance is bogus.
What is a good host for me to launch this site with? Storage wouldn't need to be too terribly high, at least initially. I'm tempted by MediaTemple's slick marketing, but I've seen on here that some people have had poor sql performance (contrary to what some personal friends have experienced, so I'm torn). I was reading about downtownhost on here, but their load times seemed slow when I hit a couple domains listed on here that are hosted by them.
This host needs to be located in the US. Honestly, I like dreamhost and their panel, save for the sql sluggishness I'm getting.
View 14 Replies
View Related
Jan 10, 2009
im running a few blogs on a shared hosting account (steadfast) and i noticed the 18 sql queries being generated by a fresh install of wordpress 2.7 are taking 0.246 seconds to execute.
<!-- 18 queries. 0.246 seconds. -->
considering this is a shared hosting server with over 800+ more domains on it, is the performance better or worse when compared to other shared hosting providers ?
archim3des
View 14 Replies
View Related
Jul 13, 2007
I am using dreamhost host 3 of my web sites and 1 blog. Dreamhost is great, offers alot space and bandwidth.
but I think they are oversellling their space, sometimes it gets really slow. (overselling ? ok, I dont really know, but sometimes its really slow, and most my asian readers said need to refresh to load the page. I am wondering if theres a way to check if they are overselling or not.)
I am thinking about buying vps, even tho, I still got 5 month left with dreamhost.
I found 2 vps companies are highly recommanded on this forum, JaguarPC and LiquidWeb.
theres already a post compared both companies in terms of price and service. I say I will pick JagarPc, cuz, its basic plan just 20 USD, and htey got promotion now, its even cheaper. and basic Liquidweb vps plan is 60 bucks.
I am wondering why Jagarpc is so cheap , are they overselling? how can we check if they are overselling.
I found a few posts saying how good jaguarPc is. and they are not overselling, but those members just signed up this month, and only have 1-3 posts. I cannot really trust those new members.
Can someone share their experience with JaguarPC? compare JaguarPc performance and liquidweb performance. antoher question is switch from dreamhost to JaguarPC basic vPS plan, will performance gets better?
last question: VPS account allows 3 IP, 3ip = 3 domains? if not, how many domains can I have?
View 14 Replies
View Related
Sep 24, 2009
I need to put a proposal for a client who has 20 TB data storage (files/images).
Will propose PHP/mysql - mysql for meta data of files and other authentication etc. Application/mysql could just be ~10 MB.
The storage would increase (incrementally) ~ 1 TB per year.
And data needs stored at a different place for Disaster Recovery point of view.
Need to figure storage costing.
Please give me some pointers:
a) How to find high storage hosting provider?
b) Has any one experimented with Cloud Computing, and if so any ball park potential costing? I attempt S3 and could not figure costing.
c) or any other suggestion.
View 14 Replies
View Related
Aug 13, 2008
company that can offer me super fast php hosting. It will be large loads with upto 20 php requests a second, the load is so much that i am currently running mysql on a dedicated mysql server.
View 14 Replies
View Related
Dec 6, 2007
My company are looking for a web hosting solution, that can handle sometime spike of 4000 connections at a time and not really a lots of Data transfers. Lets say 150 GB per month.
View 3 Replies
View Related
Aug 18, 2007
Im currently with host gator and have nothing but amazing things to say but i have a music streaming website and its eating up bandwith like crazy but at least the cpu usage of the shared hosting is not that high but i will definitley pass my 2 terabyte mark within the month. My question is even on the dedicated hosting plans on host gator they still dont offer a lot of bandwith so does anyone know any very reliable and good companies that offer a LOT of bandwith for a moderate price. I was looking at liquid web dedicated which offers 3 terabytes as i hear their super reliable and professional
View 7 Replies
View Related
May 24, 2009
I'm not into Dns Clustering that much all I know is some articles and threads about how to do it, and the benefits
Anyway I decided to give it a try 1st with minimum fees
so I want to host my Main domain website on a VPS, as it don't need much resources and using it as a Primary Dns Server
and use my current server as a Slave DNS server that is contain most of my client sites
It's logically possible as I see but..
Is there any Disadvantage? if exist what is it?
Do I have to make both on the same DC?
Will it affect my Slave DNS Server Performance, Stability and Load?
View 0 Replies
View Related
May 25, 2009
What are the pros and cons of setting of a DNS cluster over multiple VPSes?
View 1 Replies
View Related
Apr 28, 2008
I just set up a cPanel DNS Cluster today. Followed the guide, and everything didnt according to my plan. As in, setup went fine, setting it up went fine too. But I do not think I understood how this solution works in real... or atleast I had assumed it to work in a different way.
So for my questions...
1) How and where do I edit the DNS Zones. On the local system or the DNS dedicated server?
2) How do I create nameservers for this dedicated DNS server? Not the registrar part, but the A record and NS record part.
3) Small doubt, not a big one. Does the DNS server have to be in stand alone mode or synchronized mode?
View 6 Replies
View Related
Aug 25, 2007
I am trying to setup a server cluster for my client and was thinking about load balancer.
Currently he has 4 Servers and 1 VPS.
4 Servers are in the same datacenter while VPS isn't.
Now from my thoughts about the system load, I have decided to setup like this:
VPS being the load balancer
2 Servers being the Apache
1 Server being Storage server
1 Server being MySQL
This will be hosting two forums that has very high concurrent access.
Now the problem is VPS has only 200GB/mo bandwidth while he needs about 800GB total.
My question is is there anyway to load balance the servers without the traffic going
like this:
LoadBalancer-->Apache Servers-->LoadBalancer-->USER
If the traffic goes like that then VPS needs to have about 1600GB bandwidth and will have network bottlenecks.
I would like the traffic to go like the following:
LoadBalancer-->Apache Servers-->USERS
View 7 Replies
View Related
Jul 18, 2007
I'm currently using Cpanel on all our web hosting machines (total of 6 machines). Each runs its own DNS but also uses the Cpanel cluster feature to sync between all machines.
This has got to the point where DNS updates will not stay, the machines get very lagged etc etc. Now I have put 2 more machines online to do nothing but DNS.
What is the best way to set this up?
View 5 Replies
View Related
Mar 25, 2009
What sort of redundancy that is recommended for a high risk site?
My definition of high risk is it will attract more hackers than usual, more DDOS, chances of blocked by ISP and etc.
View 4 Replies
View Related
Jun 29, 2009
Other than Rackspace, can the good folks here recommend some data-centers that have experience in handling the following solutions:
SAN (F/C, 10Gbps), 9TB usable capacity, complete redundancy, and real-time mirroring to an additional SAN in a separate geographic location.
redundant Brocade SAN switches
Servers: Quad processors, quad core, 48GB RAM, 8x146GB SAS Drives, multiple 1Gbps NICs
VMWare installations and handling.
Load balancers (preferably Brocade)
Dedicated L3 switches
I'm not just looking at providing this, but someone who can truly manage it 24/7 - management from VMWare and SAN perspective.
I saw some niche datacenters, but ran into the following problems:
(a) singular location.
(b) support for VMWare and SAN provided during business hours only
(c) no prior experience in real-time SAN replication / VMWare, Oracle/MS-SQL Replication.
View 14 Replies
View Related
Oct 13, 2008
quality shared hosting solution for a WordPress-powered multi user blog. The blog currently has around 1000 posts, 5000 comments and around 2000 unique visitors every day, but those numbers will grow grow exponentially in future, so the hosting in question needs to be expandable since I will eventually have to move to a dedicated server.
Right now, however, I want a quality US-based hosting company that has good connection to EU, with fast support and basic features: PHP5, MySQL5, shell access, ~10GB HD space and ~50-100GB traffic.
First and foremost I am looking for quality and am prepared to pay as much as $25-30 per month.
View 14 Replies
View Related
Sep 14, 2008
I have a dedicated server currently hosted over by Aplus.NET
I have a 3000 GB Monthly Transfer limit and we have been going over this limit for the past few months. This has resulted in a large sum of overage fees.
I am looking to go to another hosting company that is just as good as Aplus.NET, if not better... with a better traffic rate. A friend told me about Choopa.com and I wanted to know how good of a company they were. What are some other top reliable hosting companies with premium servers and that specialize in unmetered bandwidth?
View 20 Replies
View Related
