What Would You Setup For A Traffic Intensive Site
Mar 28, 2008
Calling on all hosting and server experts here. (If you're not an expert, still feel free to take an educated stab at this. But please leave out total made up answers or foolish answers like "Have LittleJoeShmoe Hosting services do it all for $9.99/month".)
Scenario:
If you knew or were planning on developing a site that you knew would generate millions to tens of millions of page views a day, how would you go about supporting a site of such traffic? The site would not serve up videos, but the average page size would contain up to 75-100kB. It would incorporate databases (user logins, accounts, user submitted content, server side scripting, CMS, etc.)
Don't assume anything. Don't assume you have too little or too much money. Just, what would you plan out to accomodate such a scenario?
What hosting companies would you use? Would you do it in-house and build your own datacenter? Farm out the server management? How much would it cost to implement your plan? What platform would you recommend for a site to handle this much traffic?
View 5 Replies
ADVERTISEMENT
Sep 16, 2007
I am currently moving from my current dedicated servers because they simply cannot handle the load. I have a site which frequently makes it onto radio, digg and other similar sites.
I need a dedicated server that can take a beating from Digg and offline Media. For most of the month the server load is really low, the site hardly uses up anything. However, when it hits those sites, it suffers.
I am OK with using Shell, just basic tars/logs/sqldumping/httpd.conf editing/rebooting etc.. anything beyond that like installing and configuring software I cant really do.
I guess I am looking at a dedicated option (linux based) with a host that'll setup software/modules modules for me when I ask, but doesn't really need to hold my hand all the time.
How are ThePlanet.com's servers? Do they manage the servers?
View 14 Replies
View Related
Oct 3, 2008
creating a setup that will host a site which is expected to receive 50-60K visitors in the first few hours after its launch. The site is membership based and the backend (member system) runs on PHP5-MySQL5.
Here is what I have thought of until now.
Site's sales page (which also happens to be the first page that visitors hit) hosted with Amazon S3 service. All public media files are off loaded to amazon S3 service to keep the number of requests on the hosted setup to minimum.
At the front we can have a high performance firewall like Cisco ASA 5520 followed by two dedicated load balancers in Active/Active state.
Behind the load balancers we have 3 front end servers acting as web-servers. These have SAS disks, 4GB RAM, RAID 1 setup, Dual Xeon Quad core processors each.
Behind the front end servers - we have a dedicated load balancer for the database cluster.
The database cluster consists of 3 Storage/API nodes and one of the front end servers acts as the management node. Each storage node has 8GB RAM, Dual Xeon Quad core processors, 4x RAID 10, SAS setup.
The private network is on a GigaLan.
Do you see any possible/obvious flaw in this design or anything that should be added/subtracted from the setup?
View 14 Replies
View Related
Apr 8, 2009
How do i setup vpn server on windows 2003 so that client traffic go through the remote vpn server? Something like strongvpn.com service which change the client ip address.
I tried setting up, now client able to connect to the server using windows vpn. But once connected, internet cannot work. How to use the vpn server as gateway?
Server have only 1 network card.
Can windows xp act as vpn server for the same purpose?
View 1 Replies
View Related
Apr 18, 2009
We currently have a website for our client developed on ASP.net with SQL
Its a straight forward web application with 2000 visitors per day
The traffic is going too heavy and its now at 130 GB per month
Our Queries :
Can we still have this site on a shared environment ?
If yes, suggest few providers offering high bandwidth
What other alternatives are there?
View 14 Replies
View Related
May 11, 2008
I own an anime linking site which you guessed it links to anime. I have around 10000 hits a day and would like a vps with litespeed since it's so much faster.
View 7 Replies
View Related
Oct 5, 2008
In AWStats I am getting a lot of traffic from one URL but it is all spam traffic. How can I make it so that any visitors that come from that URL cannot access my site?
They have a link on their site to mine...
View 11 Replies
View Related
Jan 12, 2007
Is it common to get informative and appropriate website traffic statistics via your host? or is this some I should look at joining/purchasing independantly?
View 10 Replies
View Related
Jul 10, 2008
We expect a site on one of our boxes to receive a significantly high level of traffic tomorrow as it is an event that will be covered and has already been covered by the press. The site operator expects > 100,000 hits a second.
It's a PHP page that pulls records from a database and lets people submit a form to insert a record. We have already put a caching script in place so that refreshing the page does not result in doing another database query.
The site was overloading a shared server, and we've moved it to one of our VPS boxes - it's the only VPS on the system at the moment. The box is a Quad Xeon 5410 with 4GB RAM with a 4 10K RPM drives in a RAID5 setup.
View 3 Replies
View Related
Jun 9, 2009
Let's say I run a gallery site that is powered in php with no mysql required. Can a shared host handle traffic that receives 50k+ hits daily?
I know a shared host can handle 50k+ hits daily if the site is pure html but I'm not sure if that still applies the same for php
View 11 Replies
View Related
Jan 14, 2009
do you suggest verio for high traffic video site?
View 3 Replies
View Related
Aug 19, 2009
I am writing a financial statement for my business plan and like to know what numbers should I expect from the the host. If the potential traffic will reach 405,000 visitors a month what my requirements for the hosting company should be?
View 9 Replies
View Related
Jun 4, 2009
This is not strictly a technical issue with our server as such, but this subforum still seemed most relevant. If not, please move the thread to a better suited one.
Bizarre problem: in the last 2-3 days, visitors from certain countries (France, Holland, Serbia, Slovenia, Macedonia, Montenegro) are having difficulties in reaching our website hosted on cari.net. However, other countries (USA, UK, Germany, Russia) don't seem to have this problem at all. The server itself is a-OK.
The website hosts an IPB forum. The problem is that every so often (very often), upon clicking on a link, the browser hangs on "connecting", and often times out and returns a Bad Request message. Sometimes, when a clicked link that's waiting is refreshed, it immediately loads the page up quickly, as if nothing is wrong.
We cannot call our hosting company, as it's really nothing to do with them, the server works perfectly, it's just that traffic from certain countries has difficulties to get through a certain level3.net node in the USA (that is where tracert usually breaks down, or gets through after several attemtps).
View 2 Replies
View Related
Nov 12, 2007
Current Forum users: 550,000 users
Traffic: 700,000 to 1,000,000 page views a day with 70,000 – 100,000 visits a day.
Monthly bandwidth needed around : 18000GB (18TB)
Now can someone tell me what kind of solution i need to host this website & keep sit running smoothly.
View 4 Replies
View Related
Feb 28, 2008
My video sharing site has high traffic, alexa rate:3,000
My site has 2 servers to split the load. 2 servers share a mysql server. Using rrdns to load the balance.
Server A running mysql 5.0,lighttpd
Server B running lighttpd.
Server B connect to A's mysql database.
During peak time. B can not connect to A's mysql server. It says server not responding. But A still running fine.
When I check mysql log file.
/usr/libexec/mysqld: Forcing close of thread .....
And when run top, the load average is 20.
The spec of Server A
Intel(R) Xeon(TM) CPU 3.06GHz dual core.
2G Ram.
Here is the my.cnf
Quote:
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
max_connections = 1000
wait_timeout=60
connect_timeout=10
interactive_timeout=120
join_buffer_size=1M
query_cache_size=128M
query_cache_limit=2M
max_allowed_packet=16M
table_cache=1024
sort_buffer_size=2M
read_buffer_size=2M
My question, do I need another maching C to run lighttpd, and just keep mysql on A.
Or I can do some mysql optimization on A.
Also, if my site keeps going, can I have 1 mysql server and 5 http servers?
View 11 Replies
View Related
Nov 2, 2007
i work for a small but fairly successful and quickly rising company. we're looking for hosting for our corporate site. we could host it ourselves but that would put an extra burden on engineering and we're stretched pretty thin as it is. we moved to bluehost on my recommendation. i've been using them for 2 years now and never had a problem. unfortunately less than two days after we switched the server is now down. it's been down for over 4 hours now and we've blown past several estimated times from tech support. i know this is a rare thing but it doesn't look good and i don't think the higher ups much want to stay with them. so here's some info about our site. requires SSI, CGI, Perl 5.8+ with Perl DBI 1.32+ and Perl DBD::Mysql > 2.1021+ Perl Modules, ImageMagick 6.0.6+ Perl Module or GD Perl Module (GD.pm 2.17+), PHP 5. small traffic with occasional spikes. approx 200 uniques daily. occasional spikes if we happen to show up in the news. we do have the possibility of showing up in all major US news outlets. it's happened once before which resulted in almost 1,000 uniques which quickly tapered back off to normal levels. we're currently using 154MB of disk space and 8GB of bandwidth last month. we need multiple ftp accounts to allow marketing and contract web devs to upload their own files. probably around 10 or so would be adequate. email not required. we have a hosted exchange server. uptime and quick response to server outages important. being a corporate site and not our application downtime isn't truly horrible but it is perceived as such. this site is our identity. we're a tech company and it looks bad when our corporate site is down. even when our application isn't.
View 10 Replies
View Related
Jun 14, 2015
I'm getting a traffic notification on a site I restored from a backup that originally came from a standard Plesk edition. Web Admin Edition doesn't appear to have access to Subscriptions, and all I can find with regard to limits under Webspaces is a hard disk quota, which isn't even editable. How do I amend of remove the limit?
View 6 Replies
View Related
Jul 7, 2009
How do I setup a site to use two servers? One as a mysql server and the other as a web server.
At the moment I have the site on one server. When I get the second server what steps do I need to take to make this possible?
How do I secure the mysql server to only allow the web server to connect to it?
Do I need a control panel (i.e. cPanel) software on the mysql server?
View 14 Replies
View Related
Jul 29, 2008
I plan on purchasing a dedicated server to run a few proxy sites. However, i don't have a lot of Linux skills to administrate my own server. But i've always been able to figure stuff out so i feel pretty confident that i can get it going on my own. Anyways, My original idea was to use cPanel to run the web server for the proxy sites however i feel that using cpanel is not a great idea since i also wanted to maximize my servers performance and i hear cPanel can take heavy resources. So anyways, I just wanted to know what some other people here might think and if possible any recommended guides on setting up a linux webserver would also be appreciated as well.
View 4 Replies
View Related
Jul 14, 2008
I'm working on a new project that involves some heavy data processing in the preparation stage. As an example, one of the setup jobs is now running on a P4-3.0HT desktop at 100% CPU and I estimate it'll be finished in just under a week! Memory and i/o usage are minimal but it's extremely CPU-hungry.
There are other similar jobs to follow and some correlation between the number of cpu cycles and the quality of results so I foresee an ongoing need for computing power for at least a few months.
Viable options are to continue running my own hardware (probably supplementing the P4 with something a bit newer), Amazon EC2 or a cheap dedicated server - I can get a good handle on the cost and performance of each of these.
But I wonder if I'd get more bang per buck from a few VPSs? Some of the VPS benchmark results are spectacular so I suspect the answer could be yes, in the short term anyway. But I don't want to hog the host-node's CPU to the point where it degrades other users' performance or gets my account shut down... Then again, judging by the performance of the VPSs I've used for hosting the node seems to have a lot of spare CPU available...
So (finally) to the questions: Am I crazy to even consider this? If not, which technology would be most suitable? (I'm thinking Xen because of its reputation for better isolation.) Has anyone else done anything similar?
View 7 Replies
View Related
Aug 2, 2008
my client a dedicated server which could handle the following things.
1). Intensive DB queries done from many operators for custom project.
2).DB when info process completes, Generates a PDF which would be upto 10MB in size.and also runs MAPLAB Applications.
3). should be able to run back-up's or please suggest an alternative to always have a copy of recent data.
4). pro-active scanning on Incoming FTP content for Viruses. (is it possible ?)
Considering softlayer with:
either
Single Processor Quad Core Xeon 3220 - 2.40GHz (Kentsfield) - 2 x 4MB cache
8 GB RAM
+ 147GB SA-SCSI 10K RPM
OR Preferring
Dual Processor Quad Core Xeon 5335 - 2.00GHz (Clovertown) - 2 x 8MB cache
8 GB FB-DIMM Registered 533/667
+ 147GB SA-SCSI 10K RPM
View 9 Replies
View Related
Apr 17, 2009
I run a large adult vBulletin community with 70,000 members, 1/2 million posts, 186,000 attachments (a lot video), and closing in on 100 million downloads since our start some odd years ago. I've been battling keeping the site up for quite some time, and I am starting to wonder whether we shot too low on the server setup. I figure I would ask the pros here at WHT for some advice.
This is our current setup:
Site server:
Quote:
CPU: Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz
4 Gig ram
250 Gig sata harddrive
Unix FreeBSD 6.2
Apache
MySQL server:
Quote:
CPU: Pentium III/Pentium III Xeon/Celeron (2666.62-MHz 686-class CPU)
Cores per package: 4
4 Gig ram
750 Gig SATA harddrive
Unix FreeBSD 6.4
Apache
Do you think the site would perform better under one server and maybe a more powerful processor? What should I be looking at exactly as far as hardware goes for this type of site. I should note we push about 2.5TB of bandwidth monthly.
View 14 Replies
View Related
Jun 12, 2009
I have two VPS's which run a single MySQL intensive site. The first VPS runs cPanel and has about 50 databases, 200mb each. Second has another 30 databases, of the same size.
What would be the best method to backup this website daily?
At Current;
I use automysqlbackup to backup all the databases at midnight (this crashes the VPS's for about 10 minutes each night). This dumps each one into a zipped file.
Then the rsync copies the changed files (forum attachmenets, cpanel changes etc(whole server)) to an off site location.
Are there any easier ways to do this? The databases are most important!
View 1 Replies
View Related
Apr 14, 2008
I own a couple of servers xeon setups. All use cPanel/WHM.
A client yesterday asked if we could have ffmpeg-php installed on the server so that they could run phpfox.
I have heared ffmpeg is resource intensive? Will it make a big difference on dual core server? Should I install ffmpeg-php or risk losing a client?
If I should install ffmpeg-php I have been having trouble doing so, can anyone help me with this.
View 4 Replies
View Related
Dec 3, 2008
I've taken the scalable approach when it comes to servers for my various sites. With shared servers, I never really worried about backup or even hard drives going down. Same goes for VPS. For some reason, when I moved to dedicated servers, I outfitted them with 74GB SATA drives in a RAID setup. My understanding is that it protects me if one drive happens to fail. I've been lucky and haven't had that problem.
I'm at the point now where I'm looking to upgrade from a VPS paying around $75 per month to a dedicated server. I can stand to be down a day if a hard drive goes, if it means $75 a month in savings. My biggest concern would be suggestions on the best way to protect myself in the event of a catastrophe.
Contacted SoftLayer about possibly adding a second server for me and honoring the price I'm paying on my old server.
Finally, both the old and new site are seeing roughly 3,000 visits per day. The server I'm considering is a Clovertown 5320 1.86 dual quadcore, 4GB RAM, RAID, 2 74GB Cheetah drives,100mbps, 2000GB bandwidth. Is this overkill or the right server for the job?
View 7 Replies
View Related
Oct 22, 2006
I would like to know how resource intensive is ClamAV Scanner. Should I allow it or not to my VPS clients/resellers?
Can I set it to use it as root? How?
View 0 Replies
View Related
Oct 4, 2009
We were on shared hosting until we were asked to move off due to our database using a lot of the CPU - how much, we dont know.
Anyway, moved to a 1GB Hyper-V VPS, seems like we have one core of an E5420 but presume the server is shared between 12 people.
When our ASP application runs, the MSSQL server reaches 100% CPU and the website performs slower than the shared server we were on.
Where can we go from here? Seems like going from shared to a Dual/Quad Core Dedicated Server is a large jump and a massive jump in price.
What about a host with a high performance dedicated server for MSSQL shared between customers.
View 14 Replies
View Related
Aug 9, 2008
I have cpanel server
I found today that my server gets overloading - load spikes to 60 - 80
I sit down and watch top command for 1 hour. Suddanly I noticed that something is going on - loop0 process started to eat over 100% cpu
I tried to stop httpd on another console window using
service httpd stop
After 4 tries it stoped and then loop0 calm down and load lowered to normal size
(below 1)
View 3 Replies
View Related
Jun 25, 2009
The data centre which I use, is moving all collacated servers to a brand new data centre next week, which will mean a two hour downtime for each of my servers and customer websites.
At the moment my servers utilise two nameservers on seperate servers and when the move happens all websites will be offline with an ugly error message to any visitors.
Does anyone know a good way to setup a page which would be displayed if the web server was down? I am using MS DNS.
My thoughts so far is
1. Setup a 3rd nameserver which is off site from the data centre.
2. Purcahse web hosting / vps for a month on a seperate hosting company
3. Set it up so that it accepts * to a specific IP address in IIS or apache
4. I create an index.php script which gets the host header value sent i.e. [url]and the page then displays a nice maintenace i.e
"We are sorry joeblogs.com is currently down for maintenace, we will be back online shortly".
I think my main question is do I need to setup a 2nd www record in DNS for each site and how do I ensure the 2nd dns (backup record) only gets used when the first website / server is down.
View 0 Replies
View Related
Nov 15, 2007
What traffic monitor would everyone recommend for sites that have as many as 5,000 to 10,000 hits an hour?
View 8 Replies
View Related
