I recently found out that the Google robot spidered my site and used up 5.74 GB in bandwidth! The hits were 4498 and I even have it scheduled to return every 2 weeks.
Now my site is down for the rest of the month unless I up my bandwidth. This is the second time this has happened to me in the past year. What is going on?
In fact i am a customer of PC-CORE.net's directly customer.They send me a email to let me translate my website a few days ago.But i was in a travel then.I recieved it yesterday night. But it is too late that i cannot enter my website then.It is likely the sever ha been shutdown.My ip was 64.191.125.149.The guy rent me the space said he is powerless with it.
I did not get any backups of my site.They are gone when i format my hard drive last time.But the website's date is very important with me.
Anyone related with it?What i can do to save my site? What is going on with PC-CORE.net?
Googlebot has been absolutely ripping through my bandwidth. This has been on going for many months now, but each month the damage has gotten worse and worse. I have posted about this problem here but with no luck:
http://www.webmastertrader.com/showthread.php?t=389
To demonstrate the problem, I have decided to show you all my server stats for one of my websites.
I have no choice but to ban googlebot from my site and follow this action with a letter to google!
1) Aug: http://www.webmastertrader.com/attac...0&d=1159816704
2) Sep: http://www.webmastertrader.com/attac...1&d=1159816714
3) Oct: http://www.webmastertrader.com/attac...2&d=1159816723
We're only a day and a half into this month and Google's robots have already consumed over 1gig of bandwidth!
I really do not want to mention the name of the hosting. I have been with them since the beginning of 2007.
My VPS was shut down about 15 hours ago because there might be a problem with the recurring payment. Actually I was informed that the payment has been processed and approved successfully last week. And there is no any email telling me that there is problem with the recurring payment.
I have been asking their support to re-start my VPS first but they said this is the sales issue. It has to be handled by the sales. Just waiting....... waiting..... Very disappoint!
Will you shut down the VPS of your customers without any prior notice?
If I were to go about setting up a SMTP server on a verizon FIOS line, would they shut me down if I was sending out about 10,000 outgoing emails a day.
I want to set it up for a email newsletter service.
Upon reviewing my modsecurity log today, I found an interesting hit from google.
-------------------
Requesting IP: 66.249.65.67 is http://ws.arin.net/cgi-bin/whois.pl?...t=66.249.65.67
Date: 2006-10-03
Time: 07:10:16
Handler: mod_gzip_handler
Get: /page/index/1&show=25,07,2005?php%20echo%20$bmc_vars%5B'site_url'%5D;%20?%3E/profile.php?id=1
Mod_Security-Message: Access denied with code 406. Pattern match "echo " at THE_REQUEST
Mod_Security-Action: 406
------------
The rule that set off this 406 response was:
SecFilterSelective THE_REQUEST "echo "
What I find interesting is that I do not have any such URL structure on this website that google requested.
I'm running a web server with mod_evasive and want to know how can I prevent mod_evasive from blocking the googlebot crawler ip address.
Is there a script out there that can detect this crawler and make sure its ip doesnt get blocked by iptables or mod_evasive?
I am just wondering whether my idea will work for the google search engine.
Basically, I have my official VPS root (home/admin/public_html/) and this is where my main website will be hosted. However, since my VPS will be used for additional website, I will direct additional domains to it.
My second site hosted on this VPS may have a document root of (home/admin/public_html/advertising/). And my domain will then be setup to have that as its document root.
However, when google searches for my second site (e.g. advertising.com) will it go 'below' the domain root? For example, will it also creep the files under /public_html/ for this domain, even though the domains root is /public_html/advertising/?
edit: Or do people host multiple sites differently? Is this an appropriate method?
Yes, it’s quite true. DreamHost representatives are asking their clients to block GoogleBot trough the .htaccess file, because their websites were “hammered by GoogleBot”.
Dreamhost representatives are also guiding their clients to make all their websites “unsearchable and uncrawlable by search engine robots”, because they cause “high memory usage and load on the server”.
PS: This is NOT a rumor. Multiple people are already complaining about Dreamhost asking clients to block search engine spiders. A post in QuickOnlineTips confirms this too.
Initial news bit via Zoso.
If your server is blocking googlebot from finding your robots.txt file, how do you configure your firewall to unblock it?
I've searched through Google and I've seen may people just say your firewall is blocking it, but none mention how to really stop it from doing that. Like does Google have an IP it uses, and if so, what is the IP you should whitelist for your server?
As I keep getting that message: Network unreachable: robots.txt unreachable
and I'm sure it's due to a firewall issue, just have no idea how to fix that.
One of my customers uses Webmasters Tools from google , looking at what pages he have indexed by Googlebot, found that 180 pages are giving a "DNS lookup timeout" error, tried searching for help on google and the only thing that i found is " We received a timeout on DNS lookup."
DNS are ok, same as the zone file, everything is responding OK, I dont know what can be the issue.. ? any ideas ?
Im using Ensim Pro and RHES 4.
I have recently upgraded to a virtual private server. What is a safe speed for googlebot to crawl my website.
Can you please give me results in the format of:
xxx requests per second
xxx seconds between requests.
If I type google.com in my address bar, it forwards me to www.google.com. This is not happening for my website right now. I think its a good idea to do this, since then search engines will have only 1 main URL for the website to index.
My question is:
How do I implement this? I think this may involve mucking with CNAME settings...
I want my users to be redirected directly to my forum
so when they type in www.mywebsite.com it will redirect instantly to www.mywebsite.com/forums
I know this can be done on Cpanel... any other ways?
I m tired with Shared Hosting so i bought one vps but i haven't any knowledge how to run site through vps and how to change nameserver blah blah
I Have Window VPS
Can any one help me step by step for setting up site through window vps ..
I use Ian Lloyd's book and that's where I found out about this forum. Looks like a great forum.
I downloaded Fliezilla FTP and I'm trying to transfers files from my computer onto an angelfire web site.
Filezilla asks for a server address and I put in the URL address that I registered with angelfire. It then asks me for an administration password, and I put in my password to the angelfire site. I keep getting: Error: Connection to server lost...
Does anyone know what I'm doing wrong here? I would like to use Filezilla to upload my files (web pages) to the angelfire site.
I hosted website and sometime it is going down. Same server some of my websites working fine. Please just know why going down my website sometimes..
View 7 Replies View RelatedI have a website which is currently hosted with streamline.net on their shared msql 11 server.
We have had several issues with them over the last few weeks where someone is using most of the server and slowing everyone elses sites down so much so they crash.
This week and weekend are my busiest time of the year (I sell fancy dress) and my site it totally unuseable.
We have phoned them and they have done nothing except ask us for a log which we have provided for short periods of time.
The down time has now got so bad that I have had only 2 sales today. I estimate I am losing approx 400 per day at the moment due to this problem.
Is there anything that I can do urgently to prevent my business from being killed by someone else.
So I'm interviewing with a company and when I typed in the URL to their website, I was met with a nasty surprise: a "hacked by so and so" message! However, after looking closer, I see that I had accidentally appended a period (".") to the end of the domain name, for example: http://www.example.com./
When I removed the period, the site appeared as normal. I don't know anything about the server other than it's IIS. Is there anything I can suggest to them when I go in to interview? I'd like to point this out to them; it may even help my chances at landing the job! (It's not related to networking, though.)
Does anybody know how I can mirror a websute using PHP?
I have 2 dif domains on different servers different OS, and I need PHP to collect the page from the other server from another domain.
Is it possible to buy a dedicated server off eg Dell and host your own website on it from home ie with a www prefix to the url- Do ye know any good tutorial on it. Would 20Mbit bandwidth be enough bandwidth for a fairly busy php ,mysql site? Completely new to this.
View 4 Replies View RelatedThis is probably a pretty complicated answer so please forgive me as I'm a newbie to making my own ecommerce website.
What steps are needed to protect/prevent one's site from being hacked? I have domain privacy (on WhoIs) but I feel this isn't enough.
I have a domain name websiteexample.com and its hosted with somebody, I have built a new website under a new domain .co.uk on a new server and need to redirect my .com to the .co.uk, how the best way to handle this with serach engines in mind, I dont want to loose my .com listing if possible, is a framed forward a good way?
View 9 Replies View RelatedI want to have my email and website going to different servers:
At the moment i have it setup so that the nameserver records point to the ip of the website. That means that both site and mail go to the server where i host my site. I have then set up an mx record to point the mail to another server...
however can i make this split at the start?.. i.e. from the domain registar? can i set up an A record to point to the website and an MX record for the mail? instead of pointing them both to the website and then redirecting the mail to another server?
Just this week, I believe one of my site has been hacked...or potentially my whole server! When accessing the website (a vBulletin forum), instead of going to the main page, we get a screen that looks like Window's "My Computer" and there is a scan running. Firefox has blocked the site for suspicion.
I am stumped. Where to begin? I have full SSH access to my server (after rebooting it). Thank you in advance.
Server: CentOS Linux 4.3
I have a wordpress site, and am trying to find someone good and reasonable price to design me a template for the website so I can still use wordpress but have access and control to everything.. I am going to buy a domain as well to hook wordpress up to it so I can edit themes etc.. I was looking at maybe using mmhosting..
Also, I have a forum, but not sure what type of forum I need to be able to arrange it and add and use features don't I can't now, so basically upgrade..
My site, as I said is wordpress but I want it to look like the forums I am using..
I'm thinking of starting a website that pokes fun at news stories and photos, but I'm a complete rookie at this. All the RSS feeds I found say they are free for non-commercial use. I want to be able to sell t-shirts and caps on my site, or at least advertising. Any ways to use current news and photos that are free or inexpensive?
View 3 Replies View Relatedif i were to have 1 website get have a DDoS attack to it, would it knock out my server for a while, and can i recover the server by restarting it?
View 14 Replies View RelatedI've discovered that a site of mine has being completely copied
The sites are in a different country
[url]
[url]
The original site, that is still currently under construction with a lot of copy, rewriting and pages still have
[url]
Now, I don't mind someone taking inspiration or borrowing ideas. I'll admit as a web designer I have 1000's of book marks of sites that I've liked and will look at for ideas and inspiration, but I don't just download and put up.
Now as the sites are in a separate country I don't know yet if it is worth all the hassle as our customer base are separate, but I presume that having identical content means that google ranking could be damaged?
How to check web hosting of any Website and which hosting is more valuable and why?
View 24 Replies View RelatedMy website down several hours ago, I have submitted two tickets already, but none replied and none contact with me.
my site has a daily 10k+ traffic, I will lose 10 dollar every hour... And there are pending ads. I need to put on.