Test Robot.txt
Jul 23, 2008I want to stop cataloging of all PDF files on my site, can this be done using wild cards through Robot.txt?
What is a good way to test to make sure that the instructions in the Robot.txt are working?
I want to stop cataloging of all PDF files on my site, can this be done using wild cards through Robot.txt?
What is a good way to test to make sure that the instructions in the Robot.txt are working?
i would like the robots not to index my gallery pages so the code would look like this right?
User-Agent: *
Disallow: /gallery1.html/
Disallow: /gallery2.html/
Disallow: /gallery3.html/
where do i put this code?
One of the sites I help run has 1.5 million pages of parts that can be ordered and information about those parts. We started seeing a ton of hits at around 20-30 pages per second over the past couple days.
There is no information about them on the internet and they are also masking as GoogleBot.
Be on the lookout as I am pretty sure they are trying to steal content and post it up for search results. They are using Amazon EC2 servers to do it also.
This is how they are identifying themselves:
Mozilla/5.0 (compatible; Adtuitionbot/1.0; +http://www.google.com/bot.html)"
Offending IPs: 174.129.155.59, 72.44.52.93
In a 24 hour period, we blocked 408,821 requests from them.
Is there a simple windows based tool that we can test two websites on two different VPSs?
Basically just load and what not, but some of the tools we have found are a bit intimidating.. We are just looking for something easy to setup and use.
Is there a way to test an SSL to make sure the page really is encrypted?
View 3 Replies View RelatedI have 2 VPS in 2 differents networks. One of these 2 VPS have a WHM/Cpanel load time very high to me. I already contact with my hostiing company support and I told the problem. Support saied that the problem is with my ISP connection. I agree that is possible my ISP issue, but I already test with 3 ISP connections diferents.
So I would like a help to test it. I am creating 2 dummys reseller accounts in my 2 VPSs and I inform the WHM login details here. So I would like that you access both WHMs and try to create a new account and tell me the time to finish the creation process.
From my end one account takes around 90 sec and other take onlu 13 sec.
how much time do you give in creation account on the followinf 2 dummys accounts?
I just installed Squid V3. I set up putty to ssh socks proxy to my vps that I installed squid on.
Here is a snippet of my cache.logs, but there is no cache log of websites I have visited. The access.log is empty.
How to I tell if squid is working?
Quote:
2009/04/04 22:52:37| Starting Squid Cache version 3.0.STABLE13-20090405 for i686-pc-linux-gnu...
2009/04/04 22:52:37| Process ID 9886
2009/04/04 22:52:37| With 1024 file descriptors available
2009/04/04 22:52:37| Performing DNS Tests...
2009/04/04 22:52:37| Successful DNS name lookup tests...
2009/04/04 22:52:37| DNS Socket created at 0.0.0.0, port 36048, FD 7
2009/04/04 22:52:37| Adding nameserver 208.67.222.222 from /etc/resolv.conf
2009/04/04 22:52:37| Adding nameserver 208.67.220.220 from /etc/resolv.conf
2009/04/04 22:52:37| Unlinkd pipe opened on FD 12
2009/04/04 22:52:37| Swap maxSize 102400 KB, estimated 7876 objects
2009/04/04 22:52:37| Target number of buckets: 393
2009/04/04 22:52:37| Using 8192 Store buckets
2009/04/04 22:52:37| Max Mem size: 8192 KB
2009/04/04 22:52:37| Max Swap size: 102400 KB
2009/04/04 22:52:37| Version 1 of swap file with LFS support detected...
2009/04/04 22:52:37| Rebuilding storage in /usr/local/squid/var/cache (CLEAN)
2009/04/04 22:52:37| Using Least Load store dir selection
2009/04/04 22:52:37| Set Current Directory to /usr/local/squid/var/cache
2009/04/04 22:52:37| Loaded Icons.
2009/04/04 22:52:37| Accepting HTTP connections at 0.0.0.0, port 8080, FD 14.
2009/04/04 22:52:37| Accepting ICP messages at 0.0.0.0, port 3130, FD 15.
2009/04/04 22:52:37| HTCP Disabled.
2009/04/04 22:52:37| Ready to serve requests.
2009/04/04 22:52:37| Done reading /usr/local/squid/var/cache swaplog (0 entries)
2009/04/04 22:52:37| Finished rebuilding storage from disk.
2009/04/04 22:52:37| 0 Entries scanned
2009/04/04 22:52:37| 0 Invalid entries.
2009/04/04 22:52:37| 0 With invalid flags.
2009/04/04 22:52:37| 0 Objects loaded.
2009/04/04 22:52:37| 0 Objects expired.
2009/04/04 22:52:37| 0 Objects cancelled.
2009/04/04 22:52:37| 0 Duplicate URLs purged.
2009/04/04 22:52:37| 0 Swapfile clashes avoided.
2009/04/04 22:52:37| Took 0.02 seconds ( 0.00 objects/sec).
2009/04/04 22:52:37| Beginning Validation Procedure
2009/04/04 22:52:37| Completed Validation Procedure
2009/04/04 22:52:37| Validated 25 Entries
2009/04/04 22:52:37| store_swap_size = 0
2009/04/04 22:52:38| storeLateRelease: released 0 objects
2009/04/04 22:59:06| Squid is already running! Process ID 9886
I recently setup WAMP on my dedicated, and I'm unsure if the slow download speed is from WAMP or something else I need to remove from the dedicated. Might just be the distance I am from the dedicated because it's hitting 88.74Mb/s down and 71.19Mb/s up from a local city.
It's located in Kansas city, MO.
208.94.241.242/speedtest/
How can i test to see if a port on my box is open or not, can i ping it with the ip and the port?
View 4 Replies View RelatedI ordered my first vps and uploaded a 1 page web site with a few graphics. I browsed to the web site and it took longer than normal to load the entire site with all graphics.
Is there a way that I can test my vps account speed?
Can i have a speed test from different locations. [url]
View 14 Replies View RelatedCan a few people try to download the test file in [url] and tell me what speeds you get and from which location?
I'm comparing UK2 and RapidSwitch. I'm interested to know the d/l speeds from East/West coast USA, Asia, Australia. I'm assuming Europe would be fast.
We are a small development team mostly coding in php/mysql as an unpaid hobby. Anyways yesterday our test server had hardware problems and to get a decent new one would be $300. Well this seems a lot to ask for someone that does get any compensation for it. So we are looking for some cheap way of providing a test server. We need at least a way that provides svn, apache w/ php and mysql. We tried to use the webserver, but the head dev didnt like the idea. Any suggestions?
The three devs are in charge of wcreplays.com .
i think 216.245.208.x and 216.245.215.x DOWN, but limestonenetworks tell me :"problems well outside of our network, and I would highly suggest contacting your ISP for support on this matter" ....
View 9 Replies View RelatedI decided this probably belongs in its own thread. It's also posted in Colo Suggestions/Help. Maybe mods can remove it there?
I think it'd be nice to have a thread with test files from a bunch of different colo providers. In particular I'm looking for ColoPronto and Colo4Dallas, but hopefully people will post from data centers all over to help others that are shopping around as well.
I have tested the Colostore test file[url] from a few connections I readily have access to.
My home Comcast cable connection downloads at about 480kbps.
My work T1 connections downloads about the same. But my works Comcast cable connection can only download it at about 40kbps, and it takes well over an hour to download.
Does anyone have any insight as to why this may be?
Also, does anyone know where I can find a test file for ColoPronto and Colo4Dallas?
Anyone willing to post one for me if there are none available from the DC? I would really like to test speeds.
I got problem with my DNS. The error shows Low port test. Why I got this error?
One more question, If my DNS fail in Open DNS servers section, what will happen with my domain?
halekulanicorp.com
just need a simple smtp test......
I noticed that lately there are tons of cheap vps offers.
Before you pay for it ahead for next 5 years, test it.
From my experience with vpses going years back i do it this way.
Install Webmin if you do not have it.
Set it for https and MCS.Linux Them, one with all graphics.
Run WHT benchmark.
If you get a score 60+ great, test several times (day, night etc.).
Login into Webmin in https mode.
Go to others and click on System and Server Status.
If you get a full update under 8 sec that is good.
Reload few times and time it again.
In my opinion you are testing connectivity as well and not
only vps itself.
Otherwise i think you will be looking for another vps soon.
I've tried googling, searching on webhosting talk for 100meg test file, speed test for linux server, ect.. ect.. ect.. and I cant seem to find a damn thing.
Anyone have a recommendation for a place to download a 100mb test file to test out my speed? I'm looking for a server on a gig line.
How do I test to see if I am getting the bandwidth I am signed up for?
Right now I have a vps with futurehosting.biz IP: 75.126.49.194.
On their advertisements they say everyone will be on a 100mbit port.
However, I have never gotten speeds above 10mbit UNLESS I am downloading from another machine in the same datacenter:
dallas.futurehosting. biz/test100.zip*
When I try downloading from a machine outside the same building such as:
www .futurehosting.biz/test100.zip* (Their Chicago DC)
I never, ever get speeds over 10 mbit outside the same DC.
I opened up a ticket regarding this, and all they say over at future hosting is that Virtuozzo confirms it's a 100mbit link, and that other users in the node may be using the bandwidth at the same time.
The only explanation I can think of is that it truly is a 100mbit port, but is capped to 10mbit once it goes outside the datacenter. If this is the case, they should clearly state this in their advertisement. Or is this a common practice with VPS hosts?
* Links mangled because of the 5 post requirement....
I'm behind the firewall on a public computer and all online port scan tests I've tried show that ALL ports on a computer I'm using are closed. Is this possible? I've been previously succesfully uploaded some files through cpanelproxy.net to get access to my site, for which opened port 80 was needed.
View 6 Replies View Relatedit possible to test own server in my home(only to test)
iam haveing centos 5.4 with vmware
need 2mpbs for testing it be good i think so
but only i haveing issue is ip(iam useing dynamic ips
it anyway to test as local server with dynamic ips?
Anyone have a good download test in chicago other then FDC's?
I was told i was upgraded to a 1GBPS port for free by my host(from a 10mbit) which is sort of weird, i want to confirm. at this time FDC's 100MB file takes me about 3 seconds to download and downloads at 44.7M/s.
the file is too small for me to try and pull 1gbit, by that speed i have a 500mbit line odviosely but i want to test against something outside of FDC.
I hope all are fine and doing good, can somebody provide detail guide line how to test Penetration testing on linux servers and what security measures are required in against of that.
View 8 Replies View RelatedAnyone got a 100mbit server at corenetworks that I could speed test from? The test file that I got from corenetworks was only able to give me about 500KB/sec so I think their server might only be 10mbit.
View 4 Replies View RelatedJust moved to a 1and1 managed server in which FastCGI is enabled.
But I'm having problems getting Movable Type to run under fastcgi - which has never been a problem before, even on shared hosts.
Is there a 'hello world'-type test for fastcgi - so I can check if it is enabled?
What is the best way to test the speed of a server?
CPU/processing wise, and not bandwidth wise?
[url]
What download speed do you get
I have 100MB/Sec port and normally always get around 500-600kbps download speed with DedicatedNOW but today I am only getting around 170kbps
I have a 10mb connection and when downloading file from here [url]I get 1.2MB/sec download speed
I asked support to confirm if I had 100MB/sec port and got a reply with:
"Your server is on a 100mbps port:
-bash-3.2# ethtool eth0 |grep Speed:
Speed: 100Mb/s
-bash-3.2#"
Sounds like a local issue. I am able to download at:
Response: 226-File successfully transferred
Response: 226 32.674 seconds (measured here), 6.12 Mbytes per second
if any of you know of a gigabit url, to where I can use wget to test the speed of my gigabit uplink?
View 14 Replies View RelatedI'd like to install xen vps system to test out instead of paying Virtuozzo $1k+ for licensing.
I found this article:
[url]
I installed this on Centos 5 server. I got one tiny problem now:
[root@server ~]# virt-manager
Traceback (most recent call last):
File "/usr/share/virt-manager/virt-manager.py", line 304, in ?
main()
File "/usr/share/virt-manager/virt-manager.py", line 224, in main
import gtk
File "/usr/lib/python2.4/site-packages/gtk-2.0/gtk/__init__.py", line 76, in ?
_init()
File "/usr/lib/python2.4/site-packages/gtk-2.0/gtk/__init__.py", line 64, in _init
_gtk.init_check()
RuntimeError: could not open display
Also, that tutorial doesn't explain how to create actual vps
Anyone care to explain? Isn't there some web gui available for xen too?
I am trying to debug a dns problem.
What I need is for you to do a dns lookup and report:
1) the address that was returned, the last octect will do.
2) the area of the world you queried from
The answers *will* vary because the dns server is geolocation enabled. Or, at least they *should* vary. But, someone claims its buggered.
The host name to lookup is:
ssl.dnsmasq.com