I've been pondering something for a long time here, but never really knew the right question to ask, or how to word it... so I never even know what to search for. So I honestly have no idea what I'm asking or if this is even under the right topic. What I'm looking for is answers, links, tutorials, or just some words I can search for to help me do the following:
I have a MySQL database filled with accounts. People have to pay to get these accounts. All the accounts are set to expire on different days. I need to send an e-mail out to accounts, for example, to let them know they're expiring, say, a month before this will occur. Right now the only way I know to do this is to have MySQL queries on a PHP page that, every time the page is loaded, runs through all the accounts, checks their expirations, and sends emails to those that need it. I know there has to be a way to have this happen automatically without having to load a PHP page every day, for example.
I'm currently hosting all of my stuff with my own Wild West Reselling Account, i.e., InnoDomains.com.
Problem is I'm running multiple Wordpress Installations and a few websites that have some pretty intense MySQL work going on in the background, and I use about 250GB per month in bandwidth.
My Account gets the 503 downtime errors because according to Customer Service I cannot run more than 50 MySQL queries in one second. Well, that's not much of a challenge because a lengthy wordpress page runs about 90 queries and loads in 2-3 seconds. Time this by three including the other websites and a couple of shopping stores, and I'm easily doing more than 100-150 queries a second more than several times a day, thereby seeing that nasty, irritating error.
My other option is to split up all the websites over basic $ 3.95 per month accounts. I don't want to switch to a dedicated server yet because I'm not really making a whole lot of money (about $ 30 per month). Another thing I could do is move my MySQL testing to a local machine, and that may actually reduce the load on the damn server.
Are there any hosts out there that will allow me to do more than 100 MySQL queries per second? If so, are there any that are any good?
This has happened twice now. I have my wait_timeout variable set to 60 seconds, but occasionally my database gets overloaded with "locked" queries. Here's an example of my processlist:
This continues on with locked queries all the way up to my limit of 450 connections, then the whole server goes down. Note the time spent processing these queries, 35483 seconds and counting! Surely the wait_timeout is supposed to prevent this from happening?!
Also, this appears to start with queries in the State "statistics". Then the State for the following queries becomes "locked". Does anyone know what these mean? Can I get them to timeout normally? Queries which never timeout are guaranteed to bring your server down eventually, surely this is not the desired behaviour.
I know about the general log but want to just log queries of a specific user. Our database does 400 queries/s on a slow day, so I think it would be a pain to grep though a huge log like that. And, im not sure if it would cause any preformance issues.
What I did here was create /var/log/mysqld.slow.log for queries that are taking one second or longer. After around 36 hours, here is what I came up with. Since the file size of /var/log/mysqld.slow.log was 219KB, posting it in its entirety it not practical. Therefore, I used a program, the perl script known as mysql_slow_log_parser found on www.retards.org, to parse the mysqld.slow.log to make it easier to interpret and read. The result was that the original filesize of 219KB was converted to the smaller size of 14KB....
I have fairly a large web site that has a forum and a torrent tracker.
Currently MySQL server is handling about 150 queries an avarage per second. Here is the server spec:
Core2Duo 2.66Ghz 4Gb RAM 320GB SATA 7200RPM (Server provider does not have 1.5K RPM nor 1.0k RPM) 100Mbit Connection (servers on the same switch and the switch does not have 1Gbit port) MySQL Version: 5.0.51a
I had Master-Master Replication setup with forum running on one and the tracker running on the other. Although this has been working for about few days, we started seeing lags in the replication process. After a week, there is a major lag and the changes made on one of the servers takes about 5 hours to appear on the another. So, this doesn't work.
What would be the other ways of splitting MySQL queries concerning the same database?
While I was researching, I read about MySQL Cluster with database storage engine being NDB.
But, let's say that there is a power failure on both the nodes at the same time, then I would lose the whole database as the database is stored on the memory correct? I would not like to take that chance either, but if this is faster then replication method then maybe I will concider.
I thought about editing the forum coding to make all queries that concerns the tracker to go in to, say server B (with forum's primary MySQL server being Server A), and make the tracker use server B as MySQL backend, but it seemed like a heavy work so that will be the last choice.
I've got 25 domains on a Virtuozzo/Plesk8.6/CentOS5 VPS. Each domain has one up-to-date install of WordPress, most have very little traffic (average 200mb per month), maybe 2 domains get 5-7gb traffic per month.
I monitor port 80 connections and rarely see more than 10 at a time. That should in my opinion be no problem at all for a VPS with 768mb guaranteed ram and 2.4ghz cpu. I've got 30gb hard drive spare too.
But.... about 8 or 10 times a day it grinds to a complete halt: server load at 500-1000%, sites timing out, plesk takes 3mins to load, often I can't even connect with SSH, and the plesk web server, apache
80 seconds sounds like a huge amount of time for a MySQL insert to me! Does anyone know if this is likely to be the cause of my trouble? Some problem with Plesk and the database? Or could it be something else?
We host our DNS with DNSmadeeasy.com. We noticed that the number of DNS requests is too high. So we decided to contact their support and ask them for log file, so we can determine
what's going on.
Reply we got from them is the following:
We researched your queries and we found that many of them were coming from the SoftLayer resolving name servers. We phone their admins to report the abuse and they said that the queries were infact coming from your own server with them.
We will not charge for any overage on the queries this month so you can research what on your server is causing these queries. But we will provide you with logs of about 5% of our your traffic.
here is the example of the line from log file sent by them:
May 10 17:15:48 t145 named[29945]: client 66.228.118.72#52529: query: host-196.218.57.145.tedata.net.mydomain.com IN AAAA -E May 10 17:15:48 t145 named[29945]: client 66.228.118.72#52529: query: afontenayssb-151-1-40-153.w83-114.abo.wanadoo.fr.mydomain.com IN AAAA -E
I changed the real domain name to "mydomain.com"
I checked the apache access log and found these hosts there in exact order as they are in log file sent by dnsmadeeasy.
What is strange is that why is my server making back these requests ? Also these requests are bad because these are not valid domain names:
e.g. should be "host-196.218.57.145.tedata.net" instead of "host-196.218.57.145.tedata.net.mydomain.com"
Seems that it includes ".mydomain.com" to the end of every request.
Is there a way to discover what is making those DNS queries?
Is it possible, through some script or otherwise, to send the traffic to certain/chosen domains to a completely different set of nameservers.? Preferably without having too much server load?
I already use a domain catchall, so no NS records are actually registered on the server.
I am checking DNS Report for a domain and in nameservers records I am getting following:
Recursive Queries
I could use the nameservers listed below to performe recursive queries. It may be that I am wrong but the chances of that are low. You should not have nameservers that allow recursive queries as this will allow almost anyone to use your nameservers and can cause problems. Problem record(s) are: xxx.xxx.xxx.xxx
I am using Windows 2003 Standard and Microsoft DNS Server. Any ideas what is this all about and how to fix it?
I have just upgraded my website, and set 301 redirects for all old pages.
But on the old website I used a search engine, and in Google Webmaster there are loads of 404 pages with search queries. They look like this: [URL] ....
Simply redirecting search.php doesn't work, so I guess I need to use a wildcard?
I got a dedicated server which hosted 45 sites, one of those sites contains a vbulletin forum application since january 2007 , but from two weeks i found that there is a high amount of Queries from the this database which leads the server load to be always in RED and exceeds 200 or 300! and the memory is totally consumed!
i tried alot to solve this problem as : - reinstall the vb. - change the vb path to another ( considering it as attack) - installing BFD , APF , ( considering it as attack ) - upgrading to apache 2.2.6 , PHP 5.2.5 , MySQL 5.0.24 ( considering it as a bug)
this is the sample of queries.
Code: 46978hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep40 NULL46980hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep38 NULL46990hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep38 NULL47034hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep30 NULL47086hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep23 NULL47092hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep19 NULL47095hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep23 NULL47099hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep23 NULL47122hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep24 NULL47124hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep24 NULL47126hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep24 NULL47127hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep24 NULL47142hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery22Sending dataSELECT * FROM datastore WHERE title IN ('','options','bitfields','attachmentcache','forumcache47143hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery0Writing to netSELECT moderator.*, user.username, IF(user.displaygroupid = 0, user.usergroupid, user.displaygroup47144hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery22LockedSELECT * FROM datastore WHERE title IN ('','options','bitfields','attachmentcache','forumcache47145hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery22LockedSELECT * FROM datastore WHERE title IN ('','options','bitfields','attachmentcache','forumcache47147hakawyc_SKJI8jjlocalhosthakawyc_froumvbQuery22Writing to netSELECT * FROM datastore WHERE title IN ('','options','bitfields','attachmentcache','forumcache47151hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep16 NULL47159hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep17 NULL47166hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep15 NULL47177hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep0 NULL47180hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep16 NULL47211hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep10 NULL47251hakawyc_SKJI8jjlocalhosthakawyc_froumvbSleep0 NULL47283rootlocalhostNULLQuery0NULLSHOW PROCESSLIST Once i disabled the database the server load became normal again
another example for Queries :
Code: 117445 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 49 NULL 117497 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 42 NULL 117667 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 24 NULL SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117790 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 7 NULL 117791 jarabhar_vb localhost jarabhar_vb Sleep 7 NULL 117793 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 7 NULL 117795 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 8 NULL 117798 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 8 NULL 117799 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 8 NULL 117802 unauthenticated user localhost NULL Connect NULL Reading from net NULL 117803 unauthenticated user localhost NULL Connect NULL Reading from net NULL 117804 unauthenticated user localhost NULL Connect NULL Reading from net NULL 117805 unauthenticated user localhost NULL Connect NULL Reading from net NULL 117806 unauthenticated user localhost NULL Connect NULL Reading from net NULL 117807 unauthenticated user localhost NULL Connect NULL Reading from net NULL 117808 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 9 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117818 jarabhar_vb localhost jarabhar_vb Query 8 Sorting result SELECT post.postidntttFROM post AS postntttntttWHERE post.threadid = 52572nttttAND post.visible = 1n 117821 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 8 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117829 jarabhar_vb localhost jarabhar_vb Query 8 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117832 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 statistics SELECT post.*,ntttIF(visible = 2, 1, 0) AS isdeleted,ntttnnttteditlog.userid AS edit_userid, editlog 117833 jarabhar_vb localhost jarabhar_vb Query 7 statistics SELECT IF(visible = 2, 1, 0) AS isdeleted,ntttnntttNOT ISNULL(subscribethread.subscribethreadid) AS 117834 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 statistics SELECT languageid,ntttphrasegroup_holiday AS phrasegroup_holiday,ntttphrasegroup_global AS phrasegro 117835 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Writing to net SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117836 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117840 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT forum.forumid, lastpost, lastposter, lastthread, lastthreadid, lasticonid, threadcount, reply 117841 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117842 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117843 hakawyc_SKJI8jj localhost hakawyc_froumvb Query 7 Sending data SELECT *ntttFROM datastorentttWHERE title IN ('','options','bitfields','attachmentcache','forumcache 117845 hakawyc_SKJI8jj localhost hakawyc_froumvb Sleep 2 NULL 117846 hakawyc_SKJI8jj localhost hakawyc_froumvb Killed 0 NULL NULL 117847 unauthenticated user localhost NULL Connect NULL Reading from net NULL 117848 hakawyc_SKJI8jj localhost NULL Connect NULL Writing to net NULL 117849 unauthenticated user connecting host NULL Connect NULL login NULL 117850 unauthenticated user connecting host NULL Connect NULL login NULL 117851 unauthenticated user connecting host NULL Connect NULL login NULL 117852 unauthenticated user connecting host NULL Connect NULL login NULL 117853 unauthenticated user connecting host NULL Connect NULL login NULL 117854 unauthenticated user connecting host NULL Connect NULL login NULL 117855 unauthenticated user connecting host NULL Connect NULL login NULL 117856 unauthenticated user connecting host NULL Connect NULL login NULL 117857 unauthenticated user connecting host NULL Connect NULL login NULL
I am running a SQL test server here that is piping HUGE amounts of data for a test project I am running.
Approximately 30 queries a second - constantly. That's over 2.5M queries a day -- so we are talking big (I think?)
I am currently running this test on a
AMD Phenom 9300, 4GB RAM - SATA 500GB HDD and I am running MySQL 5.0.51a i386 on CentOS
I have programmed a process control for our applications purpose - basically it controls the launching of our SQL intense applications, and stops launching when the Load is greater than 2.5
I have plans to optimize the number of queries (I will build in a cache to some of the applications - and run INSERT statements all together) however I am looking for SQL tweaks that will improve performance. Would running the 64bit version work better?
im running a few blogs on a shared hosting account (steadfast) and i noticed the 18 sql queries being generated by a fresh install of wordpress 2.7 are taking 0.246 seconds to execute. <!-- 18 queries. 0.246 seconds. -->
considering this is a shared hosting server with over 800+ more domains on it, is the performance better or worse when compared to other shared hosting providers ? archim3des
For those who run their own DNS servers I am interested in what DNS software your running on BSD or Linux platforms and on average how many DNS queries per second it serves on what hardware specification?
I am trying to figure out how to do the following:
Say I have a URL like this: www.myreallylongsiteurl.com and I want it to automatically redirect to www.myRegularURL.com.
How would I go about doing this? Do I actually have to setup a whole new site in IIS and have a page there that just does a redirect or can I set up some sort of alias for this?
Also, my customer would like me to set it up so that myRegularURL.com redirects automatically to www.myRegularURL.com. they don't want to have to do the www in the url. Do I do this the same way as above?
I have searched for setting up automatic secondary DNS, but couldn't find anything, if I missed it, please post a link
I have 2 VPS's one running cPanel/WHM with about 15 low traffic sites on it. The other is a very slow spec VPS, hopefully just for DNS services.
I only have 1 domain server running currently, this is on the same server as the all the websites, what I want is a secondary DNS mirroring that of the DNS on my cPanel VPS, automaitically....
I was thinking of writting a little script to scp all the .db files in /var/named/ to my secondary server, then login to the secondary server and restart BIND?
The web host I use for personal web hosting (non-work-related) recently renewed my annual hosting package to my credit card automatically. The invoice was generated at around 11:45PM and I received an email reminder. By 12:03AM, I had been billed the full amount, before I even saw the 11:45PM reminder email.
Either from a client or provider standpoint, what is your take on automatic annual renewals? With or without advance warning?
In my case, it was billed to a card I no longer use, so I had to make a payment to that card that I would not have otherwise made. In other words, no serious harm done, but it was a hassle nonetheless.
I do not have a specific take either way on automatic annual renewals, so I am wondering what the community thinks.
I've been searching for a tool to automate website backups for me, that is to keep regular copies of files and databases in case of unexpected data loss. So far I found Site Vault that handles both FTP and MySQL backups at the same time. Does anyone know good alternatives to consider?
I have around 750 megabytes of data on a Windows 2003 server, and I would like to find an off-site / web service that I can use to back this data up incrementally at least nightly. Nightly incrementals would probably be 1-25MB in size. Full backups could run every other week or monthly, as the update rate is fairly low.
Ideally, I'd like to find some software to make this set-it and forget-it. Of course, periodic audits of the backup would be performed, but I don't want something I have to nanny constantly.
As I said, this is a Windows server, so a service would be ideal. A scheduled task would be just as good, I suppose.
I have 2 different shared hosting accounts with 2 different hosts. On 1 hosting account I can install Wordpress plugins by clicking on the "install" link and everything is done, all I have to do is activate the plugin and I'm in business. On my other hosting account I signed up for yesterday, when I try to install a plugin I'm taken to a "Connection Information" screen. It says, "To perform the requested action, connection information is required" and there's a box for the hostname, username, password, and connection type (FTP or FTPS).
I also noticed that when I search for a plugin, the rating stars are all white (no yellow) on the hosting account that asks for the FTP info.
Both of these accounts were installed with Fantastico. Any ideas why my new hosting account acts this way?
What's the best way to set up an automatic file backup to Amazon's S3 service? I have a Linux cpanel VPS. I have minimal VPS administration knowledge but at least the VPS is managed by a good team
Is there any way to do incremental backups to save on the bandwidth charges?
(a full backup for me would be about 15 gigs each time)
Is automatic renewal of a colocation contract for a full year enforceable? I've been with a place for over two years, never got a notice that my contract was being renewed, never got a special price for renewing for an entire year, but as I see now, I'm going to be challenged to pay 100% of the remaining months of a contract I thought I was paying by the month after the end of the first year.
In real estate, most tenants have to sign something that renews their contract for a year, I thought. If I had received notice of renewal at the end of the contract periods, I supposed I would have been more aware that the lease was for a full year each time.
I had no idea that this was standard and, of course, no idea this renewal was buried on page 6 of a 15 page contract. This starts to resemble a health club contract from hell.
Oh, and I just love the early termination fee. It consists of the remainder of the months' lease fees. Like as if I still had the server connected.
I have a virtual dedicated server which i currently host a couple of clients websites on, the control panel i am using is Simple Control Panel.
What i would like is an automated program or script that automatically sets up a users FTP access and creates a directory with a mysql database - I also require it to bill the customer on a monthly basis and if no payment is received from the customer the account then goes offline (but is not deleted)
how i can set this up and what is the best thing to use.
