VLAN Routed IP's

So I have a bunch of IP's that are routed a VLAN a bunch of machines are on which is great. They can be put on any machine, but the problem right now is it is taking up to 4 hours for them to route from one machine to another (I am doing it one IP at a time not by blocks)

The process of adding and removing ip's is being done through cPanel's WHM. The reasoning for this is due to ARP on the router for the specific IP. This is a big issue if you're looking to migrate IP's from one machine to another that contain customers this is a long wait.

Now from the datacenter I was told you're going to need to wait the 4 hours for the refresh. Now what's strange is I was also told virtuozo apparently can swap ip's to other boxes without issues because it apparently it has a mechanism in place to release the ip's from the router/switch by communicating to it so the old arp is gone. I also asked someone using OpenVZ and they said theirs was instant as well to swap between boxes.

So is there a special command I should be issueing to get the router/switch to drop the IP instantly? Or any suggestions on how to get this to be more instant than ~4 hours for an IP to route to the correct machine.