Blocking IP Addresses At Server Level

I've been carrying some weird hours lately, so I'm able to see activity on my sites that I normally don't see.

Certain IPs are trying to diddle their things into my server and need the boot. It's not consistent (ie not happening every 5, 10 mins)... it's periodically throughout the months. I'll see an IP I blocked 2 months ago just randomly show up at 4:30am and try accessing the same files it was probing during it's last visit. Assuming this is just some sort of bot, can I block it permanently?

I know APF has a collection system that purges an IP list to keep it from bloating, and I had PSM do some hardening so I'm not entirely sure about the workings of APF firewall. So far my IP blocks are blank (which is a good sign!), but I'd like to add some nuisances to it, to keep their crap from appearing in my error logs anymore as "Denied by Server Configuration"

My question is: Can I block people at server level permanently? I do not want their IP being taken out with the purge list that comes by every so often.