You could probably setup and configure plug_attack to block the offending IPs. See in particular the fail2ban rule function, with which you can fail specific requests, and temporarily ban IPs that keep trying them.
Admittedly, I do not have experience with it, but it is very similar to Ruby’s rack-attack, that offers the same features in Ruby applications, and that I used in the past for these kind of cases.
