With regard to holiday in general, strictly speaking, employers have the right to tell employees when to take annual leave, but they must give notice of two days for every day they want employees to take ie, if they want you to take four days, notice they must give 8 days’ notice. However, this is quite uncommon and not widely requested.
I would advise you to check your employee handbook/ contract to see if there are any restrictions on taking annual leave. It sounds like they are suggesting you take annual leave more fairly rather than enforcing it. If you feel they are forcing you to take annual leave then or making it difficult for you to take it when you want, I would suggest speaking to your HR department. I can understand a friendly reminder to maybe space out your annual leave, but if they are planning on enforcing this, they need to discuss this with you in more detail. As you have been taking your annual leave like this for a while, then you can also state that this has become an implied term and you will need notice of this change. I would suggest you speak with HR as well as your manager to try and come to a compromise.