There might be something wrong on leave policy section. Let me make it clear to you 🙂
Did you check these options “All Departments” and “All Designations” while adding a leave policy?
If you checked these options, then you can assign this policy to any employee or to any department you want.
But, somehow all the employees/departments/designations are assigned to all the leave policies. You can have the check leave entitlements again.
** Can you make sure that you did not check “assign to multiple employees”?
Hope this was helpful. Let me know update 🙂