Sure, 100% your call, of course.
Look at the picture, though, the amount of material you need to remove is minimal and the tools I mentioned (especially the osc tool) will get that done in a few minutes.
Although, if you simply enlarge the screw holes, it's going to get snugged down tight when...