I use a dremel with spiral cut bit. Pretty sure a stone or cylindrical rasp will work as. Only needs to be as wide as the set screw and a millimeter or so deep.