JT, I wear the gloves because a little strand of carbon fiber under the skin is painful and very hard to get out. It happened to me once and I had to dig with the hobby knife to finally be able to get it with tweezers. The gloves give me some protection against that. The gloves I use for this are nitrile gloves because they are thin and tear easily. I don't use latex or urathane gloves because they are too thick and tough and stretchy. They would pull my finger on the bit instead of just tearing.
Also, it is a small bit with fine teeth on it. If your finger touches it, it will take off some skin but you need to push and keep the finger on the bit for it to dig deep enough for it to bleed. Would I wear gloves and get my fingers that close to a saw blade? Never. I know my tools, I respect the damage they can do. So don't be yelling at me no more