The loss is from friction in both the bearings and gears. Then the energy need to spin up the extra mass.

If keeping the overall drive ratio the same. The main thing I see you gaining is some lightness (which is always good) and a slight increase in acceleration because it will spin up a little faster.

Is it worth it... That depends on the rest of the truck. There are other areas I would look at to gain some speed. If there are other reasons behind the build like you want a clean look, adding lightness, or just like to have a unique truck then go for it. I have more fun building so, I would do it just because I could.