[BrianG]

There is nothing wrong with what you are doing. And yes, it makes sense that the servo needs to have the same ground reference. After all, without a common ground, the servo doesn't have a reference point...