Batteries in Series Adapter

^^^^ Def do that, get a pair of pacs that have voltage, check them alone so you know what each has. Connect the with your adapter and the voltage should be the sum of what you just measured. I built one looks a lot like yours and checked it this way to verify it gave me the sum of the 2 pacs voltage before I tried using it.

Sounds like you may have one leg switched??