Yea drill press would be the best way but I dont have a drill press and dont plan to buy one as I dont have any room to keep it.
I know that you said you wernt planning on buying a drill press but here is a link to some inexpensive and smaller styles, there is also a drill guide where you could use a 3/8" drill and get a near drill press accuracy