Oh shoot! I forgot about ducks! And it seems there are a few more. I was 97% correct.

So, what about that 3 percent? According to Patricia Brennan, something of an expert when it comes to bird genitalia, penises have been retained in only the most basal avian species. That includes the Paleognathes, which is your ostriches, emus, rheas, and tinamous, as well as most of the Galloanseriformes within the Neognaths, our ducks and fowl.