Instead of starting with a concrete number it’s a better idea to make the quick and dirty list first. Chances are real good that should be your final list. Unfortunately, you have to then match up the number of bridesmaids on the other side of the aisle with your groomsmen...it starts getting complicated.
Guidelines for choosing your groomsmen:
Unless you’ve got an obvious issue with them that would turn your party into a war, always include your brothers and hers as well. (Unless there are a large number of brothers.)
Parties lose their significance above eight. Try to stick to this number and urge your bride to do the same.
Apply the all-or-none rule for friends that run in the same circles so you don’t have to play favorites.
If you’re having an especially hard time, consider sticking just to a best man.
There is no rule that says the bride and groom must have an equal number of attendants, but it is preferrable that they do for the pagentry of it all.