Ooh, calling out the fixies!
Like all that has been said before, there are many different factors coming into play here. For example, I ride a frame that would be a tad to small for someone my height, that is because my torso isn't that long. I have long legs, hence my seat is jacked pretty high (on my tarck bike nonetheless) which allows for a pretty comfortable ride, in my opinion.
It has been previously said that aerodynamics play in to these things as well. With drop bars, you have a more aggressive stance, and I find that it is more comfortable with saddle adjustment that is a tad bit higher than for normal flat bars.