Personally, how sexy a man is has to do more with horny he is, and how willing he is to explore and share his fantasies. While looks maybe the first thing to attract, if a man is so worried about looking "all buff", he probably will never have enough time to please a woman.