I agree with you. I am a little self-conscious about clothing - not that I would dislike it, but that growing up poor, I don't have a lot of experience with "the good stuff."
But I do enjoy dressing for special occasions.
The most fun I think I had, in dressing, was for the street parades at Fantasy Fest in Key West, Florida.