Ah the dreaded tailor tax -- the hidden cost of being a well-dressed man. It can be a pain, but approaching this reality with the right mindset is imperative to looking your best!
"Fit is king," they say! While contextual appropriateness supersedes fit, clothes that create flattering contours are the most aesthetically impactful element of style.
Put simply, clothes that fit you well make you look good. Clothes that fit you poorly make you look bad. And the only way to get clothes to fit you is to have them tailored on either the front end or back end -- by purchasing custom clothes or having off the rack garments altered.
Even the most luxurious fabrics with beautiful craftsmanship and handwork will detract from your overall appearance if they're ill-fitting. Flattering contours will make you appear stronger, more fastidious, and more capable. Any man looking to leverage style to amplify social efficacy must make fit his utmost concern. And it's for that reason that the tailoring tax must be calculated into one's overall budget.
$150 suit vs $1500 suit; the difference is in the fit and tailoring!
If you have $500 for a suit, don't buy a $500 suit. Buy a $350 suit and save $150 for tailoring. If you've $100 for chinos, buy the $60 pants and save $40 for tailoring, rather than purchasing $98 chinos that don't fit you anyway,
Thinking about it pragmatically: the clothes don't exist in a vacuum as pieces of art to be admired for their material or craftsmanship alone. They are there to make you look good and allow you to convey yourself flatteringly. That should be your foremost concern.
It's definitely worth spending money on quality and craftsmanship, as they will correspond to comfort and longevity. Yet it must be reiterated ad-nauseum that those aspects of your garment are only appreciated by you. It's the fit of your clothes that's perceived by everyone else. That is to say that uncomfortable, papery clothes that fit you well are going to serve your appearance better -- for as long as they hold up -- than luxurious, silky, durable garments that don't.
Not that you should have to choose.
Buy the best quality clothes that you can afford, while keeping enough money leftover to ensure they fit. Find a tailor that you can trust to do the alterations you need, build a relationship with them, and expect to use that service with every new purchase of dress clothes.
Sometimes you will be lucky enough to find something that fits off the rack, but that will be the rare exception rather than the norm. Most of the time, you'll be shooting for something that fits in the areas that can't be reliably altered, and then bringing it to your tailor afterward.
Or just go straight to a trusted custom clothier.
Either way, the tailoring tax must be paid if you want your appearance to do you justice. Accept it, embrace it, and take solace knowing that it's worth it every single time.