I wear a mask whenever I go inside a public space. And I won't go inside a public space unless it's pretty quiet in there anyway.
The long covid thing is concerning, and I'm very keen not to end up with it. But the people who have had it, do seem to mostly get over it eventually. I'm not convinced it necessarily leads to premature death, but it does increase risk of heart attack, I read somewhere. So some people get unlucky. Most people recover from long covid after months (not years), but again, not everyone. And who knows what undiscovered damage it's done to their organs in the mean time.
I think you're sensible. People think we're weird too, but I don't really care. It's a pandemic. They're the crazy ones. Stay safe.