I used to struggle with things like this at work. There seemed to be endless kitties, with demands to add £2 to them all. This, at a time when I was so broke that I had no social life because I felt I couldn't afford the bus fair, or to buy a drink when I got there.
I was very badly paid, 'tight' as people would say, and had other priorities for my pennies.
But saying that, I didn't refuse, because the pressure was incredible. I felt I had to pay up. So I don't have the delightful experience of not caring and saying no.
Perhaps that's where being autistic is a perk.