It depends on what you're comfortable with. If its windy, use heavier BBs. You just gotta compensate for the wind. If you still can't aim consistantly and you're not having fun then call it quits.
I don't think theres a set number for when not to play.
Edit: You answered your own question...