I was just having a similar conversation yesterday when it was fun and clean in the morning, but I was at work. And the last few times it was smallish but still fun looking I had other commitments. So it's been a couple weeks since I've been in the water, and I'm starting to get angry when I miss a day. You expect to miss a day here and there, even when you live right on the beach. But when you miss 3 or 4 days in a row when you know you could have at least gotten out and caught a few fun ones really starts to pis you off.
I guess it all depends on what your goal is or how long it's been since you've been in the water with your board. Sometimes just getting in the water and paddling around to get that work out in can be beneficial, i find it makes the good days seem that much more sweet, but if you are very use to epic conditions year round I could definitely see how one could turn their nose up at some wind slop, because you know there is a good day just around the corner that you'll be able to jump on. But like most, when a flat spell comes along a small day of wind slop can look worthy of my time.
I've had some really good days when if you read the surf reports you would never dream of going out. I wont paddle out if its a lake but if there's the slightest nugget to he had and I'm there I got nothing to lose.