Would you rather surf a foot smaller of a wave with only a few guys out, or surf a wave that is a little bigger but is 3 times as crowded
Hopefully i'm not the only person that would rather have the first option.
Around here, "a foot smaller" for the next three months means knee high slop, as opposed to waist high and ok. I usually pick the waist high waves and just deal with the fact that crowds happen, and just try to get in the water early and avoid peak times.
The other thing is....more times than not i have found that with a little bit of effort its possible to find relatively uncrowded waves without sacrificing the size and quality. People tend to flock to the known spots, cameras, and easily checked spots. That fact can be used to your advantage. Even in the crowded season.