I was pretty much thinking this exact thing before you posted. I have had this exact vibe in south jersey....5-6 guys making zero eye contact with outsiders sitting as deep on the end of the jetty as is possible and still have a chance at making the wave. Ignoring you if they don't know you and paddling for every wave as if you weren't there. Completely intended to drive you off to another spot without (as you say) resorting to threats. There are a couple of spots on Delmarva that get that way.