You should be OK in the water-its the air temp you gotta worry about. Keep paddling, stay prone as much as you can and hopefully you won't have to wait too long between sets! Also bring a thick towel or old blanket for the beach. Now hit it!
I was cold last Friday in my 3/2 in NJ, but thats cause the air and wind was chilly. There were a few guys in 4/3s and some guys had booties on. There was one dude in boardies and a wetsuit top, hes a beast though I see him out there all the time and hes got plenty of body mass to keep him warm haha.
Youll be chilly but if its all you got then its better to be chilly then not surf at all!