Layering makes it harder. I would go with a nice 3/2 with glued and taped seams, zipperless even better.
Maybe a squid lid of the wind is kicking. You lose 10-15% of body out the top of your head.
If you do layer, go with hotlineonline.com - their thermal shirts and shorts are awesome!
If you plan on going north, NC or something, the 4/3.