All you have to do is rince your suit with fresh water and hang it to dry after surfing. They only smell because you allow it to stay wet and full of piss which is a great environment for bateria and other **** to grow.
The simple answer is yes, it matters. Salt water and bacteria from your body are corrosive, and the life of your suit will be shortened if you don't clean it out with fresh water.
That being said, if you want to save the 15 mins then shower and clean the suit when you get home from work. Having the suit sit around for 10 hrs before cleaning isn't ideal, but certainly better than nothing.