Yeah I swim every day and always bring an inhaler with me. On my winter wetsuit (hooded chest zip) it gets tucked inside the chest zipper in the neck/shoulder area. On my 3mil fullsuit it gets tucked in the little "utility" pocket. In my trunks it just goes in the side pocket (velcro'ed shut).
The world of medicine said the same think in the early 90s with respect to the leukotriene pathway (Singulair). To the original poster, you DO need to be careful--if you look at the mortality rate in asthmatics, there are deaths even in "mild asthma", due to the disease, not "other" causes. Usually due to an "attack".
As for carrying your inhaler in the water, and I assume you are referring to albuterol, I would still try to insure keeping it dry in a waterproof bag. Salt water is extremely corrosive; look at the container which is the inhaler- metal. You know as well as I do, you will not change the inhaler with enough frequency to override any rust concerns, so that when you need to use it out in the water, the likelihood of failure is higher than if you kept it dry. You need the odds to be on your side.
Are you on an inhaled steroid? Fluticasone, or budesonide or similar?? If not, I suggest, because you do want to continue surfing, to have your doc place you on one. They reduce the inflammation which, as you know, is the reason airways slowly close off.
Stay well, and good luck.