For me it depends...on the face of it, being a smoker is a big off, but when it comes to knowing a person it pales into insignificance.
I have lusted over a few smokers in my time despite their habit, and find myself totally head over heels with a girl who smokes but I wouldn't change her for the world.
For me as far as sex goes, it's alot more then the physical act, but the connection with the person, and that raises above habits, good or bad!