Well if you think about it, they would have to be a pretty sound sleeper not to know. But if you caught them just right, you maybe creating a wet dream for them which would be neat to hear about once they wake up.
A couple in a healthy relationship should probably have a presumption of consent. If not, I'd suspect the relationship is either not strong and healthy or is rather Puritan or suffers from sexual hang-ups.
For instance, with my wife and I, if one of us decided to wake the other up with fondling, fingering, oral sex, etc. it is 100% permissible. If either of us didn't feel like it at the time, we'd just say something like "not now" or "let me sleep a while longer" and the other will stop. But 99% of the time it's 'a go for it' in a big way, and actually, she initiates such activity about 75% of the time.
But hey, a new relationship, a new acquaintance, or someone just crashing in your bed, or something like that, just don't do it. And don't go down the hall trying it with your roommate. Use common sense and don't be stupid.
*******
If I were to guess, based on my experiences with one-nighters, ex-girlfriends, and my wife. I'd guess that women are much more likely to fondle their partner to waking than men.