Geez, that's a toughie.
I know myself from my experiences, that I myself hold back a bit more when speaking with men. I cannot help it. Almost like they have failed me before even meeting me. I know it's not their fault, yet I can't seem to stop it.
I want to suggest doing the whole pros and cons thing but I think you have already got the two major pros and cons. And, I see that they are kinda tied right now. In my opinion, I wouldn't know what to do either. I'm sorry that it doesn't help.
Do you have a stronger view on one than the other?