In my utter, unprofessional, unclinical, uneducated opinion... I think there is a reason for the feelings you and your friends/loves have had around your dad. I've had feelings when meeting people that there was something not right about them. I tried ignoring that feeling once and ended up regretting it later. I don't have any advice on what to do about it, but I don't think you're insane.

At least not for that reason...

__________________
Put me on wheels and I'll turn tricks.

Clever? Nah, I ran out of that years ago. But if you find this, let me know, k?
"The road goes ever on..." ~ Tolkien

In memory of my friend skip...
Go then, there are other worlds than these