I only send or give Christmas cards to people that I care deeply about. I don't have a huge list, and I don't just add people at random. When I do send a card, it is heart-felt, and I typically write a short note inside. If I receive a card from someone that I didn't expect to receive one from, I appreciate it, but I don't feel compelled to return the favor - and I don't feel guilty if I don't send/give one in return.

So, to answer your question, IMO - you shouldn't feel guilty, and if you feel that you truly want to wish someone a Happy Holiday from your heart, do it.

__________________
A blessed thing it is for any man or woman to have a friend, one human soul whom we can trust utterly, who knows the best and worst of us, and who loves us in spite of all our faults.