The the thing you should do is send the kids to the grandparents for the weekend. Go to a hotel in the next town, and only come home if one of the children is in dire straits.
You need to learn to be lovers not just parents.