Marriage is based on sacred commitment, love, respect, honour and tolerance. Each of these features is a fundamental pillar of marriage. Anything that disturbs the internal cohesion of a spousal relationship must be avoided.
This married man is doing wrong by neglecting his wife, and this could lead to oppression. Should he have issues with his wife, they need to be addressed through communication, or through counceling.
If it is purely him pursuing his lust, then this will destroy the harmony of his home, and hence it will get him nowhere.
This clip speaks about how to deal with a promiscuous husband: