Seeking an affair when your marriage is on the rocks--rather than, I dunno, using your words like grownups and talking it out and seeing a counselor to see if the relationship can be saved or if it's mutually time to go your separate ways--is pretty jerky behvaiour by itself. Not outright cruel, but not very nice either.
Divorcing a spouse for the person you had an affair with is a bit further up on the Asshole Scale.
Divorcing a SICK spouse suffering from a MAJOR, LONG-TERM, DEBILITATING, LIFE-CHANGING ILLNESS for the person with whom you've been having an affair for a third of your marriage? And doing it TWICE?
There is no level of douchebaggery on any known scale comparable to that.