The Governor wasn’t ever interested in anyone’s happiness to the point where viewers could enjoy it, and while David Morrissey rocked out the part, he isn’t in the same league as Negan.
He tells jokes (that he thinks are funny), and while it’s all part of his mental domination, it’s so fun to watch.
Plus, Morgan has already said who would win in a fight, and we trust him.
Even when he’s telling that terrible story to Daryl about Dwight and Sherry, he’s smirking and acting as if it’s a good time, knowing full well it isn’t for anyone else.