In Season 10 of The Big Bang Theory, Sheldon confessed to Penny the reason why he always knocked multiple times before entering a room.
Sheldon is only 11 in Season 4, so it’s possible fans just witnessed the beginning of George Sr.’s illicit affair that Sheldon eventually discovered.
Sheldon said that when he was 13, he walked in on his father with another woman.
At the very least, we saw that George and Mary have clear problems in their marriage, and George doesn’t mind the company of other women when his wife isn’t around.