Sibling rivalry is no joke, and when a younger sibling outpaces an older sibling in any way it can be especially difficult for that older child to deal with it; this might be why George was known to do things like throw golf balls at Sheldon’s head and actually make the shot, leaving a permanent dent in his little brother’s incredibly intelligent dome.
While neither of Sheldon’s siblings were very nice to him during their childhood, George seemed to have a much harder time dealing with and understanding Sheldon.
George is widely regarded as not being very bright, but he surely understood that Sheldon was way smarter than him and special in a way that George would never come close to.