They just know the show wasn’t the same, and the most obvious evidence of that was the female Doctor running around on the screen and calling her companions “fam.”
That being said, it can be hard to disassociate Jodie Whittaker from the situation when she represented the biggest change in Season 11.
For that reason, I feel that some backlash directed at Whittaker isn’t so much about her performance or portrayal, but rather the overall change in the franchise she came to be the face of.
The common fan may not be up on the whole changes with the showrunner, theme, visual effects, and the general concept of a soft reboot to try and invite a new audience into the show.