There was the time he said he felt he had to leave over creative differences.
He’s also publicly spoken out about issues he had with the producing staff and more, issues that may have also led to him ultimately only playing the iconic character for one set of episodes.
(He even won a National Television Award and more at the time.)
Previously, the actor has detailed experiences with the show and he hasn’t had a ton of positive things to say about the experience, despite the fanbase seemingly liking him as The Doctor.