Dwight’s entire existence on The Office, from his workplace sycophancy to his home life with Mose, basically served as a meme generator, so Wilson probably doesn’t have many fan encounters that serve as normal interactions in any sense.
Rainn Wilson was obviously having some fun there, as he wouldn’t straight up tell an Office fan “Fuck you!”
Or possibly a great many people, since TV fanatics aren’t always the most normal and personable human specimens out there.
But I can’t imagine his post-2005 life has been completely exclusive of wanting to tell that to someone.