While he’s friendly to Belle, he notes that he personally isn’t in a great place due to the fact that he’s having trouble with his memory.
It’s an innocent line that you don’t really think much about… until you get to the end and realize exactly what he’s been forgetting.
Audiences first meet Mr. Potts, played by Gerard Horan, shortly after the introduction of Emma Watson’s Belle — featured as one of the friendly townspeople she runs into as she walks to return her book.
For the last nine to 10 years his wife and adolescent son have been stuck as a tea set, and the spell of the Enchantress (Hattie Morahan) has individually cursed him so that every fleeting thought he has about them winds up fogging his mind (an extension of the curse used to have everyone in town forget about the people in the castle).