He was thrown out for his chaotic actions, and it wouldn’t stretch the timeline too much to buy into him (and possibly his cohorts) to lie in wait until a point when they could return with new identities.
Could that reveal possibly tie into the long-theorized-upon Alexander Davidson, who’d been exiled from Alexandria by Deanna at one point.
Then again, maybe Dwight came back and did it, because where is Dwight these days?
Will we hear legends of someone who brought a new kind of havoc into Alexandria?