Aidy Bryant, Pete Davidson, Kate McKinnon, and Kyle Mooney announced their departures before to the May finale