well storywise he's no longer needed anyway so its no big loss and probably a good idea. the only one who should return is walter for obvious reasons. i could see them give him a cameo somewhere though.
Though, I will say, his part to play is over. He did what he set out to do and brought Muppets back to the forefront of the public consciousness. And it was awesome. Time to let the Muppets keep on keeping on. I REALLY hope they don't stop.
After seeing Muppets Most Wanted, I think it was a mistake for him not to at least write the script or just do a cameo at the end. And we can see from the box office that he was wrong not to return.