There is no official explanation, but I like to think that the ending is what Kate wanted or imagined during the high pressure trip through the vortex. The camera zeroes in on her before showing us visions of heaven and hell. Earlier in the film, Durant called the black hole something straight out of Dante's Inferno, with Pizer adding that one would expect to see someone with a pitchfork and a tail. Later, after Kate learns the truth about the crew of the Cygnus, she tells Reinhardt that if there is any justice at all, the black hole will be his grave. All of this plays into the idea of the black hole representing the afterlife.
I think these are merely visions that Kate has as they spiral through the vortex. Why else would the camera zoom in on her mind's eye exclusively? She sees what she wanted to see, Reinhardt getting what was coming to him, etc. His being trapped inside the shell of Maximillian would be justice served to a man who lobotomized his crew and made them his slave robots. Kate may be religious for all we know, as it is never addressed in the film, but if so, it could account for why she sees an angel leading them through a heavenly hallway as they exit the black hole.
As for the planet at the end, I do not think it is Earth. I think they survived the trip through the vortex and they emerged somewhere else in the galaxy. The ending of the movie is basically them confronting the unknown as they head towards the obscured mystery planet. It may or may not be a happy ending, depending on whether or not the planet can sustain life. The probe ship won't be good for anything else. For it to be Earth, one would have to take the heaven/hell sequences literally, with a real angel leading them home.
- - - - - - -
I am not a fan. I just happen to enjoy movies. Fans are embarrassing.
Good explanation. I thought the ending was a bit too surreal and conveniently/confusingly Christian.