How did the baby die?

It was never addressed in the films. However Ned Benson (writer/director) gave a small hint in an Interview

The story is about the relationship, and I know exactly what happened to that child and so did the actors. And so did everybody involved in the film. And the closest I get to it, and I allude to it in a very sort of analogical way, but it’s the moment where she is releasing that firefly and she asks it to wake up. And you know, the child died of crib death.

Lack of oxygen to the brain. That is how everyone dies.

Psycho wife ate it.