Dave was a cokehead

The Chipmunks were huge stars, and one would think they would have a huge fortune to show for it. Yet they lived in a typical suburban home, and attended public school. Meanwhile, their guardian Dave was nowhere to be found most of the time, leaving them unsupervised to get in all sorts of trouble. It went far beyond a latchkey kid situation to one of outright neglect.

When Dave did make an appearance, he was clearly coked out of his mind, glassy- eyed and given to emotional fits, often verbally abusing the Chipmunks (particularly the red-shirted stepchild, Alvin) for their exploits, though these were often cries for attention or the result of his own irresponsible absence.

I surmise that most of the profits from their records sales and concerts went straight up Dave's nostrils and the "party everyday" lifestyle which accompanied his habit, which explains his sporadic and often psychotic presence in the home, and poor performance as a parent/guardian.