Firstly, the aliens weren't a bad idea or out of character for the stories. Clearly, the first one and Temple of Doom directly showed that magic is real. So, if magic is real, then why not aliens, as aliens are actually more grounded in reality.
However, when skull came out MANY people were very excited. That's because the previous movies were largely seen as being serious adventure movies with SOME humor, fitting of the situation, and SOME fantasy elements that were acceptable. I don't think fans of the films wanted more humor or fantasy stuff, they wanted Indiana Jones personality in complex difficult situations.
What we got was too much humor and the story focusing a lot on his kid. The actor playing the kid looked nothing like his father or mother, and was annoying with an annoying name, Mutt. At one point, he swings like Tarzan through trees when he has no reason to be able to do that. So, the movie was part Indiana Jones, because Ford was good, but then part comedy and part cartoon.
It's a typical example of George Lucas kind of work where he makes everything stupid.