They always get the fabric wrong

The finest human fabric would look really coarse on the borrowers.

And if you want to be technical, they would have really squeaky voices. And they would have really spindly arms and legs as they are so light. (It's a matter of the area volume ratio and the reason elephants have such thick legs.) And when they fall or jump, their terminal velocity would be so low they could fall from any height without hurting themselves unless they fell awkwardly. Not sure about the effect on brain size or longevity.