Good question! And one that I grappled with a while back.
In essence, removing Sam’s soul from his body won’t cause his body to grow back. If it did, we’d have dead borrowers popping back to normal size all the time!
Essentially, the curse needs to be stripped from the host soul in order for them to return back to their normal size. Otherwise it leaves an imprint on the body it left behind and small they stay. This is something that Celeste, and only Celeste, knows how to do, and she’s certainly not about to just casually go break her own curse. She’s the only one able to tamper with her own archangel Grace that’s been intertwinned with the borrower souls.