What they decide to spend their wealth on is up to them
The only acceptable way to use that much money is to reinvest it in the society that enabled you to become a billionaire.
It would cost around 20 billion dollars to end homelessness in the US, why should people be allowed to have net worths of 100 billion+ when it would only cost 1/5th of that to provide everyone with a home? Is stealing from one man to house hundreds of thousands of people a bad thing to do? Is personal ownership of wealth worth more than human life to you?
Why would they need to be built in the middle of nowhere? We can build high density apartments if we need to, a roof over your head in a small apartment is better than the sidewalk.