It’s not about the users, but Facebook as a company that has its own agenda against its users in order to make as much money as possible.
It’s also about Facebook seeing other networks not as friendly co-spaces, but as competitors that it tries to crush.
If you talk about users though, the “worse servers than Facebook” are by far less powerful than Facebook, and they impose no danger to the Fediverse.
If someone plans to move to my neighborhood and that one has a record of burning down houses, it’s not a good idea to give it a chance.