Hi, I feel the same way, even though my son has 2 brothers and several cousins.
Here is the beginning of our plan and maybe it can help you create one for your son.

Right now ,we asked my husband's cousin to oversee Mitchell and his special needs account. If anything should happen before my children are out of school, she will have custody of them all and the right to turn Mitchell's account over to one of his siblings when she feels they are responsible enough.

We are leaving our house to his special needs trust (so his ssi won't disappear) and have instructions to hire a live in aide, chosen by some one we trust. Then if possible have another child live with him for constant companionship.

We have talked about buying a twin home or a home on a lot of land if we move and start the transition of him living in his own place. But that won't be for another 10 years.

He will always be with us till we pass, by our choice.
There are also lawyer agencies that set things up and can be left in charge of your child's account and that's who would control how your son's money is spent and the guardian of your child.

It is really up to you and your child if or when you choose a group home, either now or later.

I hope I at least greased the wheels a little. I would love to hear your thoughts and others too.

I’m in a similar situation with my daughter as well. No siblings, no near by family members I can trust, and her dad is married and living in another state.

I’m working on getting her SSI and need my financials to be in a better state before I can set up a trust for her… but I have at least been teaching her how to handle money. I got her a Cash card and set her up so she is able to make purchases. I’m also introducing her to Amazon to order food for her and the pets, and I’m trying to teach her how to make her own basic food items, like soda, butter, and bread so she may not need to drive (she refuses to learn). Since we are in an area without public transportation (Uber/Lyft unsafe) I would have to figure something out for her to travel with if needed. Otherwise Amazon delivers everything she could need to run the house or feed herself. It’s just maintenance of the house, cleaning, and companionship that mainly concerns me. Besides the finances, trust, and my will that is.

I’m sorry if none of this is helpful for your situation.

The big difference is that you have other children to roll with your plan once it is activated. :-) We have a plan and are saving towards it, it's just we don't know what to do because when dealing with group/residential homes you are at the mercy of their HR hiring skills and our son would be totally defenseless if THEY make a bad hire. Now that could happen anyway, but at least with a sibling, he would have someone who would be able to check on him. Thanks for the response, It was helpful to learn about the special needs trust/ssi info.