The failure of a broker might be a (very) low probability event, but if it were to happen, your portfolio could be completely wiped out in payments to creditors and a US Bankruptcy trustee before you see a cent.
I had been looking into it due to the lower brokerage fees (and potential for materially lower margin loan costs) but the failure for shares to be registered in my name through CHESS makes it a 'No' for me.