Your husband is indeed guilty of breach of trust; he has been unjust in the way he has treated you and your children. His actions cannot be justified Islamically.

You have one of two choices:  to try your best to ensure that he treats you and your children fairly and equitably. If need be, you can approach the imam in your center or mosque or any other leader or wise person in the community who can advise him in this matter. You should also pray to Allah to give you relief and comfort.

If, however, he is not willing to accept the responsibility to change, then you may want to seek divorce, if you cannot continue to live in such a relationship.

I pray to Allah to help you to make the right decision, and give you comfort.