I am really sorry to hear about your dilemma, and can see that you have gotten some really good advice so I won't reiterate what everyone else has said, except to say this: Don't let this eat you alive. There are lots of musicians in the world and you shouldn't be afraid to walk away from a project that you think is detrimental to your sanity. I can tell you from extensive experience that it is not worth it. It is also not worth it to stay with a group of musicians who do not have a unified vision of the band's future.
Good luck.