I think it's quite unrealistic to expect perfection out of a partner. I don't expect my partner to be perfectly fluffy all of the time. Real people have chips, cracks, dents, bruises, scrapes, scars, and many more flaws. If your Caregivers are also real people then they should understand those imperfections. Maybe you've been talking to aliens though!
If you're having depression episodes frequently then is it something you're talking with your doctor about? Are you seeing a therapist or professional about it? These are very important things because your Caregiver isn't likely to be a licensed psychologist, and, even if they were, they shouldn't treat you if you have a relationship with you. You have to love yourself by taking care of all of your health--including your mental health.
Also, I don't know what you're saying when you're having a depression episode but maybe you're saying something very, very scary and it's overwhelming your partner. Threats of suicide, for example, can be extremely frightening in a new relationship and also can, at times, feel manipulative if said in certain ways. If they weren't prepared for those things then maybe it's just overwhelming them and they are retreating.
If those scary words are typical for your depressive episodes then your partner should be well-aware of those things so that they can prepare for them before they happen to come out of your mouth (or from your fingertips if long-distance). They should know about those things before you two even agree to be in a relationship together. They need to be able to say, "Yes, I can handle that. Let's move forward together," or, "I'm sorry but I don't feel confident about handling those situations. We're not a great match right now."
Now, I think a healthy relationship can certainly withstand those things if both partners care deeply about each other and are realistic in their expectations of each other. Sad days are normal for people to have, but having them excessively or to an extreme can be a struggle--both partners need to be willing to fight that battle.
The real question here, I think, is how early in you knowing these people is this happening? Have you developed a strong personal bond with them? Do they know who you are outside of "being little"? Are you jumping into these relationships before spending time getting closer to these Caregivers and becoming best friends with them first?
I know you identify as a little but I recently posted a dating ad here and realized that many littles only want to talk about being little, as if they don't have anything else to mention about who they are and what they want in life. In my eyes, you aren't just a little. You are uniquely you. You are more than having a playful personality and childlike traits. You have interests, preferences, future goals, past accomplishments, and much more. You are flaws and all, not a cotton candy dream. If you aren't preparing your prospective partner for your flaws then they're going to only expect your fluff.
I answered something about relationships recently when a little felt like he was being abandoned a lot too. Let me quote some of it here to you in case it applies:
Are you making them try to get closer to you? Are you really getting to know who they are--not WHAT they are? Are you having long, in-depth, and personal conversations with them so often that you feel like you're best friends before you two even move to the next stage into the Caregiver/little labels?
Make them work to have you and then give them the big payoff of being their little. Make them want you, chase you, and desire YOU. Not what you are but WHO you are. Not what you can offer but WHO you are. Not what you like but WHO YOU ARE.
I would also be careful of things like ... saying ... ("I love you") without them having to earn that care? If so, why? Why are you throwing away that specialness so easily?
It's going to take days, weeks, and sometimes months to actually make it to the "relationship" label part. It is never going to take seconds, minutes, or hours--if it does then expect it to fizzle out just as quickly as it lit up. It isn't genuine love or care if you're meeting someone two hours before "falling in love" and devoting yourself to them because, honestly, you don't even know them yet.
I'm not saying that you are doing all of the things... Really sit down and evaluate how you're going about all of this and try to make some--probably major--changes in your attitude and conversations you're having.
You are the only person that can protect yourself in these situations. Don't let your guard down because someone says they are good. Make them prove themselves worthy of your love, and make sure that you're equally as worthy of theirs.
Lastly, as just a reminder, not everyone you talk to is going to be a match for you and you need to keep that in mind. Just because someone is X and you are Y doesn't mean you two are automatically meant to be and should be together. For example, there a lot of straight brunette women who like men with blonde hair and of straight men with blonde hair who like women with brown hair but it doesn't mean that hair color and gender should be the only (or even primary) reasons people in those brackets should get together. That isn't a good foundation for a satisfying, healthy relationship. Ultimately, we are more (so much more) than just labels, titles, and names.
I wanted to also make the suggestion that maybe you're not ready for a relationship connection right now if you're having a lot of depression. Maybe you need to fight those battles first and have some of those issues resolved being asking another person to join you in life.
It is wrong for a Caregiver to abandon a relationship out of convenience in not having to show compassion. It's wrong for anyone to abandon a relationship without explaining to the other person that they don't feel like a good match. I am sorry you've been hurt, but perhaps these painful moments will help you to become stronger and smarter with future Caregivers.
You're going to get through this though.
Stay strong and keep your chin up.