At some point you'll definitely need to talk it over and address it with him, because shutting that part of yourself off is only going to cause resentment in the long run. Try to find some middle ground or compromise so that you both feel like your needs are being met.
Maybe you could work something out where you agree to designate specific days of the week for each role. You each have 3 days to be the little and the Caregiver, with the 7th day (whichever day works best for you both) being a "free-for-all"/spontanous, or just to use as a break from roles altogether, especially from the Caregiver role. We can't all be little 24/7 due to adult responsibilities. So it wouldn't be fair to expect someone to be in their Caregiver mode 24/7. If he has a problem with trying to work out an arrangement that works for both of you, that's very disrespectful and a red flag, and will only cause more harm and problems later on if you choose to stay in that relationship.
Hope this helps, and good luck!