The good part is that you know meditation has the ability to shift and nurture our minds. I would suggest to stay on task. I'm not sure which meditation technique you are using, but there may be a reason to shift to something that will create a feeling of self-love and compassion of self.
I find that mantra meditation (stating a word or phrase) puts me in a different state of being and feeling. I have experienced self-doubt and insecurity, and during those periods of my life, I continue to meditate even though my heart was heavy and my mind was distracted.