iMessage issue iPhone

I have an iMessage issue. Sometimes when I text a certain contact, iMessage will stop working, and the text box will be green, and my contact won't receive the message. (So whenever I send a text to my contact with a green box, the message won't be received.) BUT, when my contact texts me again, (I have to tell my contact to text me again, because she never got my message) iMessage works again. Is this fixable? It's annoying how iMessage stops working mid conversation, and i have to tell the person I'm talking to to text me again in order for iMessage to work. HELP!