He’s annoyed that you’re so needy. Until he realizes he actually hurt his sweet lover’s feelings.