Count Izas wasn’t going to say anything when he left, but changed his mind because Ruche demanded an apology. The risk here is that both Ruche and Elm are commoners at that point because that’s the promise he gave in the previous episode.

Lords aren’t allowed to give in to commoners because that rule protects all lords, but he compromises and tells her to take care of Elm which keeps his promise to Elm because that’s some good old D&D rules lawyering going on with that promise to leave Elm alone.