The hurt comes from knowing what most people mean by 'soft', the anger from the fact that he's not (but also tired of proving it). But he did pull back and ask for what she meant, so... progress?

"And if I don't put my ass on the line, there's someone else who will. The C.B.I. makes sure of that." Danny's voice goes a little quieter. "This way, at least... what I went through as a kid has a purpose."