I don't really mind either

Boyfriend's brother: If you guys don't mind, I was thinking...
Boyfriend: No, that's perfectly alright. I don't mind if you think.