It was very clear that this manager will not talk to him politely and will not treat him with respect. It was clear that she had no idea who he is or what authority he has. Baker was not in uniform and to her, she thinks he is beneath her.
This made Baker think, do people respect him just because of the uniform? If he walked around without the uniform as he did now, will people show their true colors? From this experience, he has no doubt about it.