What to wear when answering the door?

The way I see it is if the caller is expected, dress accordingly. If the caller comes unannounced, they are the ones intruding and should not p judgment on what you're wearing... or not.