This appears to be a species in the Sphecidae family of wasps; about two dozen species have been observed in Maryland. We don't have an entomologist on staff to ID this to species (which sometimes can't be done from photos alone due to very nuanced differences between species), but based on body shape and coloration our best guess is the Blue Mud Wasp. Their larvae feed on spiders and the adults are solitary, so they won't have a shared nest to defend the way Yellowjackets do (so are highly unlikely to sting).

They are probably getting inside from some access point that could be sealed since they do not normally nest indoors. No pesticide treatments are needed (nor would they work well without knowing how the wasps are getting in). Someone can check exterior walls for gaps around vents, torn window screening, worn door weather-stripping, etc. Even though they might be hunting spiders indoors (unusual) they would not be nesting in the building interior. You can view typical mud nest structures in the linked BugGuide page for that Blue Mud Wasp species. (Similar nesting would apply to other species in that family.)

Miri

Thank you very much for your help!

You're welcome!