We appreciate your question, but this one may better be answered by a veterinarian if you have real concerns.  Being able to locate and to eliminate such bees would also be a gargantuan task, and likely unnecessary, especially due to the value of pollinators,  as well as tremendously costly--if taken on by a professional.  It might also mean removing certain flowering plants in your yard that you--and the bees--presently enjoy.

Our Master Gardeners did more research to give you more information in making a decision and that follows: 

Bumble bees do live in colonies but not nearly as large as honey bees, and they will sting, just not as aggressively. The queen is solitary through the winter in the ground then emerges to start her colony. Best to hire a professional to get rid of a colony. I tend to call any fuzzy bee a bumble bee if it doesn't look like a honey bee. This is not correct. There are honey bees, bumble bees, carpenter bees, leaf cutter bees...

https://www.oregon.gov/ODA/shared/Documents/Publications/IPPM/ODABeeGuide.pdf

https://www.ars.usda.gov/ARSUserFiles/20800500/BumbleBeeRearingGuide.pdf

The line below is quoted from the link above. The whole .pdf is about raising them. 

"Most bumble bees nest in the ground in cavities such as abandoned rodent burrows, holes in building foundations or stacks of firewood."

https://entomology.cals.cornell.edu/extension/wild-pollinators/native-bees-your-backyard/

This article from Cornell refers to a bee that is "solitary" but builds nests in the soil in large groups. "One of the most abundant ground nesting bees in northeastern and midwestern region of North America is Colletes inaequalis (photo 1). Even though this bee is solitary, meaning that every individual female builds her own nest, it is also a gregarious nester (photo 2). Many females (hundreds and sometime thousands) build their nests next to each other. The nests are obvious above ground because of the conical piles of dirt with a hole in the middle (photo 2). Colletes inaequalis has a strong preference for sandy soils on south facing slopes. Thus, if you have these conditions in your backyard, you may find these bees showing up every year where you live."

Tells what type of soil they like (dry, sandy). Wet the soil if you want to get rid of them.

https://blog.nwf.org/2021/05/5-facts-about-bumble-bees-and-how-to-help-them/