Mason bees nest in hollow stems of plants. Do you have the appropriately sized stems in the compost pile? And have those stems been in the compost pile since last year at this time? If not, then the” bees” you saw could be yellowjackets, or another solitary native bee that makes a nest in the ground, or in this case, in a compost pile. Mason bees should have just recently hatched and still have a few weeks of egg laying left. You may use the old compost.

An Oregon State University Extension publication, Megachilid Bees in the Pacific Northwest contains information about mason bees and leaf-cutter bees. Another publication, Nurturing Mason Bees in Your Backyard in Western Oregon gives even more details specifically for mason bees. It outlines how to make your yard attractive to native mason bees by providing some bare soil, the nesting blocks, and many flowering plants.

It sounds like you are on the right track. Be patient, surely a wise mason bee will spot your nest tube and move in.