Hi, this is Steve and thank you for your Ask An Expert question. I can’t say with 100 percent certainty what specifically made your holes. I can say I believe it is an insect and most likely one that won’t cause any harm. Since you did not find anything when you dug into the holes, I would also believe that the insect was there and has now left, which leads me to think the culprit is a cicada. I’ll attach information on the insect and if you google “cicada holes in ground”, you’ll find a pretty good match of pictures to yours.  I might even see a “shell” left by the cicada as it morphed into an adult in one of your pictures.

Again, this is my best thoughts on the holes, but at any rate, I would not be concerned with whatever critter made them. Good Luck.

https://extension.colostate.edu/topic-areas/insects/cicadas-5-590/#:~:text=The%20adult%20cicadas%20found%20in,produce%20loud%2C%20shrill%20buzzing%20noises.