I think what you have is peach tree borer, a fairly common problem with peach trees.  I found this reference that should help.

From "Disease and Insect Management in the Home Orchard" NC CES

https://projects.ncsu.edu/cals/plantpath/extension/clinic/fact_sheets/index.php?do=disease&id=7

"Peachtree borer is a clear-wing moth that lays eggs on tree trunks and the larvae bore into the base of the tree near the soil-line. This usually results in a dark yellow gum that contains saw dust-like wood particles called frass. A pyrethroid insecticide applied to the trunk of the tree (from the first scaffold limb to the ground) during the first week of September, will help to control this insect borer.

pyrethroids: permethrin, cyhalothrin, bifenthrin (These three different pyrethroid insecticides are synthetic analogs of pyrethrins, but they have enhanced activity and much longer residual activity.)"

This publication is also helpfu for peach growers.http://content.ces.ncsu.edu/growing-peaches-in-north-carolina/

Sometimes extension deletes older links.  This one should work. The borer may eventually kill the tree if there is enough damage, but that is unlikely since you have noticed it early.  Some people have tried taking a paper clip end or something thin and trying to squash the caterpillar in the holes, but that is not a research based solution. Keeping the tree from being stressed by anything else is a good idea until you can break up the life cycle of the moth.

https://content.ces.ncsu.edu/greater-peachtree-borer