Peach trees under three or four years old are not yet mature enough to produce. However, the most common causes of peach trees flowers falling include poor pollination, damage from a late frost or freeze, incorrect pruning or fertilization, too few chilling hours, pest infestation, and diseases.
There are several fungal diseases that may infect peach trees and cause the fruit to drop early. For example, brown rot can infect twigs, blossoms, and fruit. Infected fruit may fall to the ground or may become mummified, creating an ideal location for the fungus to survive the winter.
Other fungal diseases that may affect your tree and its fruit include:
Peach scab which creates green scabs on the leaves and fruit
Peach leaf curl which causes leaves to curl and turn yellow or reddish in color
Shot hole which causes lesions on the fruits and leaves

And yes there could have been some root damage that caused some stress to the tree. Root damage is often overlooked as it take several years to manifest itself. If you damaged roots within within the canopy distance, it may have caused some damage to the health and vigor of your tree.