It looks like your peach tree is infected with brown rot, a fungal disease. Insects spread the spores and trees become infected through wounds on the tree or developing fruit. If the blossoms were infected they wouldn't develop, which could explain your low crop yield.

Brown rot is not lethal to the tree, but any infected leaves, branches, and fruit should be removed. The University of Wisconsin Extension recommends that the infected material be burned or buried in order to prevent the further spread of spores. There are instructions on pruning, and sterilizing pruning tools, at this link: https://hort.extension.wisc.edu/articles/brown-rot/

If this is the first time you've seen brown rot on your tree it may not be necessary to use fungicides. If you do choose to use a fungicide, remember that a fungicide label is a legal document. Always follow the fungicide label directions attached to the fungicide container you are using. Fungicide labels may change frequently. Internet labels may not match the label on the container you are using. The site of use or plant to which the fungicide is to be applied must be listed on the label or the fungicide cannot be used. Remember, the label is the law.