Hello Katie,

It appears that your peonies might be suffering from two different fungal diseases: powdery mildew and cercospora leaf spot. Both are very common on peonies and many other flowers and vegetables. The conditions this year are perfect for these two fungi. The good news is that the same treatments can be applied to manage both. 

Even though your shrubs are heavily infected, powdery mildew rarely kills a plant. But, yes, you should remove all infected leaves even if that means you have to remove all the leaves. If there are unaffected leaves or new leaves emerging, you could use a fungicide to help manage further damage. If you use a fungicide, always follow the label directions.

Preventive measures are the best approach to managing Cercospora leaf spot (and powdery mildew as well). Cultural practices that help prevent and reduce disease incidence include:

• Avoid overwatering or watering in the late evening to reduce free moisture.
• Avoid overhead watering where the water can dislodge and disperse spores to uninfected plants.
• Space plants to encourage air movement and reduce high humidity levels.
• Remove all leaf debris from the ground to help prevent  prevent the spores from overwintering in the soil.

Fungicides are available to manage Cercospora leaf spot and powdery mildew. Many of the conventional products used to prevent black spot of roses will also protect against Cercospora leaf spot and powdery mildew. These fungicides contain the active ingredient chlorothalonil (OrthoMax Garden Disease Control) and myclobutanil (Immunox). Always read and follow the label instructions.

The following two link give excellent information on the care of peonies. I think you will find them valuable.

https://extension.unl.edu/statewide/buffalo/Yard/Peonies%20with%20Powdery%20Mildew%2010-15-2016.pdf

https://extension.umn.edu/plant-diseases/powdery-mildew-flower-garden