The yellow rust growing on the underside of the leafs is a disease called Rust. The yellow leafs is from a nitrogen nutrient deficiency.

I have not come across Rust disease on blackberries before in North Texas. I believe the environmental condition, lots of rain and humidity, led to this disease for this year. Based on my review of extension publications, it appears there is not a treatment to control this disease. The fungus that causes the disease becomes systemic and move through the plant to the roots. Therefore it is recommended that you dig up the plant, including the roots to prevent this disease next year. Please review the details in the publication at the following link:

http://ohioline.osu.edu/hyg-fact/3000/pdf/HYG_3010_08.pdf
Excerpt from this publication:

Control
1. Whenever possible, start with disease-free, certified nursery stock.
2. When diseased plants first appear in early spring, dig them out (including roots) and destroy them before pustules form, break open, and discharge the orange masses of spores. If plants are not removed, these spores will spread the disease to healthy plants.
3. Remove all wild brambles from within and around the planting site. Wild brambles serve as a reservoir for the disease.
4. Maintain good air circulation in the planting by pruning out and destroying old fruited canes immediately after harvest, thinning out healthy canes within the row, and keeping the planting free of weeds.
5. Fungicide sprays are generally not considered an effective control method for orange rust in home fruit plantings.