I do not know the exact name of the white caterpillars, but I can tell that they are in a group called sawflies.  The adults are wasp-like but they do not sting.  Many pesticides will work against them, it is just a matter of finding a product that includes the plants you wish to spray on the label.  Simple, relatively safe products like insecticidal soaps will do fine, but they must actually wet the caterpillars to kill them.  If the numbers are not very high, I suggest just squishing them!  They are not related to be the beetle in the third picture... that is an adult Japanese beetle.  They feed on the foliage and flowers of dozens of types of plants.

Thank you!
Can you tell what the flower is in the picture? It's a perennial, about 3 foot tall, with stacked small yellow flowers. There are quite a few of these buggers but they all seem to be on that particular plant which is in 3 different areas in my yard.

Plant ID... we think it might be called "yellow loosestrife" or garden yellow loosestrife, Lysimachia vulgaris.

Sorry for the long wait.