Any part of the plant, if ingested, is poisonous but just touching it is not harmful.  Farmers have been known to remove the buckeye tree from fields where cattle graze to prevent the animals from eating any part of the tree. It can cause paralysis, vomiting, diarrhea, stupor. However, Native Americans used to roast and eat them but I would not recommend trying it. Just touching the nut is not harmful – unless you count the amazing variety of craziness that occurs to the people who wear them at OSU football games.