Thank you for the question. The shrub might be a serviceberry, also called a Juneberry. There are garden huckleberry plants that are treated like annuals and look like pepper plants and there is another wild huckleberry that is closer in size to your description but the leaves and fruit don't appear to be similar. The photos also resemble buckthorn, an invasive shrub. Please read these links and re-examine the shrub. If the leaves are opposite each other on the stem and not alternating, it's probably a serviceberry.

http://www.dnr.state.mn.us/invasives/terrestrialplants/woody/buckthorn/id.html