Hi- there are several insect pests that will bore into pepper fruits. The European corn borer is the more common pest (holes around the fruit stem with frass pushed out). Fruit rots quickly follow insect feeding. Cut open some affected fruits to look for signs of insect pest feeding.

The first step is to identify the pest. All three of the following pests are difficult to prevent and control even with insecticides.

Pepper weevil, pepper maggot and European corn borer-

https://content.ces.ncsu.edu/insect-and-related-pests-of-vegetables/pests-of-pepper

Pepper maggot-

https://entomology.ca.uky.edu/ef316

http://ipm.uconn.edu/documents/raw2/Pepper%20Maggot/Pepper%20Maggot.php?aid=57

European corn borer- 

https://extension.umd.edu/resource/european-corn-borer-vegetables

https://web.extension.illinois.edu/vegproblems/ecornborer.cfm

Photo 0380 shows a small dried, brown spot on a pepper fruit due to sunscald. The hole at the bottom of the fruit in 0382 likely resulted from a fruit rot entering an area injured by sunscald or blossom-end rot.

Jon