Are you able to send us clearer photos of the pests, plants, and plant damage? These leaves look more like eggplant than pepper, but in either case, horticultural oil or insecticidal soap (note: not a home remedy soap-based spray) should work well on aphids. However, the aphids did not cause the holes in the leaves, so  perhaps other beetles or snails are at work as well.

Since ladybugs have been introduced, you need to use caution when spraying since the pesticides will also harm them with direct contact. Ladybug eggs are laid in clusters and are bright golden yellow; are these the eggs that you're seeing?

Vegetable plants growing in a greenhouse may not get their flowers pollinated well, which is needed for good fruit production. Is the greenhouse open to the outside, where bees can fly in to pollinate the flowers? A closed greenhouse would also prevent other natural aphid predators/parasites from finding and controlling the aphids in lieu of pesticides.

Miri

Thank you for the additional photos. Yes, this looks like a fairly heavy aphid infestation, though the white flecks also present are shed aphid skins. You can look for aphid mummies to see if beneficial parasitoids have found the colony and are starting to work.

The foliage appears to be stunted and the plants not vigorous enough (and having too-congested growth) to support good pepper production. Additionally, the fact that bees cannot access the flowers means that pollination and fruit set may be poor.

It will be better for the plants overall to be grown outdoors. If in-ground garden beds are not available, you can keep them in the pots, though make sure the drainage holes are free from obstructions or saucers/drip trays. The increased airflow, access to aphid predators, and improved sunlight should help boost their growth and productivity. (Even glass not covered with shade cloth blocks a measurable amount of sunlight for plants needing a full-sun exposure.)

Strong sprays of water and rubbing-down the leaves with your fingers should physically kill or remove most of the aphids, and this will save you from having to apply pesticides. If pesticides are still needed to get control of the population, you are limited in options for those safe enough to use on edible plants. Always check with the label regarding how long prior to harvest to stop making treatments, though in this case the aphids should be a non-issue by the time crops are produced on the eggplant and peppers. Horticultural oil, neem oil, and insecticidal soap are the main three organic treatments recommended for aphid control. (Use only one, not all three, or at least do not combine them or overlap sprays.)

Do not spray the plants with a pesticide during temperatures above 85 degrees, or leaf damage will probably occur. Instead, treat in the morning or evening when temperatures are cooler.

Miri