Hi Amy, thanks for the question.

What Peppers need primarily depend on your goal for the plants. Is your goal to just keep them alive over winter or to get production over the winter.  If you are going to try to get fruit, they will need to be warm with nighttime temps greater than 50°F and daytime temps around 70°F. Artificial lighting will be required. Water and fertilize as needed, being sure to not over fertilize or over water. Note that sweet peppers are less resilient than hot peppers. You can also allow the plants to go dormant. Trim them back and keep them in a cool area.  Limit the watering. If you get flowers you may need to manually pollinate them (e.g., shaking plant)

Many herbs will tolerate cooler temperatures and may not need as much daylight. They can do much better under grow lights. Fertilize lightly and less frequently. Keep an eye out for insect pests (same for peppers).

Check out the following links for more information:

Can I grow herbs indoors over the winter? | Horticulture and Home Pest News (iastate.edu)

Five home-garden plants that are suitable for overwintering | OSU Extension Service (oregonstate.edu)

How to Overwinter Pepper Plants For Bigger Yields (ruralsprout.com)

I hope this is helpful.