They look like Drosphila melanogaster -- vinegar aka fruit flies. A very common household pest. They are at the window sills for warmth in the sunlight. It'll take a while, but they can be eradicated. I'm not in favor of spraying an insecticide in the house in winter,  but you could take your plants outside on a moderate day and spray them with an insecticide, if you wish, to kill those in and on your plants. However, inside, let's try a lure & trap.
https://www.organiclesson.com/6-ways-to-instantly-get-rid-of-gnats-at-home/
You'll need to be patient. It'll take a while, because you say you have hundreds, but eventually, with persistence, you can free your house of these common insect pests. Good luck!

They don’t look like fruit flies, these are way tinier than fruit flies. I researched a bit more and they feel more close to fungus gnats. Will your remedy work for these too?

You showed me pictures of insects on window sills. Fungus gnats live on soil, which is where they tend to stay (not flying far away from where they live, feed & lay their eggs). If they are fungus gnats, two things: (1) you are likely keeping your potted plants' soil too wet, allowing a favorable environment for fungus and gnats. Too many fungus gnats can start feeding on plant roots & spread diseases to your plants. I would recommend replacing the top layer of your plants' soil so as to get rid of the fungal spores, as well as any gnat eggs that may have been laid (they can lay up to 200 eggs every 3 days).  (2) You can still use the vinegar solution to trap & drown fungus gnats. Place the lure close to your plants and replace the solution often.