In large enough quantities, any apple (or crab apple) can cause gastrointestinal upset. In addition, apple seeds contain cyanide, a highly toxic chemical to most animals. Typically, horses would need to consume a cup of seeds to feel toxic effects (that's a lot of apples!).

If your horse just won't leave the tree alone, I'd recommend removing it. If he just takes a bite of leaves every now and then, you're probably OK. Keep an eye out for the appearance of fruit. If the fruit begins to fall to the ground or get within reach of the horses via branches, they could easily overindulge. From a management standpoint, it may be difficult to constantly clean up fallen fruit, and may be easier for you, and safer for the horses, to remove or relocate the tree. Good luck!