Your very large silver maple tree definitely has the potential to become a threat to your home (or your neighbors). Its primary favorable characteristics—shade and fast growth—lend themselves to future problems with this species. Here are a few reasons why silver maples can often be a nuisance:

• Since they are fast growing, the wood is softer and more brittle than that of a slow-growing hardwood tree, making them more susceptible to wind damage..
• The tree is easily damaged on windy days and does not fare well in heavy snow or when coated in ice by freezing rain.
• Downed branches are common and a regular chore to dispose of.
• The root system of the tree grows shallow and will result in an uneven lawn. Great care is required when mowing the grass lawn growing at the base of the silver maple, as mower blades will likely strike the protruding roots. The shallow roots also make the possibility of the tree being uprooted during high winds, especially if the leaves are still attached.
  
• If planted too closely to a foundation or sidewalk, the roots can cause upheaval of the walkway and crack foundation masonry.
• The roots are also known to invade septic fields and damage well pipes, requiring costly replacements or repair.
• The branches and trunk commonly fork and require careful pruning.
• It holds its leaves longer than most other trees, providing additional wind resistance even in the winter.

Your best answer to your question will likely come from a certified arborist. An arborist is a certified and generally insured expert in trees and their problems. He or she can examine your trees and make recommendations regarding the health of the tree, any needed judicial pruning, and whether or not (in his opinion) it should be removed. You can find a certified arborist in your area here:
Click on "Find an Arborist" at the top of the page; then click on United States from the pull-down menu. Then enter your State and City. This should bring up a list of certified arborists nearest to you. Or you may be able to locate one under "Tree Services" in your local Yellow Pages. I understand your concern re. conflict of interest issues; however, remember the arborist will examine and advise, but you do not have to use any services his employer may provide. However, that conflict of interest might be easier than a conflict with your neighbors if that tree or a large limb of it falls on their home or automobile.