1. wealth or lack thereof: hunting generally is not nearly as cheap as playing a video game for recreation, hunting moose in Alaska is for the rich 2. lack of health or physical condition; this sport is not for couch potatoes or invalids 3. laws and game regs can be intimating or overbearing to some people and some game laws even can be deal-breaker 4. lack of social connections: not knowing people with land to hunt upon 5. lack of access to hunter education and knowledge: not knowing how or where to hunt 6. political climate: many areas in America are anti-hunting 7. hunting is often done with guns and there are many firearms regs to deal with 8. social shame, moral objections in certain communities or families regarding hunting
Consequently, most hunting in America is done by those who have hunting in the family as a tradition or who live in a hunting community. Most hunting culture is rooted in rural America.
Those with limited means, who are disabled and/or hail from the city would find this sport most challenging to try to enter. There is not much incentive to open up hunting to all Americans who want to do it.