Time to blame the media.

"It bears repeating: at the Met, the most expensive opera tickets are indeed expensive, but you can stand behind the orchestra section—or even sit at the upper reaches of the house—for less than the cost of an IMAX showing at the AMC Loews Lincoln Square 13 multiplex up the road. This persistent fiction of “elitism,” and contemporary classical music’s supposed inaccessibility, is one of the strongest propagandistic tools ever devised by the titans of corporate pop culture. They would prefer you not ever cost-compare a Family Circle seat to Satyagraha alongisde a 3D screening of Transformers 3."
-- Seth Colter Walls  source: http://www.theawl.com/2011/12/at-satyagraha-and-occupy-lincoln-center

Unfortunately, almost all of classical music and composed music bears the stamp of elitism in our culture. This is primarily the cultivation of ideas propagated by composers, listeners, classical musicians and orchestras.. but also by pop media, who do a pretty good job at telling you what you're supposed to like. Since classical and composed music has been pushed to the margins of society within the last 50 or so years, many organizations adopt the elitist stance as a method with coping with the fact that only a small percentage of people attend our concerts, but hey, look! We're still being funded because we're  culturally important despite the odds.

But the other question is, does the pop media trick you into think you're not smart enough/rich enough/sophisticated enough to attend a classical music concert?