Few industries have captured technology and adapted it to their use better than have the news media. More numerous channels on cable TV and internet sources have made news more available to consumers and improving computer and communications technologies have given news media the ability to transfer stories from greater distances with shocking speed. With all of these advances in the relating of current events, it is no wonder that more and more Americans are turning to some kind of news media to stay informed; in fact 78 percent of voters said that they relied upon the media for information about the 1996 presidential campaign.

With this increased dependence on news media, the American people become more susceptible to the possibility of a bias in the news media. Conservatives have long been clamoring about a liberal bias in the media, and more recently, liberals have countered that, contrary to traditional belief, the media actually have a conservative bias based upon the fact that modern media conglomerations are often owned by corporations. There are also those who have staunchly maintained that the media, in fact, have no unreasonable bias and succeeds in reporting the news both fairly and accurately. With the importance that the media plays in lives of people, and the hotly contested nature of their bias, it is more important than ever to develop accurate ways of measuring bias in the media.

None of these observers should be convincing in their assessments of media bias because their methods of examination and argument are flawed. Content analysis, used properly, has the potential to solve many of the problems with these analyses and assertions and to give researchers better empirical data with which to analyze media bias. At the very least, content analysis will clarify much of the debate about media bias and perhaps suggest new questions to ask.

This paper will first examine the claims of the media being biased in both a liberal and a conservative direction, paying special attention to the reliability and validity of these claims. It will then offer content analysis as a method whereby researchers can examine the content of media and attempt to categorize it. Next, the paper will examine ways in which the reliability and validity of content analysis itself may be improved. Finally the paper will examine the usefulness to methodology in redefining the terms "liberal" and "conservative" to make them more accurate.

Before we begin a discussion of bias, we must clearly define the term so that there is no confusion. Bias is difficult to define because it has several different definitions that are not only distinct, but can be mutually exclusive as well. In this paper "bias" will mean "any systematic slant favoring one candidate or ideology over another." The most important part of this definition is the word systematic. Bias has sometimes referred to purposefully misrepresenting facts or misquoting sources. The important word in this definition is "purposefully," which makes this a much narrower definition. Because this definition is so narrow, we are not going to be using this definition. In addition to this reason, "bias," as it is generally used when discussing media, more closely relates to our first definition. However, if the second definition (which we are not using because it is too narrow) happened in any systematic fashion, it would then fall under our first definition and so would count as a bias. That is to say, a singular mistake by a media source would not constitute bias, but only if a consistent pattern of favorable or negative coverage of an issue emerges.

"It is a widely accepted belief in this country that the media suffer from a liberal bias. Television pundits, radio talk-show hosts, and political leaders--including presidents of both parties--help propagate this belief, and their views are widely disseminated in the media." While not all people believe that the media have a liberal bias, those who do believe this far outnumber those who believe that the media have a conservative bias. It is true that many reporters tend to be liberal in their own views and tend to vote with the Democratic ticket. "[Robert] Lichter's surveys found that half of the Washington bureau chiefs and congressional correspondents say they are Democrats, only 4 percent claim to be Republicans and 37 percent label themselves as independents. But more than half consider themselves to be liberal or moderate." Granting the fact that most journalists are liberal in their personal lives, still fails to prove that these journalists let their personal views affect the way they report the news to any substantial degree. There is little or no evidence to support the claim that journalists allow their work to be controlled by their own ideologies in any significant way. In fact it would appear that this claim is nothing more than speculation, because, as many reporters themselves argue, the reporters do not choose the news and many events are worthy of being covered regardless of how reporters feel about them personally.

A similar trend of pure deduction rather than empirical data can also be noticed on the opposing side of the argument. Liberals who claim that the media are actually conservative use a similar strategy of reasoning to make their case. In "The Myth of a Liberal Media," Michael Parenti explains, "Who owns the big media? The press lords who come to mind are William Randolph Hearst, Henry Luce, Rupert Murdock, Arthur Sulzberger, Walter Annenberg, and the like--personages of markedly conservative hue who regularly leave their ideological imprint on both news and editorial content." It is the theory of liberals like Parenti that modern news organizations are actually corporations like any other and as such are responsible to those with large amounts of money, who are, invariably conservative. This theory seems very solid and very plausible, but there simply is not any significant amount of evidence to support it. Like their conservative counterparts, they simply cannot prove a media bias.

The reason these groups are having so much trouble convincing one another and the public at large of a media bias because they have yet to actually analyze the content of the media in any disciplined way. They begin at the wrong end, examining the reporters themselves in order to prove their preconceived notions about bias. Content analysis, in contrast, is a methodologically sound process whereby the content of the media is categorized so that conclusions may be drawn about the content of the media as a whole.

Simply explained, content analysis involves reviewing samples (reading an article or book, or watching a broadcast or movie) and then placing parts or wholes of those samples into one or more categories that have been previously set up. This actual act of deciding which category or categories a sample goes into is called coding. Once samples are coded into categories, a researcher can then examine how or with what frequency samples appear in different categories.

When analyzing the methodology of any given measure, a researcher should focus on two main aspects: reliability and validity. We will quickly examine exactly what reliability and validity are, as well as how one would go about assessing them.

For "reliability" and "validity" this paper will rely on the definitions of Edward Carmines and Richard Zeller. "Fundamentally, reliability concerns the extent to which an experiment, test, or any measuring procedure yields the same results on repeated trial… An indicator of some abstract concept is valid to the extent that it measures what it purports to measure." So a measure will be called reliable if it returns similar values each time the same thing is measured. For the purposes of this paper validity can be interpreted to actually mean content validity which is sometimes referred to as face validity. This type of validity means a measure accurately reflects the constructs or concepts that a researcher intends it to reflect. That is, that the question that the measure is attempting to answer is answered in a way that makes sense with what the measure is actually measuring.

On the surface, content analysis appears to have a hopelessly large number of problems with reliability and validity. These problems, however, can be fixed by careful application of this method. The first problem that strikes one is the subjectivity of this method. There is essentially one person (hereon referred to as a judge) who is placing these samples into categories. For ease and simplicity, consider a judge who happens to be extremely conservative. To him, nearly every sample he sees will be classified as liberal, and the media will of course have a liberal slant. To prevent this type of error, two steps must be taken. First, researchers must take care to choose judges wisely and make sure that they are fairly centrist on any issues they will be evaluating. The next step is to have multiple, independent judges. Multiple judges not only greatly reduce the risk of placing a judge’s own prejudices into the supposedly objective data, but they also allow a researcher to calculate a coefficient of inter-judge reliability. The formula for this is V/M where V is the number of samples that the judges placed in the same category and M is the total number of samples given to each judge. A high coefficient of inter-judge reliability indicates to a researcher that the judge’s personal views are rather absent from the data and that most reasonable people would categorize the samples in similar ways.

To increase now validity and reliability, a researcher should introduce "judge training" to his or her research. "Training judges is important to objectivity because it increases the coders' familiarity with the coding scheme and operational definitions, thereby improving interjudge and intrajudge coding reliability." Training judges will also increase validity because there is a greater assurance that each of the judges will be looking for the same thing and using the same criteria to evaluate the samples. Thus, a researcher is much more likely to actually be measuring what he or she purports to be measuring.

The next item we will examine is the number of categories. When these samples are placed into one of two categories, such as liberal or conservative, the inter-judge reliability is fairly high, but that may not be true of intra-judge reliability, and validity may suffer. Here is a trade-off between inter-judge reliability and validity. "As the number of categories decreases, the probability of interjudge agreement by chance increases. For example, one would expect greater agreement with only two categories than with five categories because of the higher probability of chance agreements." While the reliability goes down with more categories, the validity may go up much faster, depending on the categories. With more categories, there is more likely to be a definite place for each sample, and after coding each sample is much more representative of its category, as well as each of the categories being more homogenous, granting the researcher better tools so as to be able to make more accurate statements regarding his or her data.

When increasing the number of categories, one may naturally ask what these new categories should be; after all liberal and conservative are very complete categories. Herein lies the problem; each of these terms is so broad that a researcher may be faced with wildly different articles both falling into the liberal category, or similar problems with conservative category. To begin with, the researcher could add a third category entitled "moderate." This would allow judges to put borderline samples into a category without making the liberal and conservative categories too diverse too mean anything. Even better would be to separate issues within the liberal and conservative continuum into their own categories. For example, there might be an "economically liberal" category and an "economically conservative" category separate from the "socially liberal" category and a "socially conservative" category. Researchers need not stop there; they could easily expand the categories to as much as their particular sample size and diversity warranted. One important feature to note here is that a single sample is not limited to a single category; it may go in multiple categories as long as a judge deems it appropriate.

Naturally many of the problems that plague the debate of liberal and conservative bias in the media will follow into the discussion of content analysis as well. Probably the biggest problem that plagues any discussion involving liberalism and conservatism in general is that of definitions. What is a conservative? What is a liberal? Are the terms all encompassing? Are they mutually exclusive? These questions are fairly asked because there are nearly as many definitions of these terms as there are people who use them.

Every person has his own concept of what liberalism and conservatism are, but these concepts can be radically different. In fact, it is the disagreement on definitions that largely contributes to the current debate regarding liberalism and conservatism in the media. Those who claim that a liberal bias exists in the media mainly define liberalism as favoring the Democratic Party, and taking a "liberal" slant to stories that are written, while those who claim there to be a conservative bias in the media tend to define conservatism not by party affiliation, but by ownership by corporations and what issues are covered. Because liberalism and conservatism are not necessarily mutually exclusive or all encompassing, they make for poor categories in content analysis, which is very sensitive to the categories chosen. "Content analysis stands or falls by its categories. Particular studies have been productive to the extent that the categories were clearly formulated and well adapted to the problem and the content." If these two terms were used, the validity and reliability of such a study would be unacceptably low, and indeed rather worthless. If, however, a researcher decided to analyze one aspect of liberalism and conservatism he would be much better off, because he could explain to the judges exactly how the samples were to be coded. If a researcher wanted to examine the bias of the media regarding environmental issues, the researcher could instruct judges to code the samples according to whether the samples took a positive or negative stance towards the environment. This method leaves much less up to the judges and increases the likelihood of having a higher incidence of reliability. Validity is much less of a factor because the researcher could state at the beginning of his research that he is defining liberalism to be pro-environment, and conservatism to be anti-environment. In this manner the researcher avoids the trap of definitions destroying the validity of his research.

Content analysis is an extremely useful tool that can offer a wealth of empirical data to a debate that has been characterized by its appalling lack of such data. In addition content analysis allows researchers to break down liberalism and conservatism and possibly even add more dimensions to the debate, rather than just a single continuum. Deduction in this case can be augmented by this new data, so that while the debate will most likely not end, it can at least move into a more intelligible phase of discussion based on fact rather than supposition.