FOR WEB DOCUMENTS
Portions of this were taken From Web Resource Evaluation Techniques
By Jan Alexander & Marsha Ann Tate
Wolfgram Memorial Library
Widener University
Copyright 1996-1998 by Jan Alexander & Marsha Ann Tate
| EVALUATION CRITERIA
FOR WEB DOCUMENTS
Portions of this were taken From Web Resource Evaluation Techniques
|
|
Mark Poster writes in CyberDemocracy: Internet and the Public Sphere says that the Internet "is above all a decentralized communication system. Like the telephone network, anyone hooked up to the Internet may initiate a call, send a message that he or she has composed, and may do so in the manner of the broadcast system, that is to say, may send a message to many receivers, and do this either in real time or as stored data or both. The Internet is also decentralized at a basic level of organization since, as a network of networks, new networks may be added so long as they conform to certain communications protocols."
Wired News recently reported on an address by Matt Drudge (who is best know for breaking the Lewinsky story) delivered this February, in which he told his audience that "the individuality the Internet encourages is the perfect and necessary antidote to the corporatization of mass media." They went on to quote Drudge's observation that "There is a dramatic change going on: Individuals are taking over just when it seemed like corporations were going to own everything."
Drudge and Poster are examples of the kind of people you'll find publishing on the Internet these days. Drudge claims that his "Drudge Report" is read by more people every day than The Washington Post. He said he still files his report from his small Hollywood apartment, with nothing to help him but his nose for news and "a US$19.99-a-month Internet connection."
So if its possible for anyone with a computer and an Internet connection to publish on the web, how do you know if what you read in The Drudge Report, or anywhere else on the net, is true or not? In this new and seemingly democratic medium, where anyone can publish anything, how do you cut through the crap?
You begin by putting things in context. Just as you learn that magazines and newspapers appear within a context, so can you learn the context of a web page. Ingrid Volkmer, speaking at the Journalism in Cyberspace Sympoisum held at MIT on November 12, 1998 provided a kind of classification scheme for the various media cultures on the web. Recognizing the enrvironment you're in can help determine the extent to which you can trust the web site you're on.
Internet Environments
The second environment is the Post-Communist Transition environment, which is increasing media coverage on the net and is used as an additional source for information. In Russia, for example, on-line coverage is increasing; they now have nine papers with on-line versions. The information may be slow in reaching the environment, but they are working towards a new arena for public discourse that is not controlled by communism. They are now promoting free speech and the free exchange of ideas, but you shouldn't count on them to be entirely objective. The pre-communist influence is still present.
The third environment is that of State Regulation. This environment still has quite a bit of censorship and critical information does not flow very freely. Most criticism comes from outside sources. This environment includes places like China and Singapore. As a source, they are not likely to be true or accurate.
The Dualist Environment includes state regulated sites as well as commercial sites. Western European countries such as England, Italy and Germany follow this model. The Internet there is not bound by legal regulation and restriction, although other forms of media are. In this environment, the telecommunications industry controls phone charges and access which in turn affects the Internet and restricts its growth. But most of these countries governments provide valid data and can be relied upon for accurate information. You just wont find them providing more than they have to
The last environment, called the Spillover Environment, is primarily used by the wealthy, businesses, and national organizations. They target a large media market and their goals are mostly self-serving. In other words, you can be sure that the IBM, GE, and Microsoft web sites aren't going to report anything bad about their industries.
Determining the environment in which the web site operates can provide you with some indication of the reliability and worthiness of the information you find there. Having established the environment, you must move on to the application of criteria. Web evaluation techniques are only beginning to be developed and web technology is outpacing the ability to create standards and guidelines. Therefore, establishing evaluation procedures will be an ongoing evolutionary process. For now, you're more or less on your own. But the techniques which follow should provide some help.
Evaluation Techniques
1. Accuracy
Of course, it is possible to verify information that appears on the net by doing what people have always done: go out and find it. A lot depends upon what it is and where it is, but you can often find out if an incident happened or didn't happen by checking other media. And that means (gasp) dragging yourself away from the computer for a few hours. Just because its on the internet doesn't mean you can't go and dig out the truth.
Because anyone can publish on the Web, most of the information on web sites you find on web sites has not been verified by an editor or someone whose job it is to check the facts. There are really no standards that ensure accuracy and no "police" to stop anyone from publishing wild claims or exaggerations. And so long as a law suit isn't filed, the information can remain on the web, unchallenged and undisputed. Here are a couple of questions to ask to try and determine if the information you find is accurate.
Are there editors and fact checkers?
Here's the first of seven web sites we'll look at which deal with the crash of TWA Flight 800.
The Pluralistic Internet Environment is the largest, providing exclusively on-line versions of newspapers, magazines and E-zines as well as a convergence of other popular media such as radio, TV and print., which allow the viewer to download voice and video as well as having archives available. They may not all be reliable, but when you land on one you'll know where you are and what the agenda is.
There are five important things to consider when evaluating any document.
Do you believe everything that is printed and thrust at you as truth in the off-line world? Perhaps you've noticed that magazines and newspapers have their 'correction' sections printed as small as possible and in the most hideable spot.
How reliable and free from error is the information?
Based on what you know of the subject and what you can determine from other sources, select a few "facts" and check to see if they stand up under closer scrutiny. If they don't, then you probably can't trust any other part of it
See if the site indicates that there are people other than the writer who check the information for accuracy. If there are, try to determine their honesty and reliability.
Go to the page, read it, and then measure the content against the guidelines for accuracy which were discussed above.
2. Authority
Its often difficult to determine authorship of Web resources and even if the authors name is listed, his/her qualifications are frequently absent. Furthermore, publisher responsibility often is not indicated.
To determine the level of authority for a web resource, you should ask questions like these:
What are the authors qualifications for writing on the subject?
This question stems from the last. Editors and fact checkers aren't a necessity, but if there are none, then you ought to be able to determine what qualifies the author. Look to see if he or she has a degree or training in the field. Lacking that, you should look for them to have some kind of experience that qualifies them. And make sure its the right kind of experience; even though the paper carrier has delivered the newspaper for ten years, he is not a qualified journalist.
How reputable is the publisher?
If the web site is not "self-published," then the publishers reputation should be scrutinized. There's a plethora of "e-zines" now, as well as newsletters and list servers on the net, but many of them are published by persons with a vested interest in particular points of view. There are also a large number of newspapers on-line and in those instances its usually safe to assume that the reputation of the print media "spills over" to the net.
Here's the second of the seven web sites we'll look at which deal with the crash of TWA Flight 800.
Go to the page, read it, and then measure the content against the guidelines for accuracy and authority, which were discussed above.
3. Objectivity
Here are some questions to ask to determine the objectivity of the web source.
To what extent is the information trying to sway the opinion of the audience?
Are the goals and aims of the persons or groups presenting the material clearly stated?
Here's the third of the seven web sites we'll look at which deal with the crash of TWA Flight 800.
Since its inception, the Web has functioned as a "virtual soapbox." Anyone can get on the net and express their point of view. Unsubstantiated claims abound. For example, at the end of this paper you'll find some links to sites containing information about the crash of TWA Flight 800. That event, and others like it (Ron Browns death, Princess Dianas) a plethora of web sites appeared with information about the events and speculation as to the causes. Many of them were purely individual opinions, but they spurred numerous rumors
Is the information presented with a minimum of bias?
Look at the language of the information. Is it written in the first person? That's often a clue to a biased presentation. Are there a lot of emotional words? Does the writer make claims without backing them up or, at the very least, referring you to the original sources?
As we've said before, persuasive writing is nearly always disguised as some other type of writing: expressive, informative, or literary. You must be able to see through these disguises and identify the methods the writer uses to sway his or her readers opinion.
An honest writer who doesn't want to mislead you will state his or her intention up front. But be careful. As with the previous concern, that may not be an honest statement. You know the goals of groups like the A.C.L.U., Greenpeace and the Wildlife Rescue Federation. You may not agree with them, but you generally wont find them to be hiding their intentions.
Go to the page, read it, and then measure the content against the guidelines for accuracy, authority, and objectivity which were discussed above.
4. Currency
Is the publication date clearly indicated?
Here's the fourth of the seven web sites we'll look at which deal with the crash of TWA Flight 800.
Web pages often don't have dates and even when they are included, they may not mean the same thing on every page. Some web masters use the date to indicate when the page was first created, even though it may be updated after that date. Or the date may mean the date it was placed on Web or the date it was last revised. You simply cannot always be sure. The best way to check it out is to ask questions like these:
Is the content of the work up-to-date?
If you notice that some of the material is outdated, then there's a good chance all of it is. Of course, some content doesn't change. Most information on the Edgar Allen Poe biographical page wont change, nor will history and past events. But areas like the sciences are constantly changing. And web sites that deal with those areas ought to change with them.
If there is a date, try to determine what it indicates, whether it refers to when it was first created, when it was placed on Web, or when it was last revised.
Go to the page, read it, and then measure the content against the guidelines for accuracy, authority, objectivity, and currency which were discussed above.
5. Coverage
To what depth are topics explored?
Here's the fifth of the seven web sites we'll look at which deal with the crash of TWA Flight 800.
Web coverage often differs from print or other media coverage. The next chance you get, take a look at your local paper and, at their web version (if they have one.) You'll see a tendency to abbreviate web documents and an effort to recognize that the web isn't linear. So its often hard to determine if the extent of the Web coverage is greater or lesser than that of the hard copy. Here are some questions for nailing down this aspect of information found on the web.
What topics are included in the work?
Does it seem like they have dealt with all aspects of a subject? Sometimes its hard to tell, but often you'll detect gaps. The general approach a newspaper editor takes is to "cut from the bottom." Some web publishers follow that rule too and, as a result, much gets left off.
Does the work deal completely and thoroughly with the subject? Were arguments followed to their natural conclusions? Remember: any resource that provides you with more information than you need is worthwhile. You can't ever have too much information.
Go to the page, read it, and then measure the content against the guidelines for accuracy, authority, objectivity, currency and coverage which were discussed above.
Additional Challenges Presented by Web Resources
Other Languages
Use of Hypertext Links
Pages Out of Context
Blurred Distinctions
Hardware Requirements Can Limit Access
Software Requirements Can Limit Access
Web Pages are Unstable
Here are some more web sites which deal with the TWA Flight 800 crash. Go to each of them and after reading them, evaluate them on the basis of the criteria and additional elements discussed here.
There are some additional, unique problems presented by web resources which you should keep in mind as you browse and/or do your research.
There's a proliferation in other languages on the net. (Yahoo is in 15 different languages now.) There's a German Amazon.com and you can send cards in French, Spanish and Chinese. For most communication purposes though, and for people who want to take full advantage of the net, English is extremely important.
The fact that links can be included in HTML documents can be an asset or a drawback. Part of the trouble is that you may follow a link to a related document and that document may have a link which takes you somewhere else and before you know it, you have strayed completely from your original source. (I see this problem a lot when I check on web citations and cannot go to the source document.)
Search Engines are a great help, but you may have noticed that they often do not point you to the beginning of a document. You may be linked to the middle, the end, or to a graphic or a link and have difficulty finding the "front" page. When this happens, you should try to go to the "home page" to determine source of the information If you can't get there from the page itself, you may be able to edit the URL address so that you load the origin. For example, look at this URL:
http://www.washingtonpost.com/wp-srv/national/daily/july/18/twa.htm
The ".htm" ending tells you that this is a document. If you backspace and delete from there backward you will eventually reveal the address of the source: The Washington Post
With television, radio, newspapers, and magazines we can usually distinguish between advertising, information, self-expression and entertainment. But on the web those distinctions aren't as obvious. In fact, they are very often blurred. If a web pages advertising and information content are supplied by the same person or organization, you may have a conflict of interest.
A simple thing like limitations on memory (RAM), modem speed (bps), or disk space may mean that your computer isn't capable of downloading all of the information on a web site.
Your hardware may be good enough but if you don't have the latest version of a web browser, pages using java or cookies may not be displayed properly. And sometimes you may not be aware of this. It just happens.
If Alice was in Internetland, shed probably have said something like "Pages come and go so quickly here." A page that was up yesterday may be gone today, so you may not be able to refer back to it. Or a Web site
may be redesigned or reorganized so that its pages disappear or are moved elsewhere. And there's no requirement that you or anyone be notified of this. Here today; gone tomorrow. For these reasons, it behooves you to make sure that the pages you cite are stable. Document the source to the fullest extent possible and always attempt to verify information using other sources.
http://www.washingtonpost.com/wp-srv/national/daily/july/18/twa.htm