Library:Module 5 Evaluating Information Sources

Module 5 Objectives

In this module you will learn the following:

The importance of understanding the publishing life cycle of information sources.
To evaluate information sources to use in your research.
To identify scholarly and popular sources.

This module will take no more than 20 minutes to complete.

Failure to close the box before opening the next section may require you to scroll up to locate the beginning of the section.

5.1 The Publishing Life Cycle

For this module we will take a step back and discuss information sources themselves. This may not appear to be as important as finding information, but if you don't know the different kinds of information, and don't understand how it is created and published, you will not be able to select the best sources for your research. This can result in more poorly written assignments on your part.

The Assignment Life Cycle

The Assignment Life Cycle, depending on your course, usually is one term in length.

While this whole module is about information sources, this section will discuss information and how it is managed over time--but first, let's look at your assignment life cycle.

When you get an assignment you will usually be directed by your professor to find a specific number of resources (e.g. books, articles) for your topic. Sometimes you'll be given a topic and sometimes you will have to develop your own. As covered in Module 1, you will get ideas for your topic from reading course materials, books, articles and internet sources and you will select something that interests you. Often what interests us are current events. Current events provide us an interesting space for research but can cause some problems.

Once you have a topic you will begin your research, but then you might hit a brick wall. There may be no information about your topic, and you must then ask yourself why? Knowing why will help you not only select a better research topic before you begin your assignment but also manage your expectations when searching for material. We will also give you some tips on how to work around this issue without having to start again.

First, let's figure out the "why" of finding the information.

The Information Life Cycle

The Information Life Cycle, will take approximately one year to get from the event to the final output of books.

Just like your assignment, information works within a cycle that is determined by several factors. The Information Life Cycle can begin with a newsworthy event. Once this event occurs, the information about the event enters the cycle:

Day 1 - An event occurs and is reported on the web, radio and television.
Day 2-3 - Journalists write articles and publish in newspapers.
Day 3-14 - Journalists write lengthier articles and publish in popular magazines.
Months - Scholars perform intensive research and analysis on the event. An academic article is written and the process of editing or peer review begins. This can take a lot of time as several versions of the article will be reviewed and modified before publication.
Years - Scholars will perform intensive research and analysis of the event and how it is situated in larger theoretical and global perspectives. This will take a lot of time as it will require several versions of the book to be reviewed and modified before publication, marketing and additional publication processes before it is ready to be accessed.

As you can see, if you have selected a current event for your research paper, attempting to find a scholarly source within a few months of the event will be impossible. Does this mean you can't write on the topic? Of course not! It will just make the research process a little different.

To write a research paper on a current event, you can research an event that is similar. For example, if you are interested in the environmental impact of an oil spill that occurred a few days ago, look for oil spills that have occurred in the past and discuss the current event by generalizing. For assistance with this process, go to a research help desk at your library.

Note - If a research paper is not event driven, for example an article on mineralogy in British Columbia, the publication cycle may take less time BUT the process of peer review is always a lengthy one. You will learn about peer review later in this module.

Test Your Knowledge

5.1 Test Your Knowledge Peer-reviewed articles often take months or years to be published because

Scholars write slowly
Scholars procrastinate and don’t meet the publishing deadlines
Scholars often have to re-write and edit their article several times to meet the strict academic criteria of peer-reviewed journals
Scholars want students to have to work harder for their assignments and so don’t publish articles on topics they are assigning in classes.

5.2 Evaluation of Information Sources

Why is it important to evaluate your sources?

Publishing and distributing information can be very easy - especially on the Internet. Many online tools like wikis, blogs and other social media are free and getting published is as easy as typing up some content and hitting a "create" button. This means that anyone with an internet connection will be able to find a forum to publish his or her ideas. Many people do so without being experts on the topic - or using editors, fact-checkers or other forms of quality control. This is further complicated by the fact that many of these publishing channels come with slick graphics and professional quality site-design. This means that you will have to look beneath the surface to judge the quality of your sources.

Your assignments will often require that you use scholarly sources, written by experts for an academic audience. After all, you can't make reasoned arguments if your sources aren't credible.

The best way to ensure that you have a suitable source is to check it out yourself.

What to evaluate?

You will need to evaluate each resource you use for research, whether it is an online or print journal article, a website, a book, a newspaper article, or other source that you want to cite. Use the questions in this tutorial as a framework to assess how appropriate they will be for your research. Keep in mind that many publications have a particular bias or agenda, which may not be obvious at first glance.

Don't expect to be able to answer every question, all the time, for all information resources you look at.
Rather, try to use the questions as a tool to help you look at sources critically.

Authority

It seems obvious to state that no one is an expert at everything but it's easy to overlook an author's credentials - especially when reading something online. Some authors:

are experts sharing the results of their research with other scholars.
are hired to produce articles for the general public.
are passionate amateurs.
are experts writing for a general audience, e.g., a scientist writing a children's book or an economist writing a summary for a politician.
publish fabricated or unproven research for a variety of reasons.

Questions to Ask

Is there an author of the work? If so, is the author clearly identified?
Are the author's credentials for writing on this topic stated? For instance, journal articles often list the university or organization the authors are affiliated with.
If the author is affiliated with an organization, could this organization have a bias?
Have you seen the author's name cited in other sources or bibliographies? Repeated citations by others and a substantial body of work by the author can indicate expertise.
Does the source represent a group, organization, institution, corporation or government body? Is the organization a recognized authority on the topic? e.g., Metro Vancouver is an authority on regional land use, not search engine optimization.
For online sources, is there a way to contact the author and/or organization?

If you cannot find an author or an organization connected to a source, be very suspicious. If no one wants to stand behind the work, why should you believe what is written there? Even if there is a named author - make sure that the organization and/or author are who they say they are.

The website alexa.com lets you check ownership of a website and find out what other sites link to it.
You can search for an author in a database like Web of Science or Google Scholar to see if he or she has published other work on the subject and if that work has been cited by other authorities.
- Reminder: even if an author is an expert in one field, she or he may not have expertise in the field you are researching.

Accuracy

It's easy to assume that because a work has been published that it's also been checked for accuracy but that is not always the case.

In the scholarly publication process there are a number of steps journal articles go through called peer-review:

The journal editor typically assigns it to two or more independent referees, who have similar expertise to the author.
The referees review the article and write reports that recommend acceptance, acceptance with minor or major changes, or rejection.
Acceptance rates vary depending on the prestige of the journal, and the entire process can take up to a year.

When you search the web, you will usually find a combination of online scholarly journal articles (many provided to you by UBC Library) and other websites.

While individual websites may be written by experts and have some sort of editing process in place, there is no overall system for vetting the web.
This lack of review and revision process means that not all Web pages are reliable or valuable.
Documents can easily be copied and falsified, or copied with omissions and errors - intentional or accidental.

Note: as with other graphic elements on a website, commonly available software makes it easy to present data and statistics in a professional-looking graph, table or chart - whether the underlying data are reliable or not.

Data presented in a source may be original work by the author, collected by an official statistical agency, copied from somewhere else or even fabricated - so make sure to look at it as critically as any other element in the source.
For more information on good and bad graphs, see Gallery of Data Visualization.

Questions to Ask

Is the source part of an edited or peer-reviewed publication?
Can factual information be verified through references to other credible sources?
Based on what you already know about the subject, or have checked from other sources, does this information seem credible?
Is it clear who has the responsibility for the accuracy of the information presented? Is it on a site like Wikipedia which can be edited by anyone?
If data are presented in graphs or charts, is the source of the data clear? Does the data source itself seem credible?

Mini-Activity Look at the NIPCC website and ask yourself if the information seems credible and accurate.

Scope

It's important to make sure that the scope of your source matches the scope of your research. To do this you'll need to look at the purpose, intended audience and coverage of your source material.

Questions to Ask

Purpose

Why was the source created - to educate? sell a product? advocate a viewpoint?
Does the source update other works, support other works you've read, or add new information?
What is the balance between opinion and verifiable facts?

Intended Audience

Is the publication aimed at a general or a specialized audience?
Is the source too elementary, too technical, too advanced, or just right for your research needs?

Focus/Coverage of the Subject

Does the source cover the topic comprehensively, or does it only cover one aspect? Be wary if the author ignores other viewpoints or makes claims which aren't supported by data.
- For books, a table of contents and index can be helpful in assessing the coverage of the work.
For online sources, is the site complete or still under construction? Does the source seem stable, or is it likely to change much between the time you read it and the time your research is finished?
For online sources, if there is a print equivalent to the website, is there clear indication of whether the entire work or only a portion is available online?
- If it is a portion of the work have the quotes been taken out of context or otherwise misrepresented?

Mini Activity Check out this article in Wikipedia. Ignoring the warning at the top of the article's page: What elements in the article make it look scholarly? What missing element might have revealed this hoax sooner?

For a brief recap of the points covered above watch this video:

Test Your Knowledge

To help determine whether a source is suitable for academic research you should check:

The author's qualifications and affiliations.
Whether the source has been reviewed by an editorial board or peer-referees.
If the source contains graphs and charts

Answer:

a) 1 & 2
b) 2 & 3
c) All of the above

5.3 Scholarly and Popular Sources

What is a Scholarly Source?

A scholarly publication is one in which the content is written by experts for experts in a particular field of study - using the terminology of the discipline. Scholarly articles and books cite all their references and provide a comprehensive bibliography or list of works cited. They are typically published by academic presses or scholarly societies and are subject to some form of external review before publication, e.g., being reviewed by an editorial board or group of peer-reviewers. The purpose of publishing such work is to share original research or analyze others' findings. As a result, the content is much more sophisticated and advanced than what you will find in general magazines or popular books.

In short, scholarly work is:

written by experts for experts
written using discipline-specific terminology
based on original research or intellectual inquiry
provides citations for all sources used
published by a scholarly press or society
usually reviewed by fellow experts prior to publication

To see the typical components of a scholarly journal article check out the Anatomy of a Scholarly Article from North Carolina State University Libraries.

What is a Popular Source?

In addition to scholarly articles, books or other peer-reviewed sources of information, you will find many popular sources in your search results lists. Popular books and magazines are written to appeal to a wide array of readers and are much more informal in language, tone and scope. They are typically written by journalists or other professional writers and do not normally include citations or bibliographies. They usually contain consumer ads and images which are included solely to make the publication look attractive. Their purpose in being published is to make a profit for the publisher and advertisers. Examples include general news, business and entertainment publications such as Time Magazine, Business Weekly, Vanity Fair.

Note, special interest publications which are not specifically written for an academic audience are also considered "popular", i.e., National Geographic, Scientific American, Psychology Today.

Are scholarly sources better than popular sources?

The answer really depends on the nature of your assignment. Looking at a popular source is a great way to get ideas for your topic as well as some background information. Popular sources can also provide fascinating insight into social and cultural attitudes in a particular place and time - making them valuable primary source material for historians and sociologists.

On the other hand, you'll notice that many of your assignments specify that you must use scholarly materials. This helps ensure that you learn about your topic from reliable and accurate sources. In turn, you can develop your own ideas with the assurance that they are rooted in fact.

What to Check

Print Materials

Scholarly sources are easy to spot in print. There are visual and physical cues to help you out. For example, academic journals are often published in black and white, and text will take up most of the space on the page. You won't see any consumer advertizing or product samples and images will usually only appear if they are part of the article, such as graphs, charts and photographs provided by the author. Scholarly books usually have plain covers with few or no design elements.

Popular sources are equally easy to identify when you look at them in print. If a publication has slick graphics, glossy paper and ads for consumer products, you're probably looking at a popular magazine. A popular book will likely have a visually appealing book cover or dust jacket. They are designed to help the book sell - and you may also see designs or illustrations inside which only serve to make the book look attractive.

Online Materials

It can be trickier to decide what you have when the material is online. You won't have any physical clues to make your job easy so you'll have to scan through the work to make an assessment.

Questions to Ask

Is the author an expert in the field?
Are all quotes and references cited? Can you verify the citations?
Is there a bibliography?
Is the writing at a university level?
Is the publisher an academic press or scholarly society?
Are there consumer ads or product samples enclosed?
Who is the intended audience?
If the author conducted original research, is the data available in some manner for you to review?

Mini Activity Look at the two articles shown in this guide from Chalmer Davee Library. Look at the popular example: what three features make it clear to you that it really is a popular article? What features make it clear that the other example is scholarly?

Where to Look for Scholarly Sources

Limit to scholarly results

Summon

Summon, the Library's search engine: Simply type in your keywords and look at your results list. Go to Refine Your Search in the top left menu to check off either "Limit to articles from scholarly publications" or "Limit to articles from peer-reviewed publications."

Note, the scholarly limiter in Summon - or any other database/search service is not perfect. You may occasionally find a popular article which has mistakenly been included in a "peer-review only" results list. It's ultimately up to you to assess each source to make sure it's scholarly.

Databases

Databases sometimes contain articles from popular magazines or newspapers: Just like Summon, databases like this will have a box you can check to limit your results to content from "scholarly" and/or "peer-reviewed" journals.

Not sure which database to try? The Library's research guides have recommendations for the best databases for every discipline taught at UBC.

Google Scholar

Search engines like Google are a great source of popular materials but don't provide access to much scholarly information. Even if you find a few suitable items, they will be a very tiny proportion of the research which has been published on your topic.

Google does have a specialty search service called Google Scholar which provides links to a wide range of scholarly sources.
- Google Scholar looks a lot like Google, but many results will not be freely available. If the Library does not own or license an item, you will not be able to access it directly from Google Scholar. You can contact the InterLibrary Loan department if there's something you need that the Library does not have.
Go to Google Scholar from here, on the Library Website if you are working off-campus - this ensures that the system gives you access to everything the Library owns or licenses.

Books

Books from Academic Publishers: University presses and the educational/science units of major commercial presses also publish scholarly research - most of which is subject to review by editorial boards.

Note, sometimes the editorial boards are comprised of subject experts and sometimes they are comprised of scholars from a range of fields. Occasionally they will simply be professional editors.

To determine if a book has been peer-reviewed you will have to:

Look at the information about the editorial board which is included in the book or on the publisher's website.
Look at the affiliation and qualifications of the board members to see whether they are "peers" of the author or not.
- If the majority are not experts in the same field, the book has not been peer-reviewed.

For a brief recap of the points covered above, watch this video:

Test Your Knowledge

The best way to determine if a source is scholarly or popular is:

Check to see if it has glossy paper or product samples.
Check the author's qualifications, citations and language level used.
Use the scholarly or peer-review limiter when searching in a database or a search service like Summon.