Evaluating information

A vital step in the research process is to evaluate and filter the information you gather. There are important criteria to consider when selecting resources for their quality, reliability and relevance to your research. By the end of this section you will be able to critically analyse information resources and understand evaluation criteria.

Evaluation criteria: journal articles

There are five criteria to gauge the expertise and reliability of a source. Ask yourself these questions:

  1. Has the information been peer-reviewed?
  2. Are the references of high quality and sufficiently documented so that you can find the original sources?
  3. Is the information accurate?
  4. Has the author written their argument objectively and critically, minimising bias and misrepresentation?
  5. Does the author and source have authority and high impact?

Peer review

The quality of information may vary according to the source where the information is published. Information in magazines and newspapers may have less academic credibility than information in peer reviewed journals and research reports. Peer reviewed journals - also known as refereed or scholarly publications - require that the information is reviewed by several experts in the field. Other publications require that the information is reviewed only by an editor. Be wary of publications that undergo no review process at all.

How do you know if a journal is peer reviewed? Ulrichsweb is a global serial directory which lists over 300,000 periodicals. In this database you can easily find out whether a journal title is referred.

Activity – Ulrichsweb

  1. At the QUT Library Database page
    Screenshot of a search for Ulrichsweb
  2. Find and login to Ulrichsweb by searching for its name
  3. Conduct a sample topic or title search in the database, e.g. "nursing research"
  4. Note the results, some will have a referee's jacket (Refereed) alongside the title, indicating that these titles have some sort of peer review process.

Note that if the journal has a peer reviewed process this will be identified on the publisher's website.

Referencing

When reading scholarly or peer-reviewed publications you can expect to see references and citations to verify the facts and perspectives presented. If references are not available, then the information may be questionable. Remember that you may need to go to the original publication cited to gain the full perspective and to verify that research conclusions have not been misquoted or misrepresented in the citing source. Access to the original sources through the references is particularly important.

Accuracy

Authors may not be thorough or accurate in their coverage of information sources or research. Check that facts are quoted and represented accurately. Be aware that information may be given out of context or selectively chosen to align with the author's opinion or agenda.

Do not assume that any single piece of research is complete. By compiling notes on multiple authors and perspectives on the same issue, you increase your capacity to generate a comprehensive analysis of the theory or research data you are researching.

Is the information complete or abridged? Information is published either in its entirety or in a modified version. This could be an issue, particularly when considering news sources. When reading a literature review within a research work, be aware that the author will be quoting and paraphrasing from other research and publications. That author's perspective or coverage of those original sources may be incomplete or inaccurate. You may need to read original source documents to gain a true understanding of the content.

Objective

Scholarly writing is critical and presented in an objective style. Critical writing anticipates and responds to plausible arguments against the hypotheses of the author. Objective writing is clear and specific, avoiding emotionally charged language or vague and general language. Critical writing includes alternate theories or evidence on a topic and shows a comprehensive analysis of all influential views.

Author authority

An author may be an individual, a set of authors, a corporation, or a sponsoring agency such as an association or organisation. Any of these types of authors may have authority in different domains or to different degrees depending on the topic. Often within a group of authors, senior academics or researchers will be listed alongside student authors, highlighting the authority of the article. Look for bias in materials from corporations or organisations.

Some information sources, such as some news articles, blog posts, Wikipedia articles and magazine columns, do not indicate authorship or responsibility, or the credentials of the author are unclear. You can investigate an author's credentials by searching for their university or trusted organisation affiliation. You can also identify additional work by that author through database searching.

Evaluation criteria: web information

Information on the web may be published by anyone. There may be no editing or review of information by peers or publishers. Information from the web may be incomplete, inaccurate, or written by non-experts. It can be unstable and may disappear. It may be aimed to sell you a product or an idea. Consider author and authority, accuracy and completeness, stability, bias and misrepresentation, propaganda and commercial agendas when evaluating information on the web.

Author and authority

Some web sources clearly indicate who produced the information while others give no indication of authorship or responsibility. Without information about the author's credentials it is difficult to determine academic value.

You can search for information about an organisation in QUT Library databases, for example, use Factiva to search worldwide for articles about a company in newspapers such as The Wall Street Journal and transcripts from leading news sources such as BBC and CNN.

As a general rule, web addresses with '.edu' (education) or '.gov' (government) are easier to use to locate authentic authorship information than web sources with '.com' (commercial) or '.org' (organisation).

Accuracy and completeness

Be cautious of websites that provide no means of verifying accuracy and completeness. Accuracy is verified from references or citations for the information provided. Avoid websites that do not provide complete citation information. Remember you are creating an academic work.

Stability

Information published on the web changes frequently and may disappear altogether. However, the instability of web information may not be an indicator of its value to your research. If using web data, be sure to record your retrieval date and attempt to locate a version identifier from the metadata found in the source code. Some bibliographic management tools can automate the tracking of detailed information about a web page.

Bias, misrepresentation, propaganda and commercial agendas

Information on the web is frequently published by authors or organisations to advocate a particular point of view or sell a product or service. Web sites may be created for the purposes of community, discussion and political lobbying. These web sites may present only one specific point of view and may misrepresent or deliberately bias the information presented to further a particular agenda. Information from commercial sites may also be skewed or may omit content that is detrimental to the perception of the product or service. Publications from a lobby group or about a product or service should only be included if references are provided to data, other information sources or independent evaluation. If used, you should also include other sources to provide critical balance.