Evaluating Resources and Research

Evaluating Online Resources

There are three great resources on the topic of evaluating online resources easily available so instead of writing something new I'd like you to please review the following links:

In general Wikipedia is NOT considered a reliable source for anything other than popular culture items.

Evaluating Research Articles

Is it a Report on Primary Research?

The first thing to check when looking for an article reporting on primary research is that your source is a scholarly journal and not a magazine.

Once you know that you have an article from a scholarly journal, the next thing to look for is whether or not all of the normal "parts" of a research article are present. While the parts are often easier to find in reports on quantitative research, they should be present in reports on qualitative research as well. You need to look for:

  • a general statement of the problem including why the problem is important
  • a review of previous research on the problem and why it doesn't solve the problem within the given situation
  • a clearly stated research question or questions
  • a description of the methods used in the study
  • results of the data analysis procedures
  • a discussion of the results including implications for practice and future research

NW Missouri State offers another description of the structure of a research article. If these parts are not present you likely do not have an article reporting on research. Just because an article is published in a journal does not automatically make it a report on research. Journals often print other types of articles as well as reports on research. What you may consider a "regular article" is probably a review of literature on a subject synthesized by the authors with their critique and suggestions. These are considered theoretical articles or commentary articles as opposed to reports on research.

Here is a tip for you: a research report article, while mainly describing the findings of a particular research study undertaken by the author(s), will *begin* with a review of the relevant literature in the area of the research. The reviews of literature found at the beginning of these articles are often good sources of additional articles. This is especially useful when you start by searching for the most recent articles on your topic and working backwards. If you see the same article (or various articles by one author) cited in multiple other articles it is a good hint that the older article is a foundational one on the topic. You can also see how many other articles have cited a given article in Google Scholar and several other article databases.

Evaluating

Research consumers need to evaluate research producers' interpretation of results carefully for unwarranted or overgeneralized conclusions. They need to examine four aspects of the research producers' conclusions: (a) predicted results (in the case of quantitative research), (b) unpredicted results, (c) statistical and practical significance, and (d) further research or replications of the studies (Hittleman & Simon, 2005, p. 202)

To critically evaluate the value of research you need to understand how the researchers got from point A to point B.

There are many times in your life when you just accept what you are told and feel that you may have little recourse to know if what you are being told is appropriate and accurate (i.e. doctors, lawyers, computer technicians, electricians, etc.). You may have a good relationship with your accountant and be able to trust what she tells you to do despite the fact that you don't understand why. You may get a second opinion from another contractor if the first electrician says your whole house needs re-wiring before you believe him.

When you are reading research published in reputable journals, you have the assurance that it has been either peer reviewed or editorially reviewed. While this usually means there are no glaring errors, it doesn't automatically make it True with a capital "T" and it doesn't automatically make it applicable to your situation. You could get a "second opinion" by reading another report on the same topic but then what do you do if they contradict each other? How do you decide who to believe or disbelieve?

You should start by giving a close reading to the review of previous research and theory relating to the problem. An article will never have the space to review all the related literature but there should be references to some of the key studies and theories in the area. Is the research current? Does the review include contradictory studies or alternate theories to balance the review? Is language indicative of a strong bias toward a particular viewpoint used? (While this is not uncommon in qualitative studies based on critical theory or a standpoint theory, in these cases the point of view is addressed in a researcher perspective statement.)

Next, you can look at the methods used. Are they appropriate to the research question at hand. Does the author provide enough information to determine if the data was gathered in a reliable and ethical way? How were the participants selected? Were the data analyzed in a way that provided opportunities for triangulation and verification?

Finally, you can look at the discussion section. Are the conclusions drawn really based in the results of the analysis? Are the implications appropriate or are they overstated? Do the conclusions actually answer the research question? How do they support or refute previous research noted in the review of the literature done by the author? Does the author provide good suggestions for future research to expand upon this study?

The more you know about a particular area and the more you know about research methods in general the easier this process will become.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License