Friday, January 30, 2009

Week 4: Measurement and Sadistics... I mean statistics

Blog Question: What distinguishes Quantitative from Qualitative designs, what is the difference between “validity” and “reliability,” and what is meant by the terms “probability” and “significance?”

Relating Variables (Quantitative) v. Describing Variables (Qualitative)

Now, I know the title of this section Relating v. Describing may be too simplistic, but for now I will keep with it. More importantly, the distinction between the types of research is beyond a math/not math approach.

Quantitative research is based heavily upon establishing a relationship and the strength of the relationship between variables. Unfortunately, transportation fields are heavily quantified. There is an expanse of quantitative research establishing the relationship between speed and automobile fatality rates, or in my latest experience, I established the relationships of growth of marine port activity, world production and population. Goubril uses the example of online manuals and the relationship between experience and problem solving. Most importantly, qualitative research allows for the research to determine if there is truly a reason to investigate a problem. In particular, whether or not one can reject a null hypothesis is particularly useful, especially in dispelling chanch (Williams 56). Although, the test of a null is often underutilized by my own personal opinion. Morgan appears to divide quantitative research into correlational and experimental. Correlational is the same definition as qualitative in establishing relationships between variables and experimental looks for cause and effect.

I found the better definition of qualitative research to be just outside the readings in the 3rd chapter of Lauer and Asher. Lauer and Asher define qualitative research as “to give a rich account of the complexity of … behavior” (45). In particular, it is best to “expose the blindness and gullibility of specious quantitative research” (46). The problem with some (if not many) quantitative research is when the research makes and inaccurate jump to generalize to claim cause and effect (46). As a result, qualitative research asks different questions, normally referring to how or why a phenomenon exists. Morgan argues that qualitative research enables a “investigate the process” or “describe features”.

On a personal note, I find it better to incorporate both “qual” and “quant” into my research, particularly since planning is comprised my both the quantitative and qualitative disciplines.

Validity v. Reliability

To keep it simple, validity is the “ability to measure whatever it is intended to assess” (Lauer & Asher, 140). Reliability is the measurement of agreement (134). Each of the terms have separate subdivisions. The tree types of reliability are equivalency, stability, and internal consistency (135). The types of validity are content, concurrent, predictive, and construct (141). Now the differences between the two are based on their definitions, reliability is the “closeness of the data tone another, while validity is the closeness of the data to the intended target. However, there cannot be validity without reliability. Therefore, reliability influences validity. A very simple example is the bull’s-eye analogy (I will use it from Singleton and Straights Approaches to Social Research p. 94 since it was a very useful example for me a while back). If a marksman shoots a target and the many shots he takes are randomly dispersed, then there is low reliability and low validity (as well as high random error and low systematic error). But if the marksman shoots and the shoots are clustered but not on the bull’s-eye, there is high reliability but low validity because he missed the target. There is also the likelihood of high systematic error. Now if the marksman clusters his shots within the bull’s eye, then there is both high reliability and high validity.

Probability and Significance

Probability is simply the chance or percentage that something may occur.

Significance is referring to a specific probability, the acceptable probability, of making an error. Specifically Type I error, but Type II errors be included. Williams talks about a 5% or .05 level of significance, but this is referring to that there is a five percent chance that we have rejected the null but the null actually is true (a Type I error). There are specific tests for Type II errors, but there more important issues is that the more restrictive probability one uses to determine the level of significance or LOS (ie, .1, .05, .01) the there is a reduction in the likely hood of error type. A higher LOS of .01 has a smaller change of Type I error but an increased change of a Type II error, but a .1 has less of a Type II error but a higher probability of a Type I error. If anyone want more knowledge about this, please take the torture that is ExStat 801 :D

Edit: Dang I hate Word '07 formatting issues

1 comment:

  1. John,

    First, I'm interested in hearing more about your experience with quantitative studies, like, for instance, what exactly were you looking at in the marine port activity. Were you studying fatalities? These kind of examples can help not so much with the shaping of our goals as a class (I agree with Miller that what is often done is not always ideally done) but with my understanding of how these types of practices are applied by researchers. To be blunt, I'd love to hear which are particularly helpful and which are particularly painful.

    Second, I have to thank you for the bow and arrow analogy. I wish I had read that earlier.

    A follow up question: what if most of the arrows hit the bull's-eye, but Robin can't repeat the trick?

    Let me update the terms: I was playing Wii tennis over the holidays and I went through a spell of being able to flick my serves in such a way that I'd get little smoke trails to follow the balls. They were almost unreturnable. I kept this up for about two sets. Then I crumbled. I couldn't work one smoke trail for the life of me. My serves were flat and slow. Was my technique valid? Reliable? Both for a time, right? So how long or how longly accurate does something need to be before becoming valid/reliable?

    You've clarified and complicated this question for me, which is always the sign of valuable discussion. So thank you.

    Lastly, talk about time makes me wonder about the value of qualitative research if we see it mainly as a voice for variables and problems. With enough time, it seems, everything will become variable. How can qualitative research reach out to the relationships, to the solutions? My background has trained me that the inverse is true. That quantitative research is the better to to establish a problem, or to define what needs to be probed for relationships, causes and effects. Qualitative practices then try to impose some sense on the mess.

    What's at odds, I'm sure, is my definition of "research" and "practice."
    But it's a worthwhile impasse, this.