# 272 U3

The flashcards below were created by user atlborn80 on FreezingBlue Flashcards.

1. What are the 3 reasons for assessing IOA?
Consistency of measurement

Minimizes observer bias

reflects the quality of operational definition
2. What is the difference between agreement between observers and accuracy?  Is it possible to have one but not the
other?  Explain.
Agreement - how well data from independent observers correspond

Accuracy - How well the data reflects what actually happens

Yes it is possible, two people can make mistakes in the same way therefore you can have agreement but not accuracy

definition could be too broad, thus you have accuracy but not agreement
3. What is the difference between and reliability and validity? Can you have one
without the other? Explain.
Reliability - consistent measurement of the construct

Validity - measurement reflects the construct

• yes. you can measure something reliably and it not reflect the construct
• and you can measure a construct inaccurately
4. Is accuracy and validity the same? Why or why not?
No, you can accurately measure a bx that has nothing to do with the overall construct

measuring eye contact for reading comprehension
5. Complete the following sentence using the terms “accurate” and “reliable.” If data
are _________, they must be ___________. Why is the converse not true?
Give an example.
• accurate, reliable
• the converse is not true because two people can take data inaccurately in the same way which would make it reliable, but not accurate
6. Do you have to check IOA during each phase of a study?  Why or why not?
yes, IOA should be assessed in at least every phase of the experiment to ensure consistency
7. Be able to calculate IOA using the frequency ratio method, Point-by-Point
Agreement Ratio method (scoring agreements as occurrences and non-occurrences), and
frequency-within-interval method.
FREQ RATIO - smaller total/larger total X 100

Point By Point - Agreements/agreements +Disagreements X 100

Freq within interval ? pg 193
8. What does “chance” refer to?  How might
one handle expected “chance” levels of agreement?
level of agreement expected by randomly marking occurences

compute IOA using agreements on nonoccurences
9. In addition to “chance,” there are 4 other sources of bias that can obscure interpretation of interobserver agreement.
Describe each source of bias.
reactivity of reliability assessment - bx changes when you know your work is being checked

observer drift - change in the application of definitions of bx over time

Expectancies and Feedback - when observers expectations of change and feedback from experimenters combine, it may influence data taken by observer

Complexities of observations - the less complex the observations, the higher the agreement
10. Define reactivity and observer drift.
Reactivity - changes the people make when their bx is being monitored or evaluated

Observer Drift -  change in the application of definitions of bx over time
11. Given that interobserver agreement provides no information about either accuracy
or reliability, why do you suspect that this is the most common approach to evaluating measurement procedures and data?
requires the least amount of response effort  and is the best representation of the construct of accuracy and reliablity
12. When using an interval agreement procedure for calculating IOA, explain how the
percent agreement can be inflated by especially frequent or infrequent
responding.
especially frequent refers to that fact that agreement will be high because of the number of intervals in which the behavior occurs.  If it is occurring a lot high agreement may happen

INfrequent responding referes to agreements based on non-occurrence which would be the same for low freq bxs but in reverse.
 Author: atlborn80 ID: 202073 Card Set: 272 U3 Updated: 2013-05-04 05:52:24 Tags: 272 U3 Folders: Description: 272 U3 Show Answers: