RM Review For Comps

Card Set Information

RM Review For Comps
2012-06-17 01:31:22
Research Methods

Research Methods for Comps
Show Answers:

  1. Definition
    • A way of knowing characterized by the attempt to apply objective, empirical methods when
    • searching for causes of natural events. 
  2. Probabilistic
    Statistical determinism
    • Using probability to
    • determine if the events whether causal, predictive or simple relational are
    • greater than chance.
  3. Objectivity
    • without bias
    • from experimenter and participants
  4. Data-driven
    • conclusions are
    • based on the data-- objective information.
  5. Empirical
    • All information
    • is based on observation
  6. Objectivity
    • Observations
    • must be free from bias.
  7. Systematic
    • Observations
    • made in step-by-step fashion.
  8. Controlled
    • Potentially
    • confusing factors are eliminated
  9. ØLawful: 
    Every event can beunderstood as a sequence of natural causes and effects.
  10. ØDetermined: 
    The event or behavior issolely influence by natural causes and does not depend of choice or "freewill."  Organisms behave in apredictable, lawful way.  
  11. ØUnderstandable: 
    The behavior can beunderstood and part of the explanation of an event or behavior cannot byaccepted as a mystery or an unresolvable contradiction.
  12. ØDeduction:
     reasoning from a set ofgeneral statements towards the prediction of some specific event.  Based on a theory, one can deductbehavior given particular conditions.
  13. ØHypothesis: 
    the prediction aboutspecific events that is derived from the theory.
  14. ØInduction: 
    The logical process ofreasoning from specific events to the theory 
  15. ØProductivity: 
    generating a lot ofresearch studies. 
  16. ØFalsification: 
    They have to be able to beproven wrong.
  17. ØParsimony: 
    Include the minimum # ofconstructs (ideas or concepts) and assumptions that are necessary to explainthe phenomenon adequately and precisely. lthe simplest theory is preferred. 
    Principle A: Beneficence and Nonmaleficence

    • Principle B: Fidelity and
    • Responsibility

    Principle C: Integrity

    Principle D: Justice

    • Principle E: Respect for
    • People's Rights and Dignity
  19. look at the chart on nominal ordinal etc
  20. Construct: 
    hypothetical factor that can’t be observeddirectly; it's existence is inferred from certain behaviors 
  21. OperationalDefinition: 
    A way to attach asystem of measurement in a way that can be replicated and which is a faithfulPROXY of the construct.
  22. ØReliability: 
    reproducibility of ameasurement; the extent to which measures of the same phenomenon are consistentand repeatable; measures that are high in reliability will contain a minimummeasurement error.lExample: Reaction time to answer with a yes or no buttonif the word on a screen is an animal or not.
  23.  ØValidity: 
    The extent to which ameasure of a construct truly measures that construct and not something else.Example: Does this reaction time truly measure speed ofprocessing in semantic memory?
  24. Classical Reliability Theory assumptions? < X (Observed
    score) = True score + Error>  4
    1. True score is constant.

    2. Error is random.

    • 3. The correlation between
    • the true scores and errors is 0.

    • 4. The correlation between
    • errors of different measurement occasions also equal 0. 
  25. Classical Test orReliability Theory ?
    • ØClassical Test or
    • Reliability Theory is a special case of Generalizability Theory (GT) in which error is made up of only one source.

    • ØIn GT, error is separated
    • into pieces, each of which can be estimated (if we collect the data right). 
    1. Face

    2. Content

    • 3. Criterion (predictive and
    • conncurrent)

    4. Construct Validity
  27. Construct Validity Definition
    • ØDefinition: When the measure being used
    • accurately assesses some hypothetical construct; refers to when the construct
    • itself is valid; refers to whether the operational definitions used for
    • independent and dependent variables are valid.
    • ØMost rigorous type of
    • validity
    • ØCan be used as part of the hypothetico-deductive scientific method where the data we collect
    • is dictated by the conceptual definition of the construct and the data then
    • helps refine the nature of the construct.
    • MTMM is an approach to assessing the construct validity of a set of
    • measures in a study. 
  29. Two subcategories of construct validity discussed.
    ØConvergent validity: the degree to whichconcepts that should be related theoretically are interrelated in reality. ØDiscriminant validity: the degree to which concepts that should not be related theoreticallyare not interrelated in reality. 
  30. You can assess
    both convergent and discriminant validity using the MTMM
    LOOK at slide on the MTMM
  31. External Validity:
    • ØDegree to which researchfindings can be generalized beyond specific context of the experiment 
    • ØIt addresses the issue ofbeing able to generalize the results of your study to other times, places, andpersons. 
    • lE.g., If you conduct a study looking at heart disease inmen, can these results be generalized to women? 
  32. Internal
    • ØThe degree to which the experiment is
    • methodologically sound and free of confounding variables.

    • ØOnce it has been determined that the two
    • variables (A & B) are related, need to determined causality.

    ØDoes A cause B?

    • ØIf a study lacks internal validity, one can
    • not make cause and effect statements based on the research
  33. common threats to internal validity? <6>
    1. History Threat

    2. Maturation Threat

    3. Testing Threat

    4. Instrumentation Threat

    5. Mortality or Attrition Threat

    6. Statistical Regression to the Mean Threat
  34. advantages and
    disadvantage (1) of doing a Between subjects (2) design?

    • ØEach subject in a condition
    • is “fresh” and not “contaminated” by a previous treatment condition


    • ØTakes a lot more subjects
    • than a within subject design.

    • ØDifferences btwn groups may be due to
    • individual differences btwn groups and NOT because of
    • the IV. 
  35. 4 Ways to have equivalent groups:
    1. Random assignment

    • 2. Block randomization- a
    • method of randomization in which the numbers of subjects are balanced over all
    • conditions and the end of each subject assignment block rather than at the end
    • of the testing schedule; each block contain each treatment once.

    • 3. Matching (1 homogenous or
    • block matching)

    4. Within-subject design 
  36. advantages of a
    within subject design?
    • 1.One reason for doing a
    • within subject design is that it requires less subjects to obtain a good level
    • of power.  You may be interested in
    • studying a population that is scarce in subjects: e.g., Psychotic Major Depression
    • Pts, PTSD pts.

    • 2.“Perfect matching”
    • occurs. 

    • 3.Error variance is
    • reduced. 

    • 4.The reduction in error
    • variance produces an increase in power.
  37.  Two types of  Sequence or order effect: effect due to the order of treatment conditions are given to each subject. Aka practice effects.
    • ØProgressive effects: Performance changes
    • steadily from treatment condition to the next treatment condition.

    • ØCarry over effects: the effects of a
    • previously administered condition on a subject’s performance on a subsequent
    • condition in a within-subject design.
  38. Controlling for these sequence effects
    • ØCounterbalancing: systematic arrangement of
    • treatment conditions designed to neutralize sequence effects. 

    • lComplete counterbalancing: all
    • possible sequence combinations of treatment conditions are included.

    • Latin square: a form of
    • incomplete counterbalancing
  39. Null
    • Statistical
    • hypothesis describing the population parameters that the sample data represent
    • if the predicted relationship does NOT
    • exist. 
  40. Alternative
    • Statistical hypothesis
    • describing the population parameters that the sample data represent if the
    • predicted relationship does exist.
  41. Type I error
    • ØStatistical decision-making
    • error in which large amount of sampling error caused rejection of H0 when
    • H0 is true.

    • ØThere is relationship btwn IV & DV when it doesn’t
    • exist.
  42. Type
    II error:
    • ØStatistical decision-making
    • error in which large amount of sampling error caused acceptance of H0 when
    • H0 is false.

    • ØNo relationship btwn IV & DV when in fact it
    • does exist.
  43. Factors Affecting Power <6>
    • 1.The alpha level.  Reducing the alpha level will reduce
    • the power of a statistical test. 

    • 1.Eg., moving alpha from .05 to
    • .01.

    • 2.One-tailed versus 2-tailed
    • alpha test.

    • 3.As sample size increases, so
    • does power.

    • 4.Reduce error variance will
    • increase power.

    • 5.Greater subject variability
    • will decrease power.

    • 6.Using match, versus
    • within-group, versus between subject, versus mixed design
  44. T-test and ANOVA definitions? 
    • t-test: A hypothesis-testing
    • procedure used to determine if mean difference exist for only 2 treatment
    • conditions or populations.  Good
    • for only 1 IV.
    • Analysis of Variance (ANOVA): A hypothesis –testing procedure used to determine if mean
    • differences exist for 2 or more treatments or populations. 
  45. Unlike the t-test, the ANOVA can:  <4>
    ØCompare more than 2 groups(multi-level design).ØFind non-linear effects ofthe IV. ØEvaluate the effects of morethan one IV.ØEvaluate interactionseffects between 2 or more IVs.
  46. So we found that
    our results are significant. How much of an effect does IV have on
    • Effect size: To obtain the ratio,
    • estimate the population variances from the data of an experiment.

    s2A     = the estimated population treatment effects

    • s2S/A =
    • the estimated population error variance
  47. What if we want to
    test 2 IVs at the same time? 
     We do a Factorial Experiment.
  48. The Advantages of Factorial Designs (3)
    ØJoint manipulation of IVs


    • Saves time, money and effort
    • by doing one bigger experiment (2 or more Ivs)
    • than several smaller experiments (only 1 IV).
  49. Main effects (ME): 
    the separate effect ofeach independent variable AVERAGED over the levels of the other independentvariable. (AVERAGED EFFECTS)
  50. Simple effects (SE): 
    The variability among thetreatment means associated with one independent variable at a particular levelof the other independent variable. AKA simple main effects.
    When the effect of one of the IVs on the DV is not the same atall levels of the second IV.When an interaction ispresent, the ME is NOT representative of thecorresponding SEs.  
  52. look at graphs
  53. simple interaction AND AXBXC interaction (3-way interaction)
    • ØA simple
    • interaction is the interaction of 2 of
    • the independent variables with the third variable held constant. 

    • Ø An AXBXC
    • interaction (3-way interaction) is present when the simple
    • interactions btwn 2 of IVs are NOT the same
    • at all levels of the third IV.
  54. Advantages of a
    Mixed Factorial Design
    Minimizing Carryover Effects.

    • 2. Still need less subjects that complete
    • between.

    Disadvantages: Carryover effects, more subjects than within
  55. TRUE” EXPERIMENT <4 factors>
    • ØRandom
    • sampling
    • -a sampling
    • method in which each member of a set has independent chances to be selected.

    • ØRandom
    • assignment
    • -randomly assign
    • subjects into the control group and the treatment group.

    • lThe strength of randomization is that it creates two or
    • more groups that are equivalent in the very beginning on the average on all characteristics.

    lTrue for long term and reasonable size samples.

    • ØExperimenter
    • manipulation
    • -directly
    • manipulate variables to test cause-and-effect relationships

    • ØExperimenter
    • control
    • -control of all
    • other extraneous variables that might impact the dependent variables. 
  56. Common
    characteristics of quasi-experiments: <2>
    • ØMatching
    • instead of randomization is
    • used.

    • lE.g., studying effects of a new police strategy in one
    • town and finding a similar town.

    • •That other town
    • would have citizen demographics that are very similar to the experimental town.
    • It is a comparison group, and this matching strategy is sometimes called nonequivalent group design.

    • ØGroups
    • through self-selection.

    lIndividuals could enter treatment levels through self-selection, because they are in a particular category

    lProblem: self-selection or regression to the mean are alternative explanations for results
  57. Time series
    • ØSeveral observations over
    • time. While you may have some type of experimental intervention, often
    • "nature" does the experimenting for you.

    • ØMost common type of
    • longitudinal (over time) research found in criminal justice.

    ØCan be interrupted or noninterrupted.

    • lBoth examine changes in the dependent variable over time,
    • with only an interrupted time series involving before and after measurement.

    lE.g., Crime rates as a new law is taking effect. 
  58. Interrupted
    time series design, noninturrupted, and switching
    • Group 1: O1, O2, O3, O4, T, O5, O6, O7, O8ØE.g., Number of minoritiesaccepted into a university before and after a new change on admission policiesNoninterrupted timeseries design:there is NOprogram or treatment but performance is measures several times.Group 1: O1, O2, O3, O4, O5, O6, O7, O8ØE.g., Number of minoritiesaccepted into a university over a course of 8 years with no change orintervention
    • Interruptedtime series design with switching replications: the program or treatment isreplicated at a different location and at a different time.Group 1: O1, O2, T,O3, O4, O5, O6, O7, O8
    • Group 2: O1, O2, O3, O4, O5, T,O6, O7, O8
    • ØE.g., 2 similar colleges,examining # of minorities accepted before and after change of admissionpolicies.  The change occurs atdifferent time points.
  59. Correlation and what can we study with correlation?
    • Ø: a statistical procedure that is used to
    • measure and describe a relationship between two CONTINOUS
    • variables.

    • ØUsually the two variables are naturally
    • occurring events in the environment. 
    • They are usually not manipulated or controlled by the experimenter.
    • ->  1. Prediction.

    2. Validity.

    3. Reliability.

    4. Theory Verification
  60. What are some of
    the issues about r? <5>
    • 1. The value of a
    • correlation can be affected greatly by range of scores in the data.

    • ØCeiling effect: when everyone scores very high on a measure
    • and there is little variability

    • ØFloor effect: when everyone scores very low on a measure
    • and there is little variability

    ØRestriction of range: little variability

    2. Measurement error.  The more error, the smaller the r

    • 3. One or two extreme data
    • points (outliers) can greatly effect the value of a correlation

    • 4. If I square r, I get r2 aka coefficient of
    • determination: the
    • proportion of variability in one variable that can be determined by the other
    • variable

    • 5. Linear transformations:  It does not matter if scores of either variables are added
    • or multiplied by a constant, the r btwn the 2 variables will remain
    • the same.
  61. Equation for a Linear

    Y = bX + c
    • ØY is the dependent variable.
    • criterion
    • variable.

    Øb is the slope. 

    • lIt determines how much the Y variable will change when X
    • increases

    lIt describes the direction of the relationship. (+ or -).

    • ØX is the independent
    • variable. predictor
    • variable.

    Øc is the Y-intercept.
  62. ØRegression & Regression line: 
    • -the statistical techniquesfor finding the best-fitting straight line. 
    • -the straight line thatbest describes the linear relationship between two variables. 
  63. Naturalistic Observation: 
    Participant Observation: 
    • Studying the behaviors ofpeople as they act in their everyday environments. 
    • Researcher will join a groupbeing observed, as one of the participants
  64. Sampling
    Behavior ØGeneralization of
    observations (external validity) depends on how behavior is sampled.
    • 1. Time Sampling: choosing time intervals for
    • making observations either systematically or randomly.

    lUsed when the events of interest happen infrequently

    •   2. Situation or Event Sampling: studying behavior in
    • different locations and under different circumstances.

    lEnhances the external validity of findings.

    • lWithin situations, subject sampling may be used to observe
    • some people in the setting.
  65. What are the risks
    of Observational Research? <3>
    1. Influence of the Observer

    • ØIf they know they are being
    • observed, the behavior may not represent normal behavior.

    • ØTo control reactivity:
    • unobtrusive measurement, adaptation, and indirect observations of behavior.

    • ØMust consider ethical issues
    • when controlling reactivity.

    2. Observer Bias

    • ØOccurs when observers’
    • biases determine which behaviors they choose to observe and when these
    • expectations lead to systematic errors in identifying behavior.

    • ØExpectancy when they are
    • aware of study’s hypotheses.

    • ØMust recognize that it may
    • be present.

    • ØReduced by keeping observers
    • unaware of study’ goals.

    3. Absence of Control

    • ØMore difficult to make
    • conclusions about the observations.
    • Passive Observation: Researcher does
    • not attempt to control or change situation when making observations
    • Structured observations:  Researcher sets
    • up a specific situation making observation
    • Field expeeriement: person controls by minipulating the IV 
  67. What if I only
    want to study one person with very rare and unique characteristics? Instead of
    doing an experiment, I will do a case study.
    • ØThe case study is an
    • intensive description and analysis of a single individual.

    • ØCapable of uncovering causal
    • paths and mechanisms, and through richness of detail, identifying causal
    • influences and interaction effects which might not be treated as operationalized variables in a statistical
    • study.

    • ØRecommended as part of a multimethod approach
    • ("triangulation") in which the same dependent variable is
    • investigated using multiple additional procedures.
  68. What if I only want to study one
    person with very rare and unique characteristics?
    • The Case Study Method: ØCase study: an intensive
    • description and analysis of a single person

    • ØCapable of uncovering causal
    • paths & mechanisms, & through richness of detail, identifying causal
    • influences & interaction effects which might not be treated as operationalized variables in a study.

    • Single-case (N
    • = 1) Experimental Designs

    • ØBehavior following treatment
    • is compared to the baseline behavior to determine if the treatment is
    • effective.
    • ØSuppose a treatment was
    • designed to decrease the frequency of a behavior

    • ØAlthough the data suggests
    • the treatment was effective, some other factor could have caused change of
    • behavior.

    LOOK over
  70. Advantages and disadvantages
    • Advantages
    • of the Case Study Method

    • ØRich source of ideas for
    • developing hypotheses

    • ØOpportunity for clinical
    • innovation

    • ØMethod for studying rare
    • events

    • ØPossible challenge to
    • theoretical assumptions

    • ØTentative support for a
    • psychological theory

    • Disadvantages
    • of the Case Study Method

    • ØDifficulty drawing
    • cause-&-effect conclusions (limited internal validity)

    • ØPossible biases when
    • interpreting outcomes due to observer bias

    • ØProb of generalizing findings
    • for one person (limited external validity)
  71. What if I want to
    predict group membership based on results of a particular measure? 
    Use ROC CURVES: Sensitivity & Specificty
  72. ROC CURVES: Sensitivity & Specificty
    • Sensitivity: a ratio or proportion of the true-positive
    • test results divided by all patients with the DOS.
    •   sensitivity = (a / (a + c))

    •   where: a  =
    • true positives

    • a + c = true positives+ false
    • negatives = everyone with disorder

    • The better the sensitivity
    • of the test, the fewer the false negatives.

    Specificity: a ratio or proportion of  the  true-negative test results divided by all patients without the disorder.

      specificity =(d / (b + d)) 

    •   where: d =
    • true negatives

    (b + d) = sum of (false +s, true -s) = those w/o disease

    • The better the specificity
    • of the test, the fewer false positives.
  73. Finally, what
    should I have learned in
    Research Methods?
    • 1.Know which study design to use for different
    • research questions.

    • 2.Start to develop the research and statistic
    • language/ vocabulary.

    • 3.Keep these Research Methods concepts in mind
    • to help you conceptualize what you will learn in Statistics 1 and 2.

    • 4.How to develop a research study design in
    • psychology that is well thought out and ethical

    • 5.Know that all studies are not perfect. Using
    • the knowledge from this course, you should be able to critically analyze any
    • study, know its strengths and weaknesses, and NEVER take what they say at face
    • value.