Research Methods
Home > Flashcards > Print Preview
The flashcards below were created by user
faulkebr
on
FreezingBlue Flashcards. What would you like to do?

Analysis of variance (ANOVA):
 Estimate of systematic variance
 Estimate of error variance
 Calculate a ratio

Simple, one way analysis of variance (ANOVA)
F= systematic var. + error var. =
Error variance
 F = Between groups var.
 Within groups var.

Between groups variance=
Within groups variance=
 Between= variance of group means
 Within= average of variance found within each group

Simple analysis of variance (anova)
*Scores almost always vary (differ) from subject to subject... 2 reasons:
 1. Systematic variance
 2. Error or unsystematic variance

1. Systematic Variance
Difference due to different levels of I.V.

2. Error or unsystematic variance
Differences due to uncontrolled, random factors (not confounding)

Error or unsystematic varianceuncontrolled variance:
 1. individual differences
 2. random variations in testing
 3. measurement error
 4. experimental treatment not the same for each subject

If IV influences DV:
systematic variation of i.v. should cause systematic variation of the d.v. = systematic variance

Differences in iv should cause:
differences in group means

If IV doesnt have an effect:
group means are not likely to be exactly the same
No systematic var.

An effect means:
that there is a real difference in means or that IV and DV are correlated

If an effect, variations in the IV:
would not reduce variance within groups variance

If an effect:
Assume that differences in group means should be:
greater than differences within any single group

If an effect:
Difference within a group :
just due to chance or uncontrolled factors

If an effect:
Differences betwen groups due to:
controlled and uncontrolled factors

Understanding basics of anova tell you how to design and run experiments:
 1. large differences between groups
 2. small differences within groups

Would size of sample influence F ratio?
 No, not directlynot in the equation
 Influences whether F ratio is statistically significant

For a significant effect:
 1. large differences among group means
 2. small differences within each group (not confound idea)
 small individual differences
 reliable tests or measures
 consistent application of each level of iv

Effect =
difference (or variability) in means

Possible to see an effect if the study includes:
a comparison of the influence of different levels of a predictor or independent variable

Factorial design more than 1 IV:
 Each level of each IV occurs with every other level of all the independent variables
 Uses all possible combinations of levels of 2 or more independent variables

Main effect:
the effect (differences in means) of one iv, averaging,ignoring, or collapsing over the other iv's

Factorial Designs
Advantages:
 1. Economy
 2. Experimental control
 3. Check generality of the effect of an IV

Factorial designs:
Adv.
1. Experimental control
reduce error variance

Interaction:
the effect of one iv depends on the level of another iv

Factorial design:
Look for three things if there are two i.v.:
 main effect of iv A on dv
 main effect of iv B on dv
 Interaction between A and B

Describing the results of factorial designs:
 1. Set up a table of means for each condition
 2. Draw a figure
 3. Describe the means for each main effect followed by a description of the results of the ANOVA

Statistical Hypothesis Testing:
 1. Any t, F, U, orr could be due to chance/luck
 2. If a result (a, t, F...) occurs by chance infrequently decide result is statistically significant
 3. Typical reasoning
 4. examples

3. Typical reasoning:
often decide that if an outcome is unusual/rare must be due to more than chance.

Null hypothesis:
presumed true unless statistical evidence suggests otherwise

The null hypothesis is tested because:
we have information about chance, not how the independent variable works.

Type I error:
Rejuect H_{o} but it is actually true

Type II error:
Faile to reject the H_{o} but H_{o} is false

What evidence must you collect in order to conclude that X is a cause of Y?
 1. Show that changes in Y didnt occur until change in X (temporal precedence rule)
 2. Show that X and Y are related (covariation rule)
 3. Rule out other explanations forthe relationship between X and Y (internal validity rule)

Characteristics of a true experiment:
 1. manipulation of X
 2. Comparison of the effects of various levels of X on Y
 3. Subjects begin the experiment equivalent on all levels
 4. control over all other important variables so that all subjects are treated the same except for X (random assignment)

Program Evaluation Basic Strategy:
To show the program caused a change in client
 a. Change in client occurred after the introduction of the program
 b. participating on not participating in program covaries with client success
 c. Rule out other explanations for the relationship between the program and client success
 d. plausible causal mechanisms linking program to client success

If there is an effect: F =
greater than 1


More diversity =
more error variance

Interrupted time series design
 often encountered in quasi experimental research
 Have a single experimental group for which we have multiple observations before and after naturally occuring treatment.
 Instead of observing 1 or 2 3rd grade classes we could observe 3rd grade classes over several years
 We need to know when the time series is interrupted by some treatment
 Then we compare observations beforea nd after the treatment to see whether it had any effect.

Threat to internal validity in case studies:
 Source of causation;
 baseline condition maturation
 history
 selection bias

Ways to enhance internal validity in case studies
 Deviant case analysis (a non equivalent control)
 detective work

Threat to internal validity for interrupted time series:
 changes in participants and environment
 delayed effects

Ways to enhance internal validity in interrupted time series:
 nonequivalent control group
 detective work

Threats to internal validity for subject variables
 dimensions on which to match
 regression artifacts

Ways to enhance internal validity for subject variables:
 matching
 include true independent variable
 see interactions

Threats to internal validity in Age as a variable:
 Confoundings with time of testing
 generation of birth

Ways to enhance internal validity in Age as a variable:
 Cross sequential design
 include a true i.v. & seek interaction
 Converging operations

______ _________ masks true behavior in quasi experiments and matching studies.
regression artifacts

All non parallel lines=
interaction

p =
 Significance level
 it does notindicate repeatability

one way anova Error Variance (denominator) estimated by:
Calculating within groups variance (variance within each group pooled)

Krauter pseudo F ratio=
 range of group means
 (range of scores in gp 1+ gp 2+gp 3 etc..
 # of groups

2 reasons why scores in an experiment almost always vary:
 1. Systematic variance
 2. Error or unsystematic variance

MannWhitney U test
 a simple inferential statistic that can be used in place of a t test.
 It can be used in many instances in which you have tested twoindependent groups of subjects

How to do mann whitney u test:
 1. put the scores in order from smallest to largest
 2. For each score in the group w/ the lower mean, count the # of scores that are smaller in the other group. Thus you'll have 1 # for each score in the group with the lower mean that tells you how many scores are smaller in the other group than that score\
 3. Addthese numbers together
 4. For each score in the group with the lower mean, count the # of scores in the higher group that are tied or are the same. Add these ties together and divide by 2
 5. Add theresults of C and D together to obtain the mann whitney u

Program Evaluation:
 Is a particular program actually delivering the services it is designed to deliver?
 Is a law having the desired effect?

Program Evaluation
Basic Strategy
To show program caused change in client;
 a. Change in client occurred after the introduction of the program
 b. Participating or not participating in the program covaries with client success
 c. Rule out other explanations for the relationship between the program and client success
 d. Plausible causal mechanisms linking the program to client success

What kind of design is:
After school program and social skills...Every one gets the same treatment and all measured the same way
 One shot case study
 There is no comparison
 Descriptive research

If there is success in a one shot case study design, does it show the program caused the success?
 Does it answer...
 Change in the client occurred after intro to program
 Having or not having program covaries with client success
 Rules out other explanations for relationship between program and client success
 Extremely low internal validity
 Cant even show that the client changed during the program

Ongoing flow of events interrupted by the introduction of treatment
interrupted time series

Pre or non experimental design
answers the questions...
 pretest post test design (beforeafter) o x o
 Potentially suffer from all the threats of within subjects design
 change occurred after intro to program
 Program/no program varies with client success
 rule out other explanations for the correlation b/w program/noprogram and client success


O X O
Pret test post test

X O
O
Static Group Comparison (ex post facto)

r O X O
r O  O
Pretest posttest control

Quasi Experimental Design
Might manipulate X, but dont compare groups that are formed based on random assignment
More opportunity to discredit alternative interpretations of data

Quasi experimental designs: To change nonexperimental (pretestposttest) to quasi exp. design
 Add observations
 1. Additional times before and after program introducedtime series studies
 OXO > OOOOOOOXOOOOOO
 2. Additional people who havent received the program (non equivalent control group  no random assignment  STATIC GROUP COMPARISON
 OXO > OXO
 OO (non equivalent control group

Time Series Design
Get many measures before and after some natural or planned intervention

Pretestposttest design also becomes a quasi experimental design if:
add a comparison group

To improve a non experimental design:
Increase number of observations, add comparison group or both

Program evaluationquasi experimental designs
Can distinguish between effect of program and many other variables:
 1. comparison and "program" group have same amount of time to mature
 2. history should influence both groups equally
 3. testing should influence both groups equally

problem with program evaluationquasi experimental designs:
 Finding a good comparison group...
 Cant randomly assignbetween groups variable: ex post facto

Possible selection bias problem:
groups different because of the way they were selected or assigned to groupss

Program evaluation:
Quasi experiments summary
 1. no random assignment
 2. quasicomparison using additional measures before and after or non equivalent control groups
 3. moderate control
 4. often field based
 5. may never be able to eliminate confounding variables