-
Analysis of variance (ANOVA):
- Estimate of systematic variance
- Estimate of error variance
- Calculate a ratio
-
Simple, one way analysis of variance (ANOVA)
F= systematic var. + error var. =
Error variance
- F = Between groups var.
- Within groups var.
-
Between groups variance=
Within groups variance=
- Between= variance of group means
- Within= average of variance found within each group
-
Simple analysis of variance (anova)
*Scores almost always vary (differ) from subject to subject... 2 reasons:
- 1. Systematic variance
- 2. Error or unsystematic variance
-
1. Systematic Variance
Difference due to different levels of I.V.
-
2. Error or unsystematic variance
Differences due to uncontrolled, random factors (not confounding)
-
Error or unsystematic variance-uncontrolled variance:
- 1. individual differences
- 2. random variations in testing
- 3. measurement error
- 4. experimental treatment not the same for each subject
-
If IV influences DV:
systematic variation of i.v. should cause systematic variation of the d.v. = systematic variance
-
Differences in iv should cause:
differences in group means
-
If IV doesnt have an effect:
group means are not likely to be exactly the same
No systematic var.
-
An effect means:
that there is a real difference in means or that IV and DV are correlated
-
If an effect, variations in the IV:
would not reduce variance within groups variance
-
If an effect:
Assume that differences in group means should be:
greater than differences within any single group
-
If an effect:
Difference within a group :
just due to chance or uncontrolled factors
-
If an effect:
Differences betwen groups due to:
controlled and uncontrolled factors
-
Understanding basics of anova tell you how to design and run experiments:
- 1. large differences between groups
- 2. small differences within groups
-
Would size of sample influence F ratio?
- No, not directly-not in the equation
- Influences whether F ratio is statistically significant
-
For a significant effect:
- 1. large differences among group means
- 2. small differences within each group (not confound idea)
- small individual differences
- reliable tests or measures
- consistent application of each level of iv
-
Effect =
difference (or variability) in means
-
Possible to see an effect if the study includes:
a comparison of the influence of different levels of a predictor or independent variable
-
Factorial design- more than 1 IV:
- Each level of each IV occurs with every other level of all the independent variables
- Uses all possible combinations of levels of 2 or more independent variables
-
Main effect:
the effect (differences in means) of one iv, averaging,ignoring, or collapsing over the other iv's
-
Factorial Designs
Advantages:
- 1. Economy
- 2. Experimental control
- 3. Check generality of the effect of an IV
-
Factorial designs:
Adv.
1. Experimental control-
reduce error variance
-
Interaction:
the effect of one iv depends on the level of another iv
-
Factorial design:
Look for three things if there are two i.v.:
- main effect of iv A on dv
- main effect of iv B on dv
- Interaction between A and B
-
Describing the results of factorial designs:
- 1. Set up a table of means for each condition
- 2. Draw a figure
- 3. Describe the means for each main effect followed by a description of the results of the ANOVA
-
Statistical Hypothesis Testing:
- 1. Any t, F, U, orr could be due to chance/luck
- 2. If a result (a, t, F...) occurs by chance infrequently decide result is statistically significant
- 3. Typical reasoning
- 4. examples
-
3. Typical reasoning:
often decide that if an outcome is unusual/rare must be due to more than chance.
-
Null hypothesis:
presumed true unless statistical evidence suggests otherwise
-
The null hypothesis is tested because:
we have information about chance, not how the independent variable works.
-
Type I error:
Rejuect Ho but it is actually true
-
Type II error:
Faile to reject the Ho but Ho is false
-
What evidence must you collect in order to conclude that X is a cause of Y?
- 1. Show that changes in Y didnt occur until change in X (temporal precedence rule)
- 2. Show that X and Y are related (covariation rule)
- 3. Rule out other explanations forthe relationship between X and Y (internal validity rule)
-
Characteristics of a true experiment:
- 1. manipulation of X
- 2. Comparison of the effects of various levels of X on Y
- 3. Subjects begin the experiment equivalent on all levels
- 4. control over all other important variables so that all subjects are treated the same except for X (random assignment)
-
Program Evaluation Basic Strategy:
-To show the program caused a change in client
- a. Change in client occurred after the introduction of the program
- b. participating on not participating in program covaries with client success
- c. Rule out other explanations for the relationship between the program and client success
- d. plausible causal mechanisms linking program to client success
-
If there is an effect: F =
greater than 1
-
-
More diversity =
more error variance
-
Interrupted time series design
- often encountered in quasi experimental research
- Have a single experimental group for which we have multiple observations before and after naturally occuring treatment.
- Instead of observing 1 or 2 3rd grade classes we could observe 3rd grade classes over several years
- We need to know when the time series is interrupted by some treatment
- Then we compare observations beforea nd after the treatment to see whether it had any effect.
-
Threat to internal validity in case studies:
- Source of causation;
- baseline condition maturation
- history
- selection bias
-
Ways to enhance internal validity in case studies
- Deviant case analysis (a non equivalent control)
- detective work
-
Threat to internal validity for interrupted time series:
- changes in participants and environment
- delayed effects
-
Ways to enhance internal validity in interrupted time series:
- nonequivalent control group
- detective work
-
Threats to internal validity for subject variables
- dimensions on which to match
- regression artifacts
-
Ways to enhance internal validity for subject variables:
- matching
- include true independent variable
- see interactions
-
Threats to internal validity in Age as a variable:
- Confoundings with time of testing
- generation of birth
-
Ways to enhance internal validity in Age as a variable:
- Cross sequential design
- include a true i.v. & seek interaction
- Converging operations
-
______ _________ masks true behavior in quasi experiments and matching studies.
regression artifacts
-
All non parallel lines=
interaction
-
p =
- Significance level
- it does notindicate repeatability
-
one way anova- Error Variance (denominator) estimated by:
Calculating within groups variance (variance within each group pooled)
-
Krauter pseudo F ratio=
- range of group means
- (range of scores in gp 1+ gp 2+gp 3 etc..
- # of groups
-
2 reasons why scores in an experiment almost always vary:
- 1. Systematic variance
- 2. Error or unsystematic variance
-
Mann-Whitney U test
- a simple inferential statistic that can be used in place of a t test.
- It can be used in many instances in which you have tested twoindependent groups of subjects
-
How to do mann whitney u test:
- 1. put the scores in order from smallest to largest
- 2. For each score in the group w/ the lower mean, count the # of scores that are smaller in the other group. Thus you'll have 1 # for each score in the group with the lower mean that tells you how many scores are smaller in the other group than that score\
- 3. Addthese numbers together
- 4. For each score in the group with the lower mean, count the # of scores in the higher group that are tied or are the same. Add these ties together and divide by 2
- 5. Add theresults of C and D together to obtain the mann whitney u
-
Program Evaluation:
- Is a particular program actually delivering the services it is designed to deliver?
- Is a law having the desired effect?
-
Program Evaluation
Basic Strategy-
To show program caused change in client;
- a. Change in client occurred after the introduction of the program
- b. Participating or not participating in the program covaries with client success
- c. Rule out other explanations for the relationship between the program and client success
- d. Plausible causal mechanisms linking the program to client success
-
What kind of design is:
After school program and social skills...Every one gets the same treatment and all measured the same way
- One shot case study
- There is no comparison
- Descriptive research
-
If there is success in a one shot case study design, does it show the program caused the success?
- Does it answer...
- Change in the client occurred after intro to program
- Having or not having program covaries with client success
- Rules out other explanations for relationship between program and client success
- Extremely low internal validity
- Cant even show that the client changed during the program
-
Ongoing flow of events interrupted by the introduction of treatment-
interrupted time series
-
Pre or non experimental design
answers the questions...
- pretest post test design (before-after) o x o
- Potentially suffer from all the threats of within subjects design
- change occurred after intro to program
- Program/no program varies with client success
- rule out other explanations for the correlation b/w program/noprogram and client success
-
-
O X O
Pret test post test
-
X O
O
Static Group Comparison (ex post facto)
-
r O X O
r O - O
Pretest posttest control
-
Quasi Experimental Design
Might manipulate X, but dont compare groups that are formed based on random assignment
More opportunity to discredit alternative interpretations of data
-
Quasi experimental designs: To change non-experimental (pretest-posttest) to quasi exp. design-
- Add observations
- 1. Additional times before and after program introduced-time series studies
- OXO -> OOOOOOOXOOOOOO
- 2. Additional people who havent received the program (non equivalent control group - no random assignment - STATIC GROUP COMPARISON
- OXO -> OXO
- O-O (non equivalent control group
-
Time Series Design
Get many measures before and after some natural or planned intervention
-
Pretest-posttest design also becomes a quasi experimental design if:
add a comparison group
-
To improve a non experimental design:
Increase number of observations, add comparison group or both
-
Program evaluation-quasi experimental designs
Can distinguish between effect of program and many other variables:
- 1. comparison and "program" group have same amount of time to mature
- 2. history should influence both groups equally
- 3. testing should influence both groups equally
-
problem with program evaluation-quasi experimental designs:
- Finding a good comparison group...
- Cant randomly assign-between groups variable: ex post facto
-
Possible selection bias problem:
groups different because of the way they were selected or assigned to groupss
-
Program evaluation:
Quasi experiments summary
- 1. no random assignment
- 2. quasi-comparison using additional measures before and after or non equivalent control groups
- 3. moderate control
- 4. often field based
- 5. may never be able to eliminate confounding variables
|
|
