# Research Methods

 The flashcards below were created by user faulkebr on FreezingBlue Flashcards. Analysis of variance (ANOVA): Estimate of systematic varianceEstimate of error varianceCalculate a ratio Simple, one way analysis of variance (ANOVA) F= systematic var. + error var. = Error variance F = Between groups var. Within groups var. Between groups variance= Within groups variance= Between= variance of group meansWithin= average of variance found within each group Simple analysis of variance (anova) *Scores almost always vary (differ) from subject to subject... 2 reasons: 1. Systematic variance2. Error or unsystematic variance 1. Systematic Variance Difference due to different levels of I.V. 2. Error or unsystematic variance Differences due to uncontrolled, random factors (not confounding) Error or unsystematic variance-uncontrolled variance: 1. individual differences2. random variations in testing3. measurement error4. experimental treatment not the same for each subject If IV influences DV: systematic variation of i.v. should cause systematic variation of the d.v. = systematic variance Differences in iv should cause: differences in group means If IV doesnt have an effect: group means are not likely to be exactly the same No systematic var. An effect means: that there is a real difference in means or that IV and DV are correlated If an effect, variations in the IV: would not reduce variance within groups variance If an effect: Assume that differences in group means should be: greater than differences within any single group If an effect: Difference within a group : just due to chance or uncontrolled factors If an effect: Differences betwen groups due to: controlled and uncontrolled factors Understanding basics of anova tell you how to design and run experiments: 1. large differences between groups2. small differences within groups Would size of sample influence F ratio? No, not directly-not in the equationInfluences whether F ratio is statistically significant For a significant effect: 1. large differences among group means2. small differences within each group (not confound idea)small individual differencesreliable tests or measuresconsistent application of each level of iv Effect = difference (or variability) in means Possible to see an effect if the study includes: a comparison of the influence of different levels of a predictor or independent variable Factorial design- more than 1 IV: Each level of each IV occurs with every other level of all the independent variablesUses all possible combinations of levels of 2 or more independent variables Main effect: the effect (differences in means) of one iv, averaging,ignoring, or collapsing over the other iv's Factorial Designs Advantages: 1. Economy2. Experimental control3. Check generality of the effect of an IV Factorial designs: Adv. 1. Experimental control- reduce error variance Interaction: the effect of one iv depends on the level of another iv Factorial design: Look for three things if there are two i.v.: main effect of iv A on dvmain effect of iv B on dvInteraction between A and B Describing the results of factorial designs: 1. Set up a table of means for each condition2. Draw a figure3. Describe the means for each main effect followed by a description of the results of the ANOVA Statistical Hypothesis Testing: 1. Any t, F, U, orr could be due to chance/luck2. If a result (a, t, F...) occurs by chance infrequently decide result is statistically significant3. Typical reasoning4. examples 3. Typical reasoning: often decide that if an outcome is unusual/rare must be due to more than chance. Null hypothesis: presumed true unless statistical evidence suggests otherwise The null hypothesis is tested because: we have information about chance, not how the independent variable works. Type I error: Rejuect Ho but it is actually true Type II error: Faile to reject the Ho but Ho is false What evidence must you collect in order to conclude that X is a cause of Y? 1. Show that changes in Y didnt occur until change in X (temporal precedence rule)2. Show that X and Y are related (covariation rule)3. Rule out other explanations forthe relationship between X and Y (internal validity rule) Characteristics of a true experiment: 1. manipulation of X2. Comparison of the effects of various levels of X on Y3. Subjects begin the experiment equivalent on all levels4. control over all other important variables so that all subjects are treated the same except for X (random assignment) Program Evaluation Basic Strategy: -To show the program caused a change in client a. Change in client occurred after the introduction of the programb. participating on not participating in program covaries with client successc. Rule out other explanations for the relationship between the program and client successd. plausible causal mechanisms linking program to client success If there is an effect: F = greater than 1 If no effect F= around 1 More diversity = more error variance Interrupted time series design often encountered in quasi experimental researchHave a single experimental group for which we have multiple observations before and after naturally occuring treatment. Instead of observing 1 or 2 3rd grade classes we could observe 3rd grade classes over several years We need to know when the time series is interrupted by some treatmentThen we compare observations beforea nd after the treatment to see whether it had any effect. Threat to internal validity in case studies: Source of causation;baseline condition maturationhistoryselection bias Ways to enhance internal validity in case studies Deviant case analysis (a non equivalent control)detective work Threat to internal validity for interrupted time series: changes in participants and environmentdelayed effects Ways to enhance internal validity in interrupted time series: nonequivalent control groupdetective work Threats to internal validity for subject variables dimensions on which to matchregression artifacts Ways to enhance internal validity for subject variables: matchinginclude true independent variablesee interactions Threats to internal validity in Age as a variable: Confoundings with time of testinggeneration of birth Ways to enhance internal validity in Age as a variable: Cross sequential designinclude a true i.v. & seek interactionConverging operations ______ _________ masks true behavior in quasi experiments and matching studies. regression artifacts All non parallel lines= interaction p = Significance levelit does notindicate repeatability one way anova- Error Variance (denominator) estimated by: Calculating within groups variance (variance within each group pooled) Krauter pseudo F ratio= range of group means (range of scores in gp 1+ gp 2+gp 3 etc.. # of groups 2 reasons why scores in an experiment almost always vary: 1. Systematic variance2. Error or unsystematic variance Mann-Whitney U test a simple inferential statistic that can be used in place of a t test.It can be used in many instances in which you have tested twoindependent groups of subjects How to do mann whitney u test: 1. put the scores in order from smallest to largest2. For each score in the group w/ the lower mean, count the # of scores that are smaller in the other group. Thus you'll have 1 # for each score in the group with the lower mean that tells you how many scores are smaller in the other group than that score\3. Addthese numbers together4. For each score in the group with the lower mean, count the # of scores in the higher group that are tied or are the same. Add these ties together and divide by 25. Add theresults of C and D together to obtain the mann whitney u Program Evaluation: Is a particular program actually delivering the services it is designed to deliver?Is a law having the desired effect? Program Evaluation Basic Strategy- To show program caused change in client; a. Change in client occurred after the introduction of the programb. Participating or not participating in the program covaries with client successc. Rule out other explanations for the relationship between the program and client successd. Plausible causal mechanisms linking the program to client success What kind of design is: After school program and social skills...Every one gets the same treatment and all measured the same way One shot case studyThere is no comparisonDescriptive research If there is success in a one shot case study design, does it show the program caused the success? Does it answer...Change in the client occurred after intro to programHaving or not having program covaries with client successRules out other explanations for relationship between program and client success Extremely low internal validityCant even show that the client changed during the program Ongoing flow of events interrupted by the introduction of treatment- interrupted time series Pre or non experimental design answers the questions... pretest post test design (before-after) o x oPotentially suffer from all the threats of within subjects designchange occurred after intro to programProgram/no program varies with client successrule out other explanations for the correlation b/w program/noprogram and client success X O one shot case study O X O Pret test post test X O O Static Group Comparison (ex post facto) r O X O r O - O Pretest posttest control Quasi Experimental Design Might manipulate X, but dont compare groups that are formed based on random assignment More opportunity to discredit alternative interpretations of data Quasi experimental designs: To change non-experimental (pretest-posttest) to quasi exp. design- Add observations1. Additional times before and after program introduced-time series studies OXO -> OOOOOOOXOOOOOO2. Additional people who havent received the program (non equivalent control group - no random assignment - STATIC GROUP COMPARISON OXO -> OXO O-O (non equivalent control group Time Series Design Get many measures before and after some natural or planned intervention Pretest-posttest design also becomes a quasi experimental design if: add a comparison group To improve a non experimental design: Increase number of observations, add comparison group or both Program evaluation-quasi experimental designs Can distinguish between effect of program and many other variables: 1. comparison and "program" group have same amount of time to mature2. history should influence both groups equally3. testing should influence both groups equally problem with program evaluation-quasi experimental designs: Finding a good comparison group...Cant randomly assign-between groups variable: ex post facto Possible selection bias problem: groups different because of the way they were selected or assigned to groupss Program evaluation: Quasi experiments summary 1. no random assignment2. quasi-comparison using additional measures before and after or non equivalent control groups3. moderate control4. often field based5. may never be able to eliminate confounding variables Authorfaulkebr ID117294 Card SetResearch Methods DescriptionFinal-Slides Updated2011-12-05T22:15:18Z Show Answers