Reichow, B., Volkmar, F.R., & Chiccetti, D.V. (2008).  Development of the evaluative method for evaluating and determining evidence-based practices in autism.  JADD, 38, 1311-1319. DOI on paper.

This information is applicable across the different papers that I might do.  Reference early and often.

• Has any intervention been determined to be an evidence-based practice?
• older methods of determining EBP were not adequate
• determined a new method for EBP
• EBP - first began in medicine and moved to the social sciences
  • most organizations - guidlines endorsing EBP
  • many different names for the same thing
• no universally accepted method for determining EBP
  • 2 independent randomized control studies usually meets the evidence
  • some use this as the only standard
  • differences - what is the strength of the evidence?
• NRC - committee - no practice is considered strongly evidence-based - backed up by other studeies
  • many different definitions
  •  some might not have been sensitive enough
  • limitations
    • operational method for evidence
    • narrow scope of what is evicense
    • single subject not treated
      • much of the research on children with disabilities is single subject!
• development of evaluative method for determining EBP in autism
  • rubrics of rigor
  • guidelines for reporting strength
  • criteria for determining EBP
  • standardized for clinicians researchers and practitioners
    • individual and group ratings
  • allows for a corrective so that different methodologies could be included (EXCEPT QUALITATIVE)
  • first guidelines for mixed methods!
• rubrics of rigor
  • once created for single subject and another for group designs
    • primary indicators - three ratings
    • secondary indicators - two ratings
  • TABLES ARE AVAILABLE IN THE STUDY ITSELF
  • similar definitions wherever possible
    • participant characteristics
    • dependent measures
    • independent variables
  • secondary
    • IOA
    • blind raters
    • procedural fidelity
    • generalization and maintenance
    • social validity
• measures of strength
  • strong
  • adequate
  • weak
  • TABLES ARE AVAILABLE IN THE PAPER ITSELF
• criateria for EBP
  • similar to those used in other social sciences
    • established - multiple sound studies, two independent groups
    • promising - multiple studies, weak rigor or other limitations
      • use these with caution and monitor closely
• reliability and validity of the rubrics
  • field tested (Reichow et al 2007)
  • 8 group designs and 10 single subject designs
  • 2 independent raters
    • reliability was good to perfect
    • Doehring et al 2007 replication
  • validity
    • concurrent validity - aligned with previous definitions
    • content - operationalization of what is EBP
    • common usage - face validity
• future studies
  • strengthen validity by training novice raters and compare to experienced raters
  • different types of raters
• Discussion
  • one of the first criteria for EBP to provide operationalization
    • group designs
    • single subject designs
  • should not base science solely on group design
  • difficulty - how many single subject studies constitute a group design?
    • 8 participants from experiments were equal to a group design
    • needs empirical validation - WONDER HOW YOU WOULD DO THAT?
      • averaged two criteria from two different sets of recommendations
  • translation of research to practice
    • eventually - have practitoners use this rubric to evaluate evidence
    • tried to keep the definitions from using too much jargon - might seem to simple
      • but the criteria are strong
      • might not be enough to close the gap
    • access to information is crucial
      • standardized method for accessing and evaluating research reports
    • guides for locating evidence
      • Locas & Cutspec 2005
      • Petticrew & Roberts 2006
      • McHugo & Drake 2003
        • most guides in the social sciences are not user-friendly
  • different research guides - help with at standardized way of reviewing the evidence
  • could put strength ratings in reviews - a kind of meta-analysis
    • others wouldn't have to re-review?
  • future work - does the review method generalize?
• Conclusion
  • we don't have EBP for children with autism yet
  • don't lower your standards!
  • Standardize the instrument instead
  • seems to be a valid assessment tool
  • could be applied to both researchers and practitioner use