Evidence-Based PracticeChecklist: Dotting your i’s andcrossing your t’s to successSusan M. Wilczynski, Ph.D., BCBA-DBall State University
Why use the EBP decisionmaking model?O We are likely to produce the best outcomeswhen we following the EBP decision-makingmodelO We are likely to meet our ethical obligationswhen we follow this decision-making process
Reminder of the EBP decisionmaking modelSpencer, Detrich, & Slocum,2012
Why use the EBP checklist?O Prompts theevidence-basedpractitioner to thinkabout ALL relevantevidenceO It is hard toremember when andhow to incorporateall aspects of EBP
BookO Primer for how toconduct EBP.O Target population:Autism SpectrumDisorderO Checklists inAppendices
Where should I begin?O Ask the practicalquestion that isrelevant in thecurrent case“How can we increaseSally’s social initiationstowards peers?”
Steps of Evidence-BasedPractice1. Identify and reviewbest source(s) ofevidence4. Identify short- andlong-term treatmentgoals2. Review relevanttarget client variablesthat could impacttreatment selection5. ImplementIntervention3. Review relevantstakeholder andleader client variablesthat could impacttreatment selection6. Review newevidence
Step OneBegin by identifying the best source(s) of evidence – thatanswers your practical question!Systematic Reviews & Meta AnalysisNarrative Review (consensus or critical)Practice GuidelinesPrinciplesClient HistoryConfounding ExplanationsTreatment FidelityDifferent Environmental ConditionsCurrent Client DataFunctional Behavior AssessmentOther Relevant Data
Source of Evidence: SystematicReviews and Meta-Analysis“Is there evidence from acredible systematic review ormeta-analysis that answers thepractical question?”
Pros and Cons of SystematicReviews & Meta-analyses-Careful analysis of quality,quantity, and consistency ofresearch findings-Best source of informationabout treatmenteffectiveness-Produce the most credibleand comprehensive analysis-Least biased-Not always available or maynot relate to relevant target,stakeholder, or clientvariables-Relevant systematic reviewmay have not beenconducted-Systematic review may notbe relevant to the practicalquestion
Client HistoryO If a treatment hasworked for a client inthe past, it should begiven a high prioritywhen selected froman array of treatmentoptions if
“Is there credible evidencefrom the client’s history thatshows one of the remainingtreatments will not work?”
Confounding explanationsA treatment may look like it hasworked before but it is really due tosomething else.A treatment might not have “worked”before because it was notimplemented accurately
Ask if different conditionsexist now
Current Client Data“Are there current datathat have been (orshould be) collected tohelp identify the mosteffective treatment?”
Using Current Client DataFunctional Behavior Assessment Generate a strong hypothesis aboutthe function of the behavior
Using Current Client DataOther Relevant DataExample: RTIAny evidence that helps identify the“best” treatment option
Question current client dataDo the data capture allrelevant information?Are the outcomes biasedin any way?
Now that I’m skeptical
Use all of these sources ofinformationO Systematic reviewsO Client HistoryO Current Client Data ProfessionalJudgment isrequired toweigh theevidence
After identifying the bestsources of evidenceO Create a list of “best”treatments based onthe answers to thosequestionsO A list of alternativeinterventions (i.e.,lower priority from youranswers and thosethat have emerging butweaker evidence)
Step twoO Review relevanttarget clientvariables thatcould impacttreatmentselection
HealthMedicationMedical andComorbid ConditionsBiological VariablesMental Health
“Are there healthreasons to strengthenor weaken thelikelihood a treatmentshould be selected?”
Consider MedicationsO Ask aboutmedication usageboth at the onset oftreatment andthroughout servicedelivery
ExampleThink about a client you have had that hascomplained of any of the following:O HeadachesO Gastrointestinal discomfortO Being tiredDid you become familiar with medicationstaken by client and consider possible sideeffects of medication?
ExampleAre there some treatments that must be ruledout because of certain medical conditions?O Client with a history of a heart conditionshould not be held in restraints(Ishida, Katagriri, Uchida, Takeuchi, Sakurai, Watanabe, & Mimura, 2014).
Example15-30% of individuals with ASD have epilepsyBecause epilepsy can be associated withseizure-induced aggression, are we taking intoaccount comorbid medical conditions that maybe causing certain behaviors?(Iterson, Jong, & Zijlstra, 2015).
Biological VariablesO Can have a directStaminaSleepHungerand indirect impacton behavior byaltering the rate ofbehavior and thevalue of reinforcers
ExampleYour client just had a HappyMeal from McDonalds. Now,you are starting a therapy andwhat is their reinforcer?.Fruit Snacks!!How motivated is your clientto work for fruit snacks whenthey just ate lunch?
ExampleO Many individuals with ASD hold co-occurring(comorbid) diagnoses. These comorbid conditionsshould impact treatment decisions.O Use comorbid diagnosis to help interpret behavior.O Abuse associated with anxiety and depression.Almost 2/3rds of girls with disabilities experiencesome form of abuse by the time they are 18 yearsold.
ExampleO Client has a history of being bullied.O Begin process of selecting best treatmentO Peer-mediated intervention is likely notappropriate for a client - certainly not withoutvery significant supervision – and possibly notat all.
HealthRepertoirePreferenceSocialValidityWhat other targetclient variables shouldwe consider?
RepertoirePrerequisite SkillsBehavioral Cusps
Definitions & ExamplesWhat are Pre-requisite Skills?O Skills that are alreadymasteredO Pre-requisite skills must bemastered in order for atreatment to be effectiveVideo ModelingPre-requisites:O Remain still for periodof timeO Attend to all pertinentaspects of the modelO Imitate stepsO Generalize
PreferencePreference AssessmentChoicePreference as aNatural By-Product ofTreatment
ChoicePreferenceas a NaturalBy -Productof nt aHIGHERpriority
Social ValidityQuality of LifeGeneralizationTreatment Acceptability
Social ValidityGive a higher priority to treatments that:Improve quality of lifeIncrease likelihood ofgeneralizationAcceptable to target client
Don’t Forget!!O Review list to ensure allrelevant target clientvariables have influencedpriority of lists
Step ThreeO Review relevantstakeholder andleader clientvariables that couldimpact treatmentselection
What to ConsiderOIt is important to review relevant stakeholderand leader client variables that could impacttreatment selection.StakeholderclientLeaderclient
“Will implementationof any of thetreatments improvethe family quality oflife?”
Stakeholder Client VariablesFirst, review relevant stakeholder client variablesFamily Quality of LifeFeasibilitySustainability
ResourceConstraints“Is the cost identifiedby stakeholder orleader clients viable?”
EnvironmentalSupports“Can theintervention beimplemented withsufficientenvironmentalsupports?”
What are environmentalsupports?O Variables that enhanceor undermine thecapacity to implementan intervention in realworld settings, with theexception of costs
Environmental SupportsO Family EngagementO Match with CulturalNormsO Stakeholder AttitudesO Appropriate Trainingfor Stakeholder ClientsO Flexibility of TrainingO Characteristics ofStaff ProvidingTreatmentO Realistic Appraisal ofBarriersO Experience
SustainabilityNo.Not that kind!!!More like this
Leader Client VariablesNext, review relevant leader client variablesthat could impact treatment selection.O FeasibilityO Sustainability
Consider the following What are the previousexperiences of staff?Will I needtoredistributeresourcesneed forotherclients?Are theresupportpersonnelwho canassist?Is there sufficientnumber of staff?
“Can the intervention beimplemented with treatmentfidelity?”(with or without additional training)
Leader stakeholder clientsshould considerDoes treatmentrequire a lowerresponse effort forstakeholder/ leaderclients?Does treatment havea positive impact onthe organization?Does treatmentmatch culturalnorms of theorganization?Is treatmentassociated withpositive “attitudes?”Can treatment beadapted withoutsacrificing targetclient progress?
What to consider for leaderstakeholder clientsO Expected ValueO Secure New ResourcesO Staff ExperienceO New CollaborativeO Treatment ComplexityO Organizational ImpactPartnershipsO Leader ResponseEffortO Staffing Requirements
Step FourO Identify short- and long-term treatment goals
“Is there a treatment,or combination oftreatments that cananswer the practicalquestion that startedthis process?”
“If more than one treatment is selected,should they be introduced sequentially orsimultaneously?”“Has the issue of treatmentfidelity/feasibility and complexinterventions been addressed withstakeholder and leader clients?”
Treatment GoalsShort-termO What treatmentthat could beimplementedimmediately couldmake things better(even if notperfect)?Long-TermO What treatment dowe need in thelong run to producethe improvementsthat arenecessary?
Implementation PlanChecklistO Describe the treatmentO Each step implementation plan checklistshould contain:O Date it was proposedO Key people responsible for itO Coping planO What materials/resources are needed?O What are the training needs?O What kinds of data will be collected?
Then O Establish which single subject researchdesign to use to evaluate your outcomes.
Implement and ReviewTreatment!
Step fiveImplementing the TreatmentO Identify unanticipatedresource constraintsO Identify unanticipatedlimitations toenvironmental supports
Analyze TreatmentEffectivenessO Review target client progressO Review target client preference/toleranceO Review stakeholder/leader satisfactionO Schedule times for review
Step 6Determining next stepsO Data are criticalO But data are uselesswithout analysisO Analysis is morecomplex that just“improvement” or“no improvement”
Should the treatment beretained?O YesO The treatment isworking!O The target client ishappy.O The treatment canbe sustained untilimprovement hasreached criterion.O NoO It isn’t workingO The client ismiserableO The stakeholderclients can’t sustaintreatmentO The environmentcan’t support theintervention anylonger
If you retain the treatment O Make a plan forfading the treatmentO Treatments shouldnot be required“forever” but mayneed to be sustainedfor a long period ionInterventionIntervention
Does the treatment need to beadapted?O YesO It’s working butthere are problemswith the client,stakeholder clients,or contextO NoO If adaptation wouldmean theprocedure no longerresembled thetreatment that hasevidenceO If adapting meansprogress is lost
If you adapt the treatment O Pick the smallestchange needed toproduce favorableoutcome.O Make sure it doesnot violate principleson which treatmentis basedO Quickly re-evaluateoutcomes
Does the treatment need to berejected?O Return to step 1
Does the treatment need to berejected?O Return to step 1 – you already have your list.Re-examine it with your new evidence in hand “Best”O Best 1O Best 2O Best 3O Best 4O Best 5 AlternativeO Alternative 1O Alternative 2O Alternative 3O Alternative 4O Alternative 5
Does the treatment need to berejected?O Return to step 1Spencer, Detrich, &Slocum, 2012
Questions?Contact me at:Susan M [email protected]
Susan M. Wilczynski, Ph.D., BCBA -D Ball State University . Why use the EBP decision-making model? O We are likely to produce the best outcomes when we following the EBP decision-making model O We are likely to meet our ethical obligations when we follow this