Transcription

Auburn University – Vocational Evaluation Forensic Certificate ProgramImplementation & Outcomes Survey Year 1RSA Innovative GrantMarch 2021RSA Innovative Grant PurposeThe RSA Innovative grant project at Auburn University, known as the VocationalEvaluation Forensic Certificate (VEFC), aims to improve employment outcomes for individualswith disabilities by focusing on the employment of those working in vocational rehabilitation(VR). The main outcome of the VEFC is to reduce the number of VR vacancies by providingevidence-based practice (EBP), as well as greater forensic, vocational evaluation, andemployment expertise to professionals and paraprofessionals. By allowing students andprofessionals with a bachelor’s degree or higher into the program and providing financialfunding, those enrolled in the VEFC can participate in on-campus or distance education coursesrelated to forensic rehabilitation and vocational evaluation. Administered through theDepartment of Special Education, Rehabilitation, and Counseling (SERC) in the College ofEducation, this program requires one year of coursework that, upon completion, results in agraduate training certificate that will advance the field of vocational rehabilitation and improveemployment outcomes for professionals and ultimately consumers.Implementation & Outcome Survey PurposeThe Implementation and Outcome Survey is a method of gauging the efficiency andeffectiveness of the grant program in accomplishing its goals and objectives. As a component ofthe second year workplan, the I & O Survey was sent to students, faculty, staff, and otherstakeholders in September of 2021 to evaluate the goals from year one, which was a planning

year. This self-report survey allowed those involved with the VEFC to evaluate the program onits effectiveness and efficiency in meeting each of its objectives, in addition to providingfeedback about the program outcomes that have been accomplished thus far. The followingsections provide the survey results of the Year 1 (2021) I & O Survey.Results of the Implementation SurveyAll data were collected using Qualtrics, a web-based survey instrument, and analyzedusing SPSS statistical software. The survey invitation was initially sent during the first week ofSeptember 2021 and remained open for about 8 weeks, with two reminder invitations sentwithin that time period. A total of 25 completed surveys were obtained, resulting in about a56% response rate.Findings: DemographicsParticipants were asked to provide their gender, ethnicity, race, occupation, years ofexperience, and primary affiliation with this project. As individuals may have multiple roleswithin the project, respondents were permitted to provide more than one response to primaryaffiliation. Additionally, one participant did not report any demographic information, thus thedemographic data reflects only 24 of the total 25 respondents. As displayed in Chart 1: Gender,most of the respondents were female (N 22, 91.67%), with males representing 8.33% (N 2).

Chart 1: Gender8.33%91.67%MaleFemaleIn terms of race and ethnicity, respondents could choose among the following: American Indian or Alaska NativeAsianBlack or African AmericanHispanic or LatinxNative Hawaiian or Other Pacific IslanderWhite (neither Hispanic nor Latinx)Middle EasternTwo or more racesPrefer not to sayOtherAgain, one person refrained from answering, so only 24 respondents reported theirrace/ethnicity. Chart 2: Race and Ethnicity shown below indicates that of those whoreported their race and ethnicity 66.67% (N 16) identified as White (neither Hispanic norLatino/a), 16.67% (N 4) identified as Black or African American, 12.5% (N 3) identified as

Hispanic or Latinx, and 4.17% (N 1) identified as having Two or more racial/ethnicidentities.Chart 2: Race and Ethnicity4.17%12.50%16.67%66.67%WhiteBlack/African AmericanHispanic/LatinxTwo or More Races/EthnicitiesWhen asked about their affiliation to the program, only 23 respondents reported theiraffiliation while 2 did not respond, thus the following data reflect answers from the 23respondents. Chart 3: Affiliation shown below illustrates that most of the respondents (65.21%;N 15) were students and 17.39% (N 4) were faculty. Respondents also identified as eithersomeone with a disability, a VR Representative/Counselor, Member of the Advisory Board,Staff, or Other who made up the remainder of the sample. In viewing the information in Chart3: Affiliation, it is important to note that respondents were permitted to select more than onecategory of affiliation. For this reason, the sum total of responses in certain categories is higherdue to multiple responses. In all, six participants reported affiliations with more than onecategory.

Chart 3: tudentFacultyStaffAdvisory Board MemberOtherVR RepresentativeSomeone with a disabilityTo identify primary occupation, participants were able to choose from the following: Faculty Administration Rehabilitation Counselor Lawyer/Judge Forensic Rehabilitation State Vocational Rehabilitation Vocational Evaluator OtherEach of the participants responded to this demographic question. On this section, participantscould only choose one response. Chart 4: Primary Occupation shown below illustrates that themajority of respondents chose “Other” (28%; N 7). The second largest responses indicated

“Rehabilitation Counselor” (20%; N 5) and “State Vocational Rehabilitation” (20%; N 5) forprimary occupation. Thirdly, “Vocational Evaluator” (12%; N 3) was a primary response forthis section. Other responses included “Faculty” (8%; N 2), “Administration” (8%, N 2), and“Forensic Rehabilitation” (4%; N 1).Chart 4: Primary Occupation4%8%8%28%12%20%20%OtherRehabilitation CounselorState Vocational RehabilitationVocational EvaluatorFacultyAdministrationForensic RehabilitationParticipants were able to indicate the number of years of experience in their occupation. Thechoices available for this section included “Less than 5 years”, “5-10 years”, “10-15 years”, “1520 years”, and “More than 20 years”. Participants could only choose one response. Chart 5:Years of Experience displays the participant’s responses. Most of the participants indicated “510 years” of experience (36%; N 9) and “More than 20 years” (24%; N 6). Other responsesincluded “Less than 5 years” (16%; N 4), “10-15 years” (16%; N 4), and “15-20 years” (8%, N 2).

Chart 5: Years of Experience24%16%8%36%16%Less than 5 years5-10 years10-15 years15-20 yearsMore than 20 yearsFindings: Project AreasTo facilitate the evaluation process, project objectives and performance measures weredivided into the following 6 areas:1. High-tech Experience with Updated Curriculum2. Student Recruitment3. Student Access4. Advisory Board5. Implementation6. Program Related OutcomesParticipants were asked a series of questions related to each of the program areas and providedresponses using a 5-point Likert-type scale, including the following response choices: 1 – VeryInefficient; 2 – Inefficient; 3 – Neither Efficient nor Inefficient; 4 – Efficient; and 5 – VeryEfficient. It is important to note that program areas or career-related outcomes after

completion of the training certificate are not included given that this data is not yet available,and thus these program areas only represent the areas that are applicable to the coveredperiod of October 1, 2020 to September 30, 2021 (Year 1). This was a planning year for theprogram. Student outcomes will be provided in the next I&O Survey, which will cover year 2.All 25 respondents completed each question regarding the following program areas.1. High-tech Experience with Updated CurriculumThe High-tech Experience with Updated Curriculum program area included threequestions pertaining to the goals of refining curriculum content and coursework for theVEFC, creating a new online instructional platform, and creating an accessible websitespecific to the VEFC.Refinement of curriculum content and coursework: According to the survey, 72% ofrespondents (N 18) reported that the curriculum content and coursework decisionswere “very efficient,” while 24% (N 6) reported as “efficient,” and 4% (N 1) reported as“inefficient.” The mean score of this program goal was 4.64 (SD 0.7).Creation of new online instructional platform: Sixty percent (N 15) reported “veryefficient” for the creation of a new online instructional platform, while 36% (N 9)reported “efficient,” and 4% (N 1) reported “neither inefficient nor efficient.” The meanscore of the new instructional platform was 4.58 (SD 0.58).Creation of an accessible website: In regard to creating a program website for the VEFC,56% of respondents (N 14) reported the accessibility of the website as “very efficient,”in addition to 36% (N 9) who reported “efficient,” 4% (N 1) as “neither inefficient more

efficient,” and 4% (N 1) as “inefficient.” The mean score of the effectiveness of theaccessibility of the website was 4.44 (SD 0.77).2. Student RecruitmentThe Student Recruitment program area consisted of three questions pertaining toawarding RSA scholarships to program applicants, recruiting and giving priorityplacement to those with minority identities from traditionally underrepresented groups,and/or those with a disability, and lastly, assisting applicants with the admissionsprocess.Award 15 RSA scholarships to VEFC program applicants: According to the survey, 84% ofrespondents (N 21) reported “very efficient” for awarding the 15 scholarships while16% (N 4) reported “efficient.” The mean score was 4.84 (SD 0.37).Recruit and give priority placement to individuals with minority identities fromtraditionally underrepresented groups, including individuals with disabilities: Sixtypercent (60%; N 15) reported this area of recruitment as “very efficient” while 20%(N 5) reported “efficient,” 16% (N 4) reported “neither inefficient nor efficient,” and4% (N 1) reported “inefficient.” The mean for minority recruitment was 4.38 (SD 0.91).Assist applicants with the admissions process: Sixty percent (60%; N 15) reported thegoal of assisting applicants with the admissions process as “very efficient,” while 36%(N 9) reported “efficient” and 4% (N 1) reported “inefficient.” The mean for assistancewith the admission process was 4.56 (SD 0.58).

3. Student AccessThe Student Access program area contains four questions that pertain to providing twoeducational formats (on campus and distance education), enrolling students whocurrently work in State VR agencies, providing the VEFC website and other onlinecampus resources to enrolled students by the fall semester, and providing the fallcourses (Introduction to Rehabilitation and Case Management and ProprietaryRehabilitation) during the first the fall semester of the program.Provide two educational formats: The goal of providing the on-campus and distanceeducation formats was rated as “very efficient” by 72% (N 18) of respondents. Twentyfour percent (24%; N 6) reported it as “efficient” and 4% (N 1) reported it as“inefficient.” The mean score for providing two educational formats was 4.60 (SD 0.87).Enroll students currently working in State VR agencies: According to the survey, 60%(N 15) reported enrolling students who work in State VR as “very efficient,” while 32%(N 8) reported it as “efficient,” 4% (N 1) as “neither inefficient nor efficient,” and 4%(N 1) as “inefficient.” The mean score was 4.48 (SD 0.77).Provide the VEFC website and online resources by the fall semester: Sixty-four percent(64%; N 16) reported providing the website and other online resources by the fallsemester as “very efficient.” Another 28% (N 7) reported it as “efficient” and 8% (N 2)reported it as “neither inefficient nor efficient.” The mean score for providing theseresources by the fall semester was 4.56 (SD 0.65).Provide fall courses by the fall semester: Eighty-four percent (84%; N 21) reported thisprogram goal as “very efficient,” while 12% (N 2) reported it as “efficient” and 4% (N 1)

reported is as “neither efficient nor inefficient.” The mean score for the effectiveness ofproviding the fall courses by the fall semester was 4.80 (SD 0.5).4. Advisory BoardThe program area related to the advisory board consisted of only one question at thistime, which asked about the effectiveness of the VEFC in recruiting Advisory Boardmembers from Region IV and various other states.Recruit Advisory Board Members from Region IV and other states: According to thesurvey, 48% (N 12) reported this program area as “very efficient.” Another 36% (N 9)reported it as “efficient,” while another 16% (N 4) reported it as “neither inefficient norefficient.” The mean score for the effectiveness of recruiting for the advisory board was4.32 (SD 0.75).5. ImplementationAt this time, only one question related to the program area of implementation wasincluded in this survey, which asked about completing the Implementation & OutcomesSurvey (the current survey) on time.Conduct a program I & O survey for year 1: The result indicated that 72% of respondents(N 18) gave the rating of “very efficient” while 24% (N 6) gave the rating of “efficient”and 4% (N 1) gave the rating of “inefficient.” The mean score for this program goal was4.64 (SD 0.70).6. Program Related OutcomesRegarding program related outcomes, three questions were included in this survey. Thequestions were related to completing the quarterly reports and submitting them to the

RSA on time, submitting presentation proposals to professional conferences, and usingcutting-edge technology and software.Complete all quarterly reports to the RSA on time: This goal was reported to be “veryefficient” by 68% of respondents (N 17), “efficient” by 24% (N 6), and “neitherinefficient nor efficient” by 8% (N 2). The mean score of completing and submitting thequarterly reports on time was 4.60 (SD 0.65).Submit presentation proposals to professional conferences: Seventy-two percent (72%;N 18) reported this goal as being “very efficient,” while 16% (N 4) reported “efficient,”8% (N 2) reported “neither inefficient nor efficient,” and 4% (N 1) reported“inefficient.” The mean score of submitting presentation proposals was 4.56 (SD 0.82).Use cutting-edge technology and software: The results indicated that 68% ofrespondents (N 17) reported it as “very efficient.” Twenty-four percent (24%; N 6)reported it as “efficient,” and 8% (N 2) reported it as “neither inefficient nor efficient.”The mean score for the program area and goal of using updated technology was 4.60(SD 0.65).Findings: Qualitative ResponsesSurvey participants were given the opportunity to add additional thoughts andobservations. The prompt was “Please provide any additional comments or feedback regardingthe efficiency of the aforementioned areas or other suggestions for improvement.”Below is a list of responses from various participants.

“I was chosen for the scholarship (white), but my colleague (black) was not chosen.” “The program is very heavy in Alabama law. I work in [another state] so I am not surehow applicable some of the info regarding [Worker’s Compensation] is for my career.” “The ability to reach more students utilizing the distance education platform isamazing!”Summary and RecommendationsThe purpose of the present survey was to assess the efficiency and effectiveness of theVocational Evaluation Forensic Certificate’s seven current program areas. By listing the programgoals that pertained to year 1 of the program (Oct 1, 2020 to Sept 30, 2021), students, faculty,Adviosry Board Members, and other stakeholders were asked to rate the efficiency in meetingeach of the goals within the given time frame. A total of 25 stakeholders completed the surveywhich resulted in about a 56% response rate. Additionally, there was an opportunity forrespondents to provide feedback, thoughts, and other observations at the end of the survey.The participants reported to be mostly female (91.67%, N 22)(male 8.33%, N 2). In termsof race or ethnicity, 66.67% (N 16) of respondents identified as White (neither Hispanic norLatino/a), 16.67% (N 4) identified as Black or African American, 12.5% (N 3) identified asHispanic or Latinx, and 4.17% (N 1) identified as having two or more racial/ethnic identities.Lastly, most of the respondents (65.21%; N 15) were students and 17.39% (N 4) werefaculty, while 6 participants selected affiliations in more than one category, (e.g., someone witha disability, currently working in State VR, staff, advisory board member, or other).

Overall, the year 1 I & O survey results ranged from “inefficient” to “very efficient,” with themajority of responses at the “very efficient” rating. There seemed to be a consistent singleindividual who answered “inefficient” for many of the questions, which may have skewed theresults due to response bias or clerical error, or common to survey research, accidental reversalof the scale. Altogether, most of those involved with the VEFC reported the program areas andgoals to be efficient, as evidenced by mean scores of 4.32 or above (based on a 5-point scale)on all items in each of the seven program areas.Although the results suggest positive impact and efficiency, we believe that there are stillmany changes that can be made to improve the VEFC and its efficiency and effectiveness inmeeting the grant’s purpose and objectives. Based on these results and the general knowledgeof the program in its current state, we propose the following recommendations:1. Student Recruitment and Enrollment:a. Enrollment: We are actively looking into areas of expansion for the VEFC within thedepartment and as such hope to continue with cohorts of 15 students each year.We believe that the flexibility of the two educational formats is an attractivecharacteristic of the program, as well as the 15 RSA scholarships that are awardedeach year. A goal for years 2 and 3 would be to recruit all 15 scholars as well as anumber of other students who enroll in the training certificate.b. Recruiting minority students: While we have a fairly diverse first cohort in both race,ethnicity, age, and ability status, we aim to increase the number of individuals with aminority identity enrolled in the program and awarded a scholarship. We hope to

expand on relationships with local HBCU’s in the state of Alabama, as well as recruitfrom other organizations and academic institutions outside of the local area. To dothis, we plan to coordinate informational zoom meetings with related universityprograms at the undergraduate and graduate levels so that they will have theopportunity to hear about the VEFC and ask questions. The goal is to increase theexposure of the program to miniorty students and begin building relationships withtheir educational programs.2. Continue Recruitment of the Advisory Boarda. While we have a number of Members that have agreed to serve on the advisoryboard, we are still actively recruiting a few more State VR counselors within RegionIV, which will likely influenced the rating of this program goal having the lowestmean score of 4.32 (SD 0.75). We believe that the difficulty in finding Members isin part due to the general sense of burnout among helping professionals within theCOVID-19 era, as well as the program being new and still making a name for itself.Our goal of finding consultants to complete the Advisory Board will be revised andmethods of recruitment will be improved. To date there is only one state in RegionIV that does not have a State VR Representative, which is Mississippi. We willcontinue to explore ways to secure an Advisory Board Member from this state.

Rehabilitation Counselor Lawyer/Judge Forensic Rehabilitation State Vocational Rehabilitation Vocational Evaluator Other Each of the participants responded to this demographic question. On this section, participants could only choose one response. Chart 4: Primary Occupation shown below illustrates that the