PROJECT SUMMARY

After demonstrating efficacy at improving interview skills, self-confidence, and access to employment across 5 randomized controlled trials, the research team sought and was awarded funding from the National Institute of Mental Health (R01 MH110524) to evaluate whether participants randomized to use VR-JIT within an IPS model would demonstrate better employment outcomes within 6 months compared to participants randomized to receive IPS-as-usual. The study is led by Dr. Matthew Smith, Associate Professor at the University of Michigan’s School of Social Work in partnership with Northwestern University and Thresholds, Inc.  Click here to download a pdf of the results.

METHODS

The study is a 2-arm randomized control trial in which clients 18 years or older with a history of serious mental illness and enrolled in IPS services will be randomized to receive IPS-as-usual or in combination with VR-JIT. Participants complete study measures, among them mock job interviews and self-report measures on interview anxiety and self-confidence prior to and after the intervention period. Participants also complete a 6-month follow-up visit and receive monthly follow-up phone calls asking about job interview completions and employment status.

2. Improved job interview skills for participants using VR-JIT. 

• IPS+VR-JIT trainees (n=14) had an 8.0% (Cohen’s D Effect size: 0.51) increase in interview skills compared to IPS only trainees (n=11) who showed a 1.1% score increase (Cohen’s D Effect: 0.04).

4. Increased interviewing self-confidence for participants using VR-JIT.

• IPS+VR-JIT trainees had an 6.9% (Cohen’s D Effect size: 0.30) increase in interviewing self-confidence compared to IPS only trainees who showed a 3.3% decrease (Cohen’s D Effect: -0.13).

5. IPS employees may be able to use their time more efficiently when clients are using VR-JIT.

• In order to analyze VR-JIT’s effect on IPS employee effort, Threshold Employment Specialists (ESs) were asked to complete bi-weekly “cost capture surveys.” These surveys asked ESs to estimate the number of hours they spent on various categories of tasks. An average of 34 weeks of data were collected from 13 ESs. The ESs were split into two groups: those who had study participants on their caseload assigned to the VR-JIT intervention and those without any participants on their caseload assigned to the VR-JIT intervention.

Trends were noticed across 4 categories (See Figure 5):

• Job Interview Training –Staff members with participants on their caseload assigned to VR-JIT spent an average of 6 hours less per week per staff member on interview preparation and mock interviewing than individual staff who did not have participants on their caseload using VR-JIT.
• Job Application Skills– Individual staff with members on their caseload using VR-JIT spent an average of 45 hours less per week working with clients on job application skills than those staff who did not have participants on their caseloads using VR-JIT.
• Administrative Tasks/Documentation – staff with members on their caseload using VR-JIT spent an average of 8 hours more per week on administration tasks and documentation, suggesting that clients using VR-JIT may free up more time in staff’s day to work on other administrative duties.
• Job Developing/Supports – Staff with and without clients on their caseload using VR-JIT spent similar hours per week on average helping clients with other skills to support their employment.

6. Pre-implementation Costs. Data was collected on the time and cost of preparing to implement the intervention, which included creating a computer lab and training employment staff (6 Employment Specialists and 4 Team Leaders) to run it. The process took a total of 261 staff hours from the Thresholds team (15 total research and employment staff) and 305 staff hours from the Northwestern/University of Michigan research team (1 Principal Investigator, 2 staff) which amounted to a total labor cost of $22,882. The non-labor costs to create a computer lab with 5 work stations amounted to $2599.

CONCLUSIONS

Preliminary findings are promising and suggest that using VR-JIT as a supplement to IPS services may lead to higher rates of competitive employment, increased interviewing skills, decreased anxiety and increased self-confidence for clients around interviewing. Integrating VR-JIT into IPS services may also lead to a more efficient use of staff time, including less time performing job interview training and application skills and more time completing administrative tasks. The overall cost to plan for and implement the intervention is reasonable.

Funding