Daily Experience of Simulation Training (DEST)

Abstract

The Daily Experience of Simulation Training (DEST; Chiu et al., 2023) is an instrument designed to assess daily fluctuations in self-reported confidence among child welfare trainees, gather feedback on the training team, and provide trainees with an opportunity for self-reflection. This instrument plays a crucial role in evaluating the simulation training delivered by the Child Protection Training Academy (CPTA), a program initially developed at the University of Illinois at Springfield (UIS) and sponsored by the Illinois Department of Children and Family Services (DCFS) (Goulet et al., 2020). The DEST, comprising 20 items, was evaluated using a sample of new child protection investigators, with reported results for factor analysis and reliability. The instrument’s primary purpose is to evaluate each cohort participating in a week-long statewide simulation training program for new child protection investigators.

Keywords

Child Protection Trainees; Child Welfare; Confidence; Simulation Training Effectiveness

Authors

Chiu, Yu-Ling; Cross, Theodore P.; Wheeler, Amy B.; Evans, Susan M.; Goulet, Betsy P.

---

Purpose

The purpose of this instrument is to evaluate every cohort in a week-long statewide simulation training program for new child protection investigators.

Validity

No validity indicated.

Reliability

Internal Consistency: The Cronbach’s alpha reliability coefficients of the confidence scale at each of six time points were all larger than 0.95, indicating excellent internal consistency of the multi-item scale.

Factor Analysis

Principal Component Analysis: Principal component analyses with varimax rotation yielded one factor at each of six time points. The eigenvalues for the first factor ranged from 9.28 (a total of 71.4% of the variance) to 9.94 (a total of 76.5% of the variance).

Instrument: Daily Experience of Simulation Training (DEST)

Test Type

Original Inventory/Questionnaire

Format

On each administration, trainees rate their confidence for each of 13 items from 1 (low) to 7 (high). Trainees are also asked to provide feedback for each professional role involved in their training, based on their perceived quality of feedback on a 4-point Likert scale from very helpful to very unhelpful for both the simulation trainer and the classroom trainer. Open-ended questions are also included.
Administration Method: Electronic

Language Available

English

Population Group

Human; Male; Female

Age Group

Adulthood (18 yrs & older)

Population Details

Location: United States
Respondents: New Child Protection Investigators

Test Methodology

Test Reliability; Internal Consistency; Factor Analysis; Principal Component Analysis

Keywords

Child Welfare; Professional Competence; Professional Development; Program Evaluation; Protective Services; Training; Professional Personnel Measures

Authors

Author ORCID Identifier

No data is Available

Affiliation Email Addresses

• Chiu, Yu-Ling: [email protected] (University of Illinois at Urbana-Champaign, School of Social Work Children and Family Research Center)

• Cross, Theodore P.: No data is Available (University of Illinois at Urbana-Champaign, School of Social Work Children and Family Research Center)

• Wheeler, Amy B.: No data is Available (University of Illinois at Springfield)

• Evans, Susan M.: No data is Available (University of Illinois at Springfield)

• Goulet, Betsy P.: No data is Available (University of Illinois at Springfield)

Correspondence Address

Chiu, Yu-Ling: [email protected]

Permissions & Fee and Test Year

Permissions: Contact Publisher
Commercial: No
Fee: No
Test Year: 2023

References

Chiu, Y.-L., Cross, T. P., Wheeler, A. B., Evans, S. M., & Goulet, B. P. (2023). Development and application of a self-report measure for measuring change during simulation training in child protection. Journal of Public Child Welfare, 17(2), 239–257. https://doi.org/10.1080/15548732.2021.2016546

Items of the Daily Experience of Simulation Training (DEST)

This measure consists of 20 items.
Subscale: Confidence.

Part I: Survey Content

• At which site are you taking the training?

• With (1) being lowest and (7) being highest, please check the appropriate number to indicate your level of confidence in the following skill areas TODAY.

• Please answer the following questions regarding the feedback that you received in today’s training:

• Today’s reflective log: What were the most meaningful concepts or skills you learned today?

---

Part II: Implementation Guidelines for Simulation Trainers

1. BEFORE the simulation training:

• Please send the following survey link to trainees before the simulation training week: (link withhold).

2. Implementation timing on each day DURING the simulation training week:

• Day 1 morning: do the baseline survey as soon as the orientation starts. Give them time to do it BEFORE you proceed to do any simulation training preparation or activities.

• Day 2 through Day 5 afternoons: Do the survey in the afternoon after the debriefings but before making arrangements for the next day.

• Day 5: Do the survey BEFORE you tell them the true stories of the Rhodes and Jones cases.

3. Trainee Privacy:

• Please respect trainees’ privacy when they are taking the survey.

4. What to tell trainees:

• Please explain the purpose of doing the survey every day and why the survey is important to DCFS and to trainees. The log section can help them to reflect on what they’ve learned every day.

• The survey can be done on laptops, computers, smart phones, or tablets.

• The email address has to be all in lowercase. They have to use their DCFS email throughout the week. (Please contact the evaluation team ASAP if you have a trainee without a DCFS email. We will make a special arrangement for the week).

• Only the CFRC team has the access to their email information and won’t use it for any analysis.

• The survey is anonymous. They can quit anytime and they skip any questions.

5. Trouble shooting:

• Please ask trainees to click the link from your email every time when taking the survey. If they would like to save the link on their browser, they have to remember to reload the page in his/her web browser. Otherwise, they won’t be able to access the right survey. (How to reload a webpage: Ctrl + F5 for Windows, Command + Shift + R for Mac, or Command + R on Safari for Mac).

• There have been instances where upon clicking the ‘Submit’ button at the end of the survey, if their answers were not submitted or even saved, please tell trainees to reload the page in his/her web browser.

• Should this happen, please have trainees re-enter their responses and submit again.

• After successful confirmation and submission of their answers, trainees should arrive at a final “Thank You” page, which explicitly states that their responses have been saved in our database.

• On this final “Thank You” page, trainees should click the “Exit” button to clear any responses saved on their devices.

• Please report any technical issues with the survey immediately to the evaluation team.