Case Overview:

The Children’s Hospital of Philadelphia inpatient pharmacy department piloted the use of generative AI simulations to enhance communication skills training for pharmacy trainees. Limitations of traditional communication training include trainer time constraints, unrealistic role playing, and lack of standardization. To address these challenges, we piloted AI role-play simulations that allow learners to practice discharge medication counseling with a virtual caregiver in a safe, repeatable environment. Trainees can participate through either text-based or voice-based interactions and receive structured feedback after each session.

Tool and Project Details:

Existing trainer-led simulation materials were modified and formatted as a large language model (LLM) prompt that could be inserted into the basic version of a chatbot. The prompt was composed of four main components: activity introduction, scenario background, interaction rules, and feedback criteria. Each scenario was customized with de-identified progress notes, discharge medication lists, caregiver description, and caregiver questions/concerns that were incorporated into prompts that guided the AI to act as a pediatric caregiver during discharge medication counseling simulations. Trainees copied the provided prompt into the LLM, conducted either text- or voice-based simulations, and submitted transcripts for preceptor review and feedback. No vendor customization or EHR integration was required.

Key Elements of Success:

A key success factor was the iterative design process used to refine the simulation prompts. Initial versions produced unrealistic interactions and overly positive feedback. Through repeated testing and trainee feedback, the trainer added structured conversation flow rules, communication objectives, assessment criteria, and requirements for constructive feedback. The initiative was led within pharmacy education and training activities using existing resources.

Impact on Outcomes:

Trainee feedback was overwhelmingly positive. Learners reported that the conversations felt realistic and provided opportunities to practice communication skills in a low-risk environment. AI simulations reduced scheduling barriers and trainer time requirements associated with traditional role-playing activities. We estimated 45 hours of trainer time saved per year if utilized with new trainees and recertification of current pharmacists. Although formal outcome metrics were not collected during the pilot phase, user satisfaction and adoption supported continued expansion of the program.

Role of the Pharmacy and Pharmacists:

No specialized AI certification was required; however, prompt design skills and communication training expertise were important for successful implementation. Pharmacist trainers reviewed learner transcripts and provided individualized coaching after simulation completion.

Budget & Resource Allocation:

This initiative began as a low-cost pilot using existing educational resources and publicly available generative AI tools. No additional software development, EHR integration, or hardware purchases were required. The primary investment was pharmacist time dedicated to prompt design, testing, and iterative improvement. Value was demonstrated through increased scalability of communication training, reduced dependence on trainer scheduling, and the ability to provide structured practice opportunities for multiple learners. These efficiencies supported continued use and expansion of the program to other departments with similar communication training needs.

Lessons Learned:

Prompt engineering proved critical to creating a meaningful educational experience. Early versions of the simulation produced excessive interruptions, unrealistic interactions, and feedback that was overly positive. These challenges were addressed by incorporating clear conversation rules, realistic patient scenarios, assessment rubrics, and requirements for actionable improvement feedback. The project reinforced that AI simulations require ongoing refinement and user feedback to remain effective. Another lesson was that AI should supplement—not replace—preceptor coaching and live communication training.

Future Goals & Ongoing Monitoring:

The goal is not to replace preceptors or live training but to increase the frequency, consistency, and accessibility of communication practice opportunities. Future plans include expanding AI simulation use beyond discharge counseling to additional communication scenarios, including secure chat messaging with providers and nurses, platform presentation practice, and high-stakes workplace conversations. This project and its findings have been shared through presentations with department leadership, hospital leadership, and as an ASHP educational webinar.