Founded in 1902, the American Urological Association (AUA) is a premier urologic association, providing invaluable support to the urologic community. The AUA Annual Meeting is the largest gathering of urologists in the world, delivering unparalleled access to groundbreaking research, new guidelines, and the latest advances in urologic medicine.
Melissa Goodman, AUA Programs Manager, and her team member, Erin Nichols, wanted a better way to track learner retention at the Annual Meeting. They wanted to set up benchmarks that determine learner knowledge before, during, and after each session.
Why track learner knowledge, retention, and outcomes?
Before we get to the details, let’s first tackle why Goodman and Nichols wanted to track learner knowledge and retention.
- Needs Assessment
The first reason they want to track learner outcomes is to identify gaps in knowledge and retention.
By gathering data on learner knowledge before the conference, they can tailor content to those needs. Understanding gaps in knowledge during sessions allows speakers to react in real-time and fine tune their presentations to facilitate understanding. Following up with learners gives AUA the opportunity to understand which topics weren’t focused on heavily enough because retention was low in certain areas.
All of this together is a powerful combination. AUA is able to fine-tune the learner experience as they go along and deliver content to attendees that they want and need. It takes the guesswork out of conference programming and allows planners to strategically react to learner needs.
- Attendee Engagement
The second reason AUA tracks learner outcomes is to deliver a better educational experience to attendees. They want attendees to be more involved in sessions and participate in educational activities.
The value of something like this is more subjective than a needs assessment, but Goodman explains why it’s still important:
“Having speakers ask questions in large general sessions invites participation,” she says. “Moderators can filter questions and hide those that are irrelevant. The best questions help speakers create a discussion that is relevant and timely, so attendees receive value in every sessions and stay engaged with the conversation.”
The goal is to deliver a personalized experience to a large group of people.
Sourcing questions from speakers
Okay, now that we have a better understanding of AUA’s motivations, let’s explore their challenges and solutions. The first challenge was that Goodman and Nichols wanted to source questions directly from speakers. They wanted experts in their fields to be able to contribute directly to the live engagement, but didn’t want to email every speaker individually to source this information.
Enter CadmiumCD. CadmiumCD has a product called the Conference Harvester, which allows planners to build custom tasks. Speakers then log in and complete these tasks by completing forms and uploading materials. The tasks themselves are designed to be easy to complete in under 5 minutes.
“It’s an effective way to manage busy speakers and collect the content you need,” Goodman says.
AUA used this tool to collect questions for the audience response system before the conference. Speakers could then use a PowerPoint plug-in to trigger live polling during their sessions, which would bring up the questions they submitted pre-conference.
“It’s a great solution for clients,” says CadmiumCD’s Client Success Manager, Chiara Fortunato, “because the Harvester is completely integrated with the eventScribe App, which attendees use to respond to polls.”
Tracking learner retention and increasing engagement with education
AUA used a method called benchmarking to track learner knowledge, engagement, outcomes, and results. They gathered information about attendee knowledge before the session started, bookended the session with the same questions to track new knowledge gained, then followed up six months later to analyze retention. Here’s how…
Pre-Test
At the beginning of each participating session, speakers polled attendees with 5-10 questions and showed the results in real-time. They used a plugin for PowerPoint developed by CadmiumCD to launch the poll. Attendees had direct access to answer questions in the eventScribe App. The poll would begin on their smartphones and tablets as soon as the speaker triggered it.
“The good thing about the software is that it was a simple one-click to launch questions and calculate results,” says CadmiumCD’s Chiara Fortunato. “The results show up in bar graph format so attendees can easily understand their own knowledge as well as their peers’.”
Live Polling
Speakers then delivered their presentations, and shared research and knowledge, before running a second round of live polls. This one was to test knowledge again directly after they delivered relevant information. Attendees could see how much their knowledge improved and AUA had a solid record of learner activity and knowledge gained.
Using live polling also provides the opportunity to run case scenarios. Melissa Goodman describes these as “What would you do?” type questions.
“Essentially a speaker presents a scenario and asks the audience what they would do in that situation,” Goodman says. “This is the perfect opportunity for live learning. The speaker can see the results instantly and create a discussion around the answers, then share what he personally would have done and why some responses are stronger than others.”
Crowdsourced Q&A
Another benefit of the eventScribe Audience Responses System is that it comes equipped with a questions and answers tool. Attendees can pose questions via the app and other attendees in the session can upvote the best questions.
Goodman explains why this is important:
“In large general sessions, the crowdsourced Q&A tool gives everyone a chance to participate,” she says. “The vote up aspect is great because it lets you know what topics and questions are most relevant, and what’s on people’s minds. Moderators can easily filter these and hide irrelevant questions, and speakers could answer the best ones.”
Follow Up Survey
The final piece to successful benchmarking is to follow up after the event to determine retention. AUA uses a survey tool, similar to the Survey Magnet, to reach out after the conference. They use this to determine where attendees’ knowledge ends up in the long run.
“We check leaners in at the beginning of each session so that we have a good gauge on who attended,” says Goodman. “Six months later, we deliver a post-conference test to determine how well learning was retained.”
This helps AUA gauge how well their attendees are responding to and using information delivered in their programs. It shows where attendees are lacking retention, so they can continuously build educational programs that are relevant to attendees.
Conclusion
All in all, AUA has built a process that helps them deliver exceptional experiences and education to their attendees. CadmiumCD’s Conference Harvester and eventScribe platform are essential components of this process. Together, CadmiumCD and AUA have formed a partnership that maximizes success for all stakeholders.