Written by CadmiumCD Project Manager, George Heitzmann
This is Part II of the “Using Survey Magnet for Virtual Events” series. Read Part I here.
In this post, we will focus on using the survey magnet for on-demand content or enduring materials.
An internet enduring material is defined by ACCME as an online enduring activity that can be accessed whenever the learner chooses to complete it. The content can be accessed at any point during the lifespan of the activity and there is no specific time designated for participation. Examples include online interactive educational modules, recorded presentations, podcasts.
With enduring materials, there is a different set of challenges when it comes to setup from live events. Items to be considered include – how will the content be hosted? Is the user required to participate in the activity to be eligible for credit? How is participation tracked and verified? How should all this information be reported back to your accrediting body or provider.
This may all seem complicated, this post will attempt to summarize the process in 5 easy steps:
- Delivery of Enduring Material
- Delivery of materials and documents
- Delivery of audio and video
- Activity Record and Authentication
- Pre and Post-test Quizzes and Assessments
- Collection of post tests
- Post test best practices
- Certificates & Transcript
- Reports & PARS Integration
1. Delivery of Enduring Material
Delivery of materials and documents
In the event that the enduring material consists of documents and handouts – you can load these materials to the Survey Magnet portal and attach those materials to surveys and quizzes. Learners can then download the materials to review prior to completing evaluations and quizzes on the materials.
Delivery of audio and video
While the survey magnet itself is not a portal to learner users to view AV materials, you can easily link to content hosted elsewhere from the survey magnet.
You can link to content directly from the survey magnet download task or from the session or general evaluation. Note – this will be a link to content hosted elsewhere, the user will be taken away from the survey magnet to view the content.
Expert Tip – To improve the user experience, we suggest using the eventScribe website and app to host enduring materials. This includes handouts, slides and synced slides and audio.
2. Login and Authentication
Once the learner has watched the enduring material content, the learner can then log in to the Survey Magnet to complete evaluations or quizzes for credit.
But how can you verify the user actually watched the content? This requires the system to track if the learner actually reviewed the content which can get really complex!
A possible solution is to host this content on eventScribe website or app. EventScribe can be upgraded to include “play for credit” mode which forces the user to watch the video in its entirety or a percentage threshold that’s selected by you. Once complete, the user activity record is updated with the presentation information which allows the user to complete evaluations and quizzes for the presentation to receive credit.
If this content is hosted elsewhere (like a separate LMS system), the activity record can also be updated via an integration between the content provider and Survey Magnet. Note, this is contingent on the ability of the hosting provider to send the Survey Magnet this information via an integration. Please discuss with a CadmiumCD sales person before proceeding.
3. Pre and Post-test Quizzes and Assessments
Collect and Administer Tests and Quizzes
The Survey Magnet provides a simple, intuitive interface to create graded tests and quizzes for your learners to complete. The survey magnet supports test specific passing scores, ability for participants to take tests multiple times to achieve passing score and disseminate timely post-test feedback, including a brief explanation and/or reference(s) for the correct answer.
The survey magnet also allows you to create a quiz for each presentation in the schedule or a subset of the schedule that is eligible. You can use the harvester to collect these quizzes and import them directly to the survey magnet using the quiz questions import tool.
Below are some important considerations for presenters when submitting quiz questions based on ACCME SAMs requirements
Guidelines for developing quiz/test questions:
- SAMs questions should test important concepts that are medically (clinically) relevant. Questions should link to the learning objectives.
- The ABPath recommends all questions be multiple choice, single best answer with 3-5 choices. True/False or Yes/No questions are not acceptable. If an item truly has only two choices, and it is not a T/F or Y/N question, such an item is acceptable.
- An ideal question is one that can be answered without looking at the choices. Higher order questions that require interpretation, judgment, or problem-solving are better than simple recall of information.
- Questions should be stated as a positive (do not use no, not, etc). Do not use “all of the following except”.
- Do not use absolutes such as “all”, “none”, “always” and “never”.
- “All of the above” or “none of the above” are not acceptable choices.
- No possible question answers should include other possible answers, e.g. “C: Both A & B” (No K Type Questions)
- Answer choices should be in alphabetical or numerical order and approximately the same length.
- Responses must be logical and homogenous (e.g. all IHC stains, all laboratory test results, all clinical associations).
- References should be provided when appropriate.
- A brief narrative/explanation of the correct answer must be provided.
Is a pre-test necessary?
A pre-test is recommended to identify or demonstrate participant gaps in knowledge. The typical flow is users would login to complete the pre-test prior to the conference and note their scores (note – while these quizzes are graded, they typically used as a benchmark to compare post-test results against).
After watching the enduring material, the learner can then complete the post-test which could include the same questions or some additional questions. You can then compare the post test scores with the pretest scores to gauge learner improvement.
4. Transcripts & Certificates
Certificates and transcripts can be printed by the learner as soon as all the requirements have been met. The Survey Magnet portal supports multiple certificates for enduring materials including having separate certificates for enduring materials and live content, for specific content the user purchased and also separated by accreditor and learner speciality (nurse, physician, dentist etc).
5. Reports & PARS integration
The Survey Magnet has a selection of stock evaluation, credit and quiz reports depending on requirements. Reports include summary tabulated formats as well as detailed answer set information showing all responses each individual provided.
These reports also come in a variety of export options including word, PDF and excel export formats. To see a detailed list of survey magnet reports, click here! Custom reports are also available at a fee.
5.a. integrate with ACCME’s online database (PARS) to send credits for the attendees
The Survey Magnet supports PARS integration to send learner data directly to the PARS system from the survey magnet. This includes an additional information task to allow the user to review and update their membership information (including Membership ID, ABIM/ABP MOC Number and Date of Birth) prior to submitting to PARS.
ACCME (PARS) Reporting Tips for Enduring Materials
- The reporting requirements for Internet enduring material activities for ACCME is a little different than traditional live activities. Activities should be reported in PARS for each year in which they are active, either for the entire year or any part thereof. For each year the activity is provided, the number of learners who participated in it during that year, as well as the income related to the activity for that year should be reported.
- Do not report cumulative data for an activity spanning multiple years. The start date entered for the activity should be the first date the activity was available for learners or the date of the most recent review of the activity, whichever is later. If the internet enduring material activity spans multiple years, the start date should be consistent across reporting years.
- When reporting the number of learners for an internet enduring material activity, you should count all learners who completed all or a portion of the activity and whose participation can be verified in some manner. The accreditor would not consider individuals that only downloaded or accessed the activity but did not actually complete a portion of it to be learners.
- The hours of instruction for an internet enduring activity should indicate the amount of time it takes an individual learner to complete the activity.
Want to use Survey Magnet for your continuing education events?
Request a one-on-one demo with an expert, or submit an RFP to get started today.
About George
George Heitzmann manages software projects focusing on events and continuing education management. Originally from South Africa, he graduated from the University of Cape Town with a Bachelor of Business Science Degree in Marketing and received a Project Management Professional (PMP) Certification from the Project Management Institute. At CadmiumCD, George is project manager for the Survey Magnet.