Study 3: From insecurity to confidence: evaluation of an innovative panel members’ training for guideline development – a survey

Guideline panels involve diverse roles, each with unique responsibilities and skill sets, making tailored training essential. Therefore, this study aimed to investigate how a panel member-tailored program grounded in five educational design principles: contextual, constructive, blended, just-in-time and collaborative learning can support guideline panel members during guideline development.

Methods

The training was co-created with guideline methodologists, educationalists, and health economists. Intended learning outcomes were developed using the Responsible, Accountable, Consulted, and Informed (RACI) matrix. An anonymised survey with a Likert-scale of 1 (completely disagree) - 5 (completely agree) and open-ended questions based on the training design principles was used to evaluate the training.

Data were analysed using descriptive statistics and qualitative content analysis. Results: Six multidisciplinary guideline panels participated in the training. Half of the panel members (n=33) and 60% of the project leads (n=9) completed the survey.

The training improved panel members’ understanding of the guideline process (85%, 23/27) and their preparedness for their tasks (53%, 15/28). Contextual learning enhanced relevance by aligning the training to panel members’ roles and tasks (82%; 23/28), and the examples were realistic (89%; 25/28). Constructive learning elements encouraged active engagement with the learning material (73%; 19/26).

The blended and just-in-time principle provided flexibility, allowing the training to accommodate the diverse development processes. However, the synchronous in-person components could be further strengthened. Approximately half of the participants indicated that the online modules provided sufficient knowledge to actively participate in the panel meetings (55%; 15/27).

Furthermore, 80% (20/25) of the panel members agreed that combining online preparatory assignments and in-person training supported their learning. Lastly, the training raised awareness about the importance of cost considerations (55%;12/22) and encouraged consideration of how costs could be included in the guidelines (59%; 13/22). Conclusion: Training grounded in blended, constructive, contextual, collaborative, and just-in-time design principles can meaningfully support guideline panel members during guideline development.

Data Attached, you find the SPSS data and syntax used to analyse the data.