By the end of the course all learners will be able to:

1. Recognize and advise on when and in which settings to use the Classical Model (CM) for performing Structured Expert Judgment
2. Account for uncertainty assessments in complex decision-making context when data pose issues
3. Use the CM to analyze expert data and obtain answers to questions of interest
4. Participate in an optional IDEA Protocol module, which uses a different method of performing Structured Expert Judgment.

Verified learners will have the added benefit of being able to:

1. Get an in-depth perspective on the CM method
2. Analyze expert data to apply Structured Expert Judgement methods to real world scenarios
3. Participate in optional modules about dependence elicitation and eliciting probabilities.

WEEK 1: Why and when to use SEJ?
Consider why and when to use Structured Expert Judgment (SEJ) and the Classical Model (CM), and then apply the model to applicable scenarios.

WEEK 2: Statistical accuracy (calibration) and information score
Learn how to use two key performance measures within the CM effectively.

WEEK 3: Performance-based weights and the Decision Maker
Learn how to aggregate expert opinion based on performance-based weights. Review other weighting schemes and evaluate them with respect to in-sample and out-of-sample validation techniques.

WEEK 4: Data analysis using Excalibur
The learners receive expert data that they will use to:
i) Compute the statistical accuracy and information scores for each expert,
ii) Aggregate their assessments with various weights, and
iii) Comment on the performance of the resulting Decision Makers.

WEEK 5: Applications of CM
Learn about real CM studies using an available TU Delft SEJ dataset and discuss particulars of the different studies provided.

WEEK 6: Practical matters (biases, experts, elicitation)
Consider the practical matters that are necessary for running the elicitation. Special attention will be given to biases and how to train experts to assess uncertainties.
Optional modules about another SEJ approach (the IDEA protocol) will be provided for learners who are keen on learning about an alternative method. Modules on dependence elicitation and eliciting probabilities will be provided to verified learners who want to learn about other contexts for which SEJ methods are appropriate.
Additionally, a more advanced course will be available for learners who are keen to apply the model to a project in their own situation to a problem of interest.

Sithara Buddhini (Sri Lanka) :
“This course was really helpful and informative in allowing us to understand the method even if we do not have any mathematical background. Just after completing this course, I designed a questionnaire according to the method and got amazing feedback and output. I really like the way this course is delivered, it was really simple to grasp the core. There is more research that is using these methods which are really interesting and powerful for analyzing expert judgments.”