We often create scenario-based questions because we consider them to be more “challenging” than the regular ones. However, just adding a scenario doesn’t make a question challenging. The scenario that you design has to be authentic and real. In fact, I will stick my neck out and say that it’s better to test for recall than design a scenario that is unreal and makes the question a complete give-away.
Here is a sample scenario-based question for a topic on communication. What is the problem with this scenario-based question?
Here is what I find inappropriate in this question.
Firstly, it reinforces stereotypes – bossy and dictatorial is bad, while submissive and soft-spoken is good. While working in an office, everything is context-dependent. Some situations demand a strong approach, while others may need to be tackled sensitively.
Secondly, the question is posed from Myra’s point of view. Obviously, the answer then would be “Yes”. This ignores the fact that communication is a two-way process, and requires that both parties find a way to listen and to be heard.
Finally, from a testing perspective, it is hardly challenging. The answer is clearly evident and almost everybody will get it right. Basically, the difficulty index for this question is low. It doesn’t challenge the learner at all.
Here is a modified version of this question.
In this modified version, you can see that William has been attributed to some positive traits as well. And, the question directs the learner to analyse the whole situation, and identify portions that depict the strengths in William’s communication style. You can build on this by adding another question that directs the learner to identify areas of improvement in William’s communication style. You can do the same for Myra – by getting learners to identify her strengths and focus areas. This makes it real and authentic from a scenario perspective and challenging from a question perspective.
So, the next time you write a scenario-based question, ask yourself if the scenario is adding any value and if it is making the question challenging. If it isn’t, then it’s better to go with a simple question.
Tools
Type
Companies
Agree with you Anand, and that’s why I believe that scenario-based questions for highly complex and science/engineering material must be designed by SMEs.
Another issue with scenario based questions is the vagueness or incomplete picture of the scene been provided by the Trainer. This is more evident in the Technical & Engineering courses.
This leaves the student wondering what could be the correct answer, as then there would be several of the ways, the things can move if some of the parameters which are not defined in the question changes.
For example if the Question is about find a cause of a particular behavior in a machinery system, while not stating the process and sourrounding conditions to a certain extent cannot be answered in crisp manner and can have large variances of correct answers which requires the trainer’s substantial time to assess.