"Questions with Feedback" Activities

Questions with feedback are questions designed to test learners' knowledge. By default, they include the following elements: 

• A question title (mandatory)
• A correct answer (mandatory)
• One or more incorrect answers (mandatory)
• The selection of the confidence level by the learner (optional, but highly recommended. This is configured in the sequence)
• An automatic correction by B12 with a generic feedback phrase "Correct answer" or "Incorrect answer" (see more details below). (mandatory)
• Feedback providing explanations for the correct and incorrect answers, containing text, and if necessary images, documents, hyperlinks, and even videos (optional, but highly recommended)
• Within the question, you can add content to complement the questions (texts, images, videos, downloadable documents).  This content can be placed: before the question, after the question and before the answer choices, or after the answer choices. (optional)
• Each question can be classified according to: its difficulty level, its knowledge topic, and one or more tags.  (See below for more details) 

Question before the learner's answer:

After the learner's answer: 

💡 It is important to differentiate questions with feedback from "survey" type questions.  
In a survey, there are no right or wrong answers.  For more information, see "Questionnaire" Activities: Surveys and Logs

Questions with feedback come in 4 types: 

• Single answer 
  • Only one possible answer.  The learner must select THE correct answer; all others are incorrect.
  • This includes two-choice questions, such as True or False.
  • Avoid "all of the above" answer choices; if multiple answer choices are part of the correct answer, use the Multiple answer type instead.
  • A small rule used everywhere on the Web: when there is a single choice, the button is round (radio button), and if the learner clicks on another choice, the activated button moves to the new choice.
• Multiple answer 
  • Multiple possible answers.  The learner must select ALL THE CORRECT ANSWERS to be correct.  An incorrect answer is given as soon as a choice is missing, or if an extra choice is selected.
  • It is not possible to award partial points for correctly identified choices (e.g., 2 correct answers out of 4, thus 0.5 points awarded).  It's all or nothing ;-)
  • There is no automatic instruction; we recommend adding it in the question. Example: "Multiple answers possible, select all correct answers."
  • Generally, these questions are much less successful for learners.  You can make it easier by clearly indicating the number of correct answers to find. Example: "Select the TWO correct answers from the four choices provided."
  • Generally, these questions will be classified as medium or difficult level.  This does not mean they should be avoided; on the contrary, complexity benefits reinforcement. 
  • A small rule used everywhere on the Web: when there are multiple answers, the button is square (checkbox). Each click activates or deactivates a checkbox without deselecting other already selected choices.
• Open question 
  • The learner must write the answer in a text field, without answer choices.
  • You can choose if the answer must be exact.  This can be relevant if you want to validate the exact knowledge of a term. You can allow certain mistakes, by specifying if the answer must match (select the relevant options) :
    • The exact characters (case, accents and spaces will be ignored)
    • The exact case
    • The exact accents
    • The exact spaces
  • If the answer does not need to be exact (to allow typos or alternative ways to describe the concept), artificial intelligence will be used for correction, based on the expected answers in the answer key. A certain degree of flexibility will therefore be allowed in validating the answer, as the correction is based on an estimate of the correspondence between the answer provided and the answer in the answer key. (See below for steps in the Studio, see also the section about Flagged questions.)
  • Unless it is a simple word or concept, these questions are generally classified as medium to difficult level, as no answer choices are present to give hints. Although more difficult, these questions are perfectly suitable for knowledge reinforcement! 
• Image hotspot 
  • An image is displayed on the screen, and the learner must place a marker on the area of the correct answer.
  • It is possible to select multiple areas to identify.  The designer must indicate if ALL the areas must be identified to be correct, or if AT LEAST ONE of the areas must be selected to be correct.
  • In the feedback, the area(s) are indicated in green on the screen, and an explanatory text can be added.
  • This can be used to: identify buttons to click in a web form, identify areas on a map, create a process diagram and ask the learner to identify the steps and errors in the diagram, etc... Using an image allows you to add some creativity to your single or multiple answer questions if you wish! 

For more on best practices in question writing, see the following article on the Apprentx website blog here: https://apprentx.rocks/fr/rediger-des-qcm-efficaces

Create and Edit a Single and Multiple Choice Question

In the Studio, you can create and edit questions from a sequence or from the Activities tab. For more information, see Create Activities

To create a new single or multiple choice question: 

1. Click on Create an Activity.  A sidebar will open with activity types. 

2. Click on Single answer or Multiple answer.

3. Enter the following mandatory fields: 

• The question
• At least two answer choices (generally, for a multiple choice question we recommend three answer choices. For more information: https://apprentx.rocks/fr/rediger-des-qcm-efficaces
• Indicate the correct answer(s) by clicking the button in front of the choice.
   

4. Enter the other recommended fields: 

• The difficulty level of the question.  For more details, see Question Difficulty Level
• The knowledge topic of the question.  For more details, see Knowledge Topics
• If relevant, one or more tags.  For more details, see Tags
• The feedback: different feedback can be set for correct or incorrect answers, or you can use the same one.   
• The complementary content: you can add text and media to enhance your question.  This could be a scenario to read, images to analyze, videos, downloadable task aids, etc. Hover your mouse over the " + " icon to display the button and activate the editing box. This content can be placed in different locations within the question:
  • Before the question
  • Between the question and the answer choices
  • After the answer choices

5. Reminder: elements added by default

• The choice of confidence level for the learner as a button to submit their answer (can be removed in the sequence settings).
• Automatic correction with the mention "correct answer" / "incorrect answer". The phrase displayed will vary depending on the confidence level and the correction of the answer.

  • If "Certain" and correct answer:

    You were certain and you were right! Correct answer.

  • If "Confident" and correct answer:  
    You were confident and can continue to be! Correct answer.
  • If "Uncertain" and correct answer:  
    You were uncertain and were wrong to be! Correct answer.
  • If "No idea" and correct answer:
    You had no idea, but rest assured: correct answer!

  • If "Certain" and incorrect answer:

    You were certain, but it is incorrect. Incorrect answer.

  • If "Confident" and incorrect answer:
    You were confident, but it is incorrect. Incorrect answer.
  • If "Uncertain" and incorrect answer:
    You were uncertain and were right to be. Incorrect answer.
  • If "No idea" and incorrect answer:  
    You had no idea, so nice try! Incorrect answer.

Create and Edit an Open Question

To create a new single or multiple choice question: 

1. Click on Create an Activity.  A sidebar will open with activity types. 

2. Click on Open question.

3. Enter the following mandatory fields: 

• The question
• At least one possible answer (possibility to add several).
• If the answer must be exact, check the box.  Otherwise, artificial intelligence will do the correction and allow for typos or synonyms.  You can test different answers in the preview on the right to see the correction of different possible answers.
• Then add the other recommended fields: difficulty level, knowledge topic, tag, feedback, complementary content. (see above for more details) 

🚩 Flagged Questions 

Only for these questions, we have added the possibility for the learner to indicate that they contest the correction, to adjust the AI correction if necessary. 

For the learner: a button "Report a problem: My answer should be correct / My answer should be incorrect". 
The benefit of the doubt is given to the learner; the question will be considered correct if they answer "My answer should be correct" and vice versa.  During review, the designer can adjust this if the suggestion is ultimately rejected and the learner should indeed have received an incorrect answer. 

For the designer:  

An alert will be displayed on the Program Overview page. 

An icon will also be displayed in front of the targeted question: 

 

You can then open the activity and accept or reject the suggestions.  

If you reject the suggestion, the success calculation will be adjusted (since initially we grant a correct answer to the learner who declares they should have had a correct answer). 

Create and Edit an Image Hotspot

To create a new single or multiple choice question: 

1. Click on Create an Activity.  A sidebar will open with activity types. 

2. Click on Image hotspot

3. Enter the following fields: 

• The question
   
• By default: the instruction "Place a marker on your answer" will be displayed.
• An image with at least one correct answer area.
• If two or more areas are added, indicate whether the learner must select ALL areas to be correct or at least one area. 

• The correct answer area(s) will be displayed in green in the feedback.
• Note: you can adjust or delete the correct answer area using the gray corners of the area.
• Then add the other recommended fields: difficulty level, knowledge topic, tag, feedback, complementary content. (see above for more details).