Enabling Peer Review in Moodle: A Look at the Workshop Activity

In my experience providing Moodle support and training at NC State, I’ve seen a lot of instructors making use of popular Moodle activities like Assignment, Forum, and Quiz. However, Moodle includes a variety of other tools as well that aren’t as widely used, and in this article I will examine one of them. The Workshop activity in Moodle is similar in some ways to the Assignment activity, but it adds several additional features. Most notably, the Workshop activity includes a peer review feature, which allows students to assess each other’s work based on criteria established by the instructor. Students’ Workshop grades are calculated as a combination of two scores: a score based on their own submission and a score based on their assessment of other students’ work.

Advance Planning is Key

A Workshop activity progresses through several sequential phases, and the instructor determines when the activity advances from phase to phase. Prior to the five phases of a Workshop activity, however, I like to think of there being a “Phase 0,” during which the instructor plans ahead and makes certain choices in advance about how the Workshop activity will proceed. The choices that are made during the planning phase affect how the instructor will create and administer the Workshop activity, which determines what students will experience throughout the activity.

Perhaps the most important choice to make when planning a Workshop is to select a grading strategy. The grading strategy determines the method that will be used to score student submissions, and thus it defines the assessment form that student reviewers will use when evaluating their peers’ work. There are four built-in grading strategies from which to choose; all of them allow the instructor to use multiple assessment criteria, but the four options differ in how the criteria are labeled and structured. For example, the “Accumulative Grading” option allows the instructor to create a simple point value for each criterion, while the “Rubric” option allows for a more complex matrix of descriptive scoring levels.

Another option to plan in advance is the allocation method for student submissions. This setting refers to the various ways that student work might be distributed among other students for peer review. Submissions can be allocated manually, with specific students assessing other specific students, or they can be distributed randomly to peers within parameters set by the instructor.

The Workshop activity also allows instructors to create detailed instructions for students. In addition to the overall Workshop instructions, additional specific instructions may be provided for both the submission component and the assessment component of the student’s work. The instructor can also provide examples for both submissions and assessments, in order to demonstrate what kind of work is required from students during the activity. Deciding on what instructions and examples to provide is another important part of an instructor’s work when planning a Workshop activity.

Sequence of Workshop Phases

The Workshop activity is organized into a series of phases, and different things happen during each phase. Throughout the activity, the instructor is responsible for moving the Workshop from one phase to the next. The 5 phases of the Workshop activity (described below) are:

1. Setup

2. Submission

3. Assessment

4. Grading evaluation

5. Closed phase

During the Setup phase, the instructor configures settings, creates instructions, and most importantly, creates the assessment form that the activity will use. Students do not access the Workshop activity during the Setup phase. After setting up the Workshop, the instructor advances the activity to the Submission phase, during which students submit their work. The instructor configures the allocation method, and the submissions are allocated to other students for assessment. At this point, the Workshop is advanced to the Assessment phase; during this phase, students review each others’ work and assign scores based on the assessment form that the instructor has provided. Next, the instructor moves the activity into the Grading evaluation phase, where student scores (for both submission and assessment) are calculated. Finally, the instructor advances the Workshop into the Closed phase. After the Workshop activity has been closed, students can see their own grades, peer reviews of their work, and any other feedback.

The diagram below illustrates the overall flow of a typical Workshop activity throughout planning and delivery (click diagram for larger version).

Calculating Workshop Grades

The grade for a Workshop comprises two components: the submission grade and the assessment grade. When setting up the activity, the instructor determines the maximum score for both components (this is another decision to make when planning a Workshop). By default, the submission grade is set to 80, and the assessment grade is set to 20, making the maximum Workshop grade 100, but instructors can customize these values to fit their own needs.

A student’s submission grade (the first component of the Workshop grade) comes from the mean score of all of the peer assessments of that student’s submission. If, for example, a student receives four assessments with scores of 100, 100, 95, and 60, then that student’s submission score will be (100+100+95+60)/4 = 355/4 = 88.75. Calculating a student’s assessment grade (the second component of the Workshop grade) is somewhat more complicated; this score is based on comparisons with other assessments of the same submission. In general, an assessment that is more similar to other assessments will get a higher score, and an assessment that differs from other assessments will get a lower score. For example, if a particular submission gets assessment scores of 100, 100, 95, and 60, the reviewer who gave the grade of 60 is more likely to get a lower assessment grade than the other three reviewers. Instructors can adjust how strict or lax the score comparison is when calculating assessment grades.

Where to Get Help

The Workshop is one of Moodle’s more complex activities, so I recommend it for use by instructors who have some degree of experience using Moodle, and who already feel comfortable using the Assignment activity. To reiterate what I said above, the main piece of advice I would give to an instructor coming to the Workshop activity for the first time is to plan ahead and think through the entire process before implementing a Workshop in a class. For more in-depth assistance using the Workshop activity (or any other part of Moodle), NC State instructors may contact DELTA’s LearnTech help desk, attend one of our training workshops, or schedule an individual consultation with our staff. All of these services are found on the Get Help page of the DELTA website.