Our coders are undergraduates from the University of Pennsylvania – primarily majoring in Communication at the Annenberg School – who are recruited for their exceptional academic performance, attention to detail, and interest in the project. Another advantage is that these coders closely represent the target adolescent population our measures address.
New coders undergo a rigorous training regimen in order to familiarize them with the data collection procedures as well as to ensure inter-coder reliability. We use Krippendorff’s alpha measure of 0.7 as our guide in determining reliability among coders.
After studying our code book, the trainees code a practice film in order to become familiar with the coding procedures. Once they are comfortable with the coding instrument, they progress through the next two training phrases. During the second phase, coders watch movies along with the provided answers in order to learn to interpret our code book during various situations. At the third phase, coders make their own judgments during the movies before being presented with the standardized responses so that they may reflect on their decisions. After coding multiple movies in this fashion the students move to the final training phase of coding several full-length films as well as additional segments that expose them to rare content. Inter-coder reliability is tested for all content measures and further training is conducted if necessary. Once satisfactory reliability has been established for each student, they start coding from the project’s actual sample.
