Introduction
During the course of this experiment, you will be presented with a series of sentences near the top of your screen. In each sentence, one word will appear in boldface type.
Below this sentence, you will see several descriptions of meanings that may or may not apply to the boldfaced word. The descriptions will usually consist of a word (colored blue) that is similar in meaning along with a short definition like what might be found in a dictionary. Your task is to rate, for each of these descriptions, how well they reflect the meaning of the boldfaced word in the sentence.
The five-point judgment scale
The scale that you will be using for your judgements ranges from 1, meaning that the description does not reflect the meaning of the boldfaced word at all, to 5, meaning that the description exactly reflects the meaning of the boldfaced word. You are free to use a rating more than once if you think two descriptions apply to the same degree, and you do not need to use all of the ratings with any given sentence.
Note that there are no right or wrong answers in this task, so please respond based upon your opinions alone. However, please try to be consistent in your judgments.
Progressing through the task
For each sentence that you are presented with, please supply ratings about the applicability of each description. Language is often ambiguous. Before reading the meaning descriptions, decide upon the most plausible meaning of the boldfaced word in the sentence. With this meaning in mind, determine to what extent each description reflects that meaning. You are free to assign the same rating to different descriptions, or to give them different ratings.
If you wish to leave a comment at any point during the task, just type it into the comment field that appears on the annotation screen. Once you have rated a given set of descriptions, click the “Submit” button near the bottom of the page.
As you proceed through the task, if you change your mind about a rating that you have previously supplied, you can navigate through the task using the Navigation panel located on the left side of your screen. You will see two options for navigating through the task: Current Sentence and Completed Sentences. If you click on “Completed Sentences”, you will be taken to a listing of all the sentences for which you have provided annotations thus far, allowing you to go back and make the changes you wish.
Once you have made your change, don't forget to click the “Submit” button again on that page to make sure your changes are recorded.
Once you are finished, clicking “Current Sentence” will take you back to the point where your forward progress in the task stopped, and you can continue annotating.
Erroneous or ungrammatical sentences
The sentences included in this task were gathered from various sources, including the World Wide Web. They may be very long or short and some may be badly formed in some way. You may find that there are things that make a certain sentence hard to understand. Try to ignore these issues; focus only on the meaning of the boldfaced word in the context in which it occurs. If you find that a sentence is so flawed as to impair your ability to understand what the boldfaced word means, or that the meaning of the boldfaced word is ambiguous in the sentence, please be sure to leave a comment to this effect.
Leaving and coming back
If you wish to stop working on the task and return to it later, simply click the link near the bottom of your page that says “logout”. Your progress through the task will be saved, along with all your ratings and comments. When you return, simply log back in and you will be taken directly to where you left off.
Thank you again for your participation in this task. These instructions will be available during the course of the task; just click the link that says “Click for Full Instructions” and they will open in a popup window.
back to: graded sense and usage annotation