Green border indicates well annotated example and red border indicates poorly annotated example

Instructions for Caption Quality

Contents

• Summary of the task
• Making decisions

Summary

In each task, you will look at a pair of (generated) images and judge which image better reflects a short text caption. Sometimes both images will be of roughly equal quality, in which case you can choose a third option, "Not Sure".

Making decisions

Sometimes, a caption will contain multiple separate elements, and your job is to judge which image captures more of these elements. In the below comparison, the caption describes both a glass of wine and a wine bottle. The image on the left distinctly shows both elements, whereas the image on the right only shows a wine glass clearly while the wine bottle is indiscernible. Therefore, the left image better captures the caption.

In some cases, both images will satisfy the caption completely (or a similar amount), but one image will look much more realistic than the other. In these cases, select the more realistic image.

Sometimes, it will be difficult to tell if either image is better. For example, they both might look completely unrealistic and jumbled, or both images might reflect the same amount of information from the caption. In these cases, if no image intuitively feels higher quality, chose "Not sure".

इन दिशानिर्देशों को पूरा पढ़ने के लिए आपको और निचे की और जाना होगा जब तक लास्ट पेज दिखाई देने ना लगे ?