Community Evaluation

Phase 2: Relevance
Judging Criteria

Read this post for information about Community Evaluation.

The task in this phase is to evaluate whether videos accurately represent the chosen emoji and are expressive enough to convey the emotion behind it.

👍 = video accurately represents the emotion behind the chosen emoji
👎 = video doesn't represent the chosen emoji

We're looking for videos that would be a good suggestion for someone looking to express the chosen emotion in a chat.

The video must express a reasonably close emotion to its emoji. It is acceptable if the underlying emotion is correct, but a better-matching emoji exists in the same range. (e.g. 😡 / 😠 or 😢/ 😥).

Acceptable reaction videos:

1. Show the correct emotion. Some videos may be visually similar to an emoji but express a completely different emotion.
✅ Good Examples, 🚫 Bad Examples

2. Convey the emotion clearly and unmistakably.
✅ Good Examples, 🚫 Bad Examples

3. Express a single emotion. Some videos show a more complex situation or state of mind that can't be described with the chosen emoji.
✅ Good Examples, 🚫 Bad Examples

4. Describe the emotion behind the emoji. It is not enough to show a visually similar action. It is acceptable to show a visually different action with the correct emotional payload.
✅ Good Examples, 🚫 Bad Examples

5. Have sufficient length to properly convey the emotion.
🚫 Bad Examples

Before you begin, make sure you understand the proper meaning behind each main emoji, available here. You should also study our examples, listed above.

If you see the same video twice or recognize a video from round 1, press the ‘👀x2’ button before rating it. This is optional, we will use algorithms to remove duplicates before the final results are announced.