AMOWA Evaluation Instruction:

Some online social media objects (such as YouTube videos or online News articles) include useful and interesting comments. However, due to the huge number of comments, it is often time-consuming and challenging to identify useful comments.

AMOWA is a Web service, which provides automated support for faceted browsing and ranking of social media comments. The service extracts the main topics discussed in the comments and characterizes each comment according to different facets (e.g., subjectivity, emotional level, Informative, and offensiveness of comments). You can use the system to explore the comments through a combination of topics and facets that will allow you to filter and extract those that fit your interests.

The goal of this study is the evaluation of the AMOWA service. In order to evaluate the service you should (1) use the AMOWA service and rank comments for a video based on time (chronological order) (2) use the AMOWA service and rank comments for the same video accordance with your preferences by selecting provided facets and topics. (3) for each ranking strategy (chronological or faceted ranking), vote for each comment if the comment is:

  • Interesting: If it contains interesting content for you personally and not necessarily for others users.
  • Relevant: If it contains relevant content to your selected facets and topics. Please note that the comment does not necessarily have to be directly relevant to the video content. The facets and topics refer to the comments, which not necessarily match the topic of the video.

Please follow the instruction below carefully:

PLEASE NOTE: IN ORDER TO GET APPROVAL FOR THIS TASK YOU SHOULD COMPLETE ALL 6 STEPS AND FOUR SUB_STEPS (4a-4d).

  1. Create an account using the following address: http://amowa.cs.univie.ac.at:8080/Frontend/register.html. Please note it is important that you use a real e-mail address as user name, so we can reach you for payment.
  2. Use the AMOWA service: http://amowa.cs.univie.ac.at:8080/Frontend/ Start by ranking comments for the selected video based on time (YouTube default). To do this you should:
    • (a) select the title of a video from the Option Box and click the load-button. Some videos have lots of comments, therefore the loading will take up to 40 seconds, please DO NOT press the loading button again.
    • fig1
    • (b) the system will show the video along with the list of topics and facets related to the comments of the video on the left.
    • (c) watch the selected video.
    • (d) For chronological order, select “chronological” and you will immediately see comments on the right side.
    • fig2
  3. For each comment listed on the right side, you will see two voting choices ("interesting" and "relevant"). For the chronological ranking, just vote on all or at least the first 30 comments, if the comment is interesting for you or not. You do not need to vote on "relevant" for the chronological order.
  4. fig3

    After voting for at least 30 comments, click on “Save votes" at the end of the list. Please note that you should vote for at least 30 comments, otherwise the system does not let you save your votes.

    fig4
  5. You have just completed the "chronological" ranking step. Now, use the AMOWA service to rank comments based on your preferences by selecting different topic(s), facet(s), or both. Please follow the four steps below::
    • (4a) Select one or more Topics in accordance with your preferences and vote on all or at least the first 30 comments in the list : “is the comment interesting for you or not” and also vote “is the comment relevant to your topic selection”. After finish voting, click on Save votes at the end of the list.
    • fig5
    • (4b) Select one or more Facets from these Facets (Religious referenced, Subjective opinion, Affective, Offensive, Anger oriented, Sad oriented) in accordance with your preferences and vote on all or at least the first 30 comments in the list: “is the comment interesting for you or not” and also vote on “is the comment relevant to your selection of facets”. After finish voting, click on Save votes at the end of the list.
    • (4c) Then select one or more Facets from these Facets (Informative, Video timestamp) in accordance with your preferences and vote on all or at least the first 30 comments in the list : “is the comment interesting for you or not” and also vote on “is the comment relevant to your selection of facets”. After finish voting, click on Save votes at the end of the list.
    • (4d) Select a combination of a Facet(s) and/or a Topic(s) in accordance with your preferences and vote on all or at least the first 30 comments in the list: “is the comment interesting for you or not” and “is the comment relevant to your facets and/or topics selection”. After finish voting, click on Save votes at the end of the list.
  6. Having completed the 4 steps above, repeat the steps 2 to 4 for the second video.
  7. After completing all ranking steps for both videos please use the following online form and provide us your background and feedback http://amowa.cs.univie.ac.at:8080/Frontend/feedback.html. Please note this is a very important step, which helps us check and approve your contributions and we need this information to be able to send you the gift certificate..
  8. fig6