[2021.07.12] TUNiB ranked 1st in 2021 AI Online Competition

 

 

온라인_경진대회_노하우.pdf

8.63MB

 

 

TUNiB participated in the 2021 AI Online Competition - Emotional Classification Model for Elderly hosted by the Ministry of Science and ICT of South Korea.

TUNiB ranked 1st in both Public / Private / Final Leaderboard utilizing data analysis, data augmentation, ensemble techniques, and more.

What is the 2021 AI Online Competition?

The AI Online Competition, which was first held in 2019 and marked its third anniversary this year, is a competition in which prospective entrepreneurs and startup companies compete for the ability to develop AI algorithms. The purpose of this competition was to discover companies with outstanding artificial intelligence technology and provide commercialization support opportunities to those companies.

• [Figure 1] Problems for the 2021 AI Online Competition

Emotional State Classification Model for Elderly

TUNiB participated in the 'Emotional State Classification Model for Elderly' task which was to predict the emotional state of the elderly through conversations between the elderly and the system response. The detailed conditions of the assignment are as follows.

---

[Problem] Emotional State Classification Model for Elderly

1) Problem Definition

• Task classifying the emotion state of the Elderly based on the conversations between the Elderly and the AI.

2) Background

• As the number of elderly people living alone increases rapidly, depression problems caused by economic difficulties and social isolation are emerging socially.
• Through talking with robots and AI speakers, efforts are being made to preemptively identify depression in the elderly generation.

3) Data Structure

• Input: Data for two to four turn conversations between elderly speakers and system responses.
• Output: One of the six classes of anger, sadness, anxiety, hurt, embarrassment, and joy.
• Train (4.57MB)
  • train.csv : 10,150 elderly-chatbot conversations with gender information and emotion state.
• Test (580KB)
  • test.csv : ID and 1,276 elderly-chatbot conversations with gender information.

4) Metric

• Macro F1 Score
  • F1 Score is the weighted average of Precision and Recall.
  • Macro F1 Score is primarily used in multi-class problems, calculated by averaging F1 Score by Class.

5) Restrictions

• Training time: up to 36 hours
• Inference time: up to 3 hours

---

Strategy

Considering that the data given is small and this is a competition, TUNiB used data augmentation and ensemble method as our primary strategy and also applied other approaches such as Pattern Exploiting Training (PET) and Task Adaptive Pre-Training (TAPT).

You can find out more about TUNiB's 1st place winning solution through the presentation video and the slides above.

---

Result

After many attempts, TUNiB was able to proudly rank first on both the Public / Private / Final Leaderboard. 🎉 🎉During the competition, TUNiB was able to check the score with the Public score that was scored using the part of the test data, and after the competition, the final score was re-ranked in addition to the Private score that was scored with the rest of the test data.

• [Figure 2] Final Leaderboard

Finally, the organizer checked the reproducibility and cheating problem after the end of the competition and announced the final ranking.

• Announcement of the final results of the 2021 AI Online Competition

 

Closing

Through this competition, we are pleased to inform that TUNiB has high natural language processing technology. We will continue to share our AI research and results, so please look forward to our next moves. Thank you.