=================================================================================== TAC 2020 STREAMING MULTIMEDIA KNOWLEDGE BASE POPULATION EVALUATION RESULTS (TASK 1) =================================================================================== Team ID: GAIA Organization(s): GAIA ******************** Run: teamisi-ev-202010162016 English Evaluation Results: Score Metric 0.4379 CoreferenceMetric_ENG_Entities_F1 0.3794 TypeMetricV1_ENG_Entities_F1 0.0000 CoreferenceMetric_ENG_Events_F1 0.0000 TypeMetricV1_ENG_Events_F1 0.0000 ArgumentMetricV1_ENG_Events_F1 0.0000 FrameMetric_ENG_Events_F1 0.0000 FrameMetric_ENG_Relations_F1 0.0000 TemporalMetric_ENG_Events_S 0.0000 TemporalMetric_ENG_Relations_S 0.0000 FrameMetric_ENG_F1 Spanish Evaluation Results: Score Metric 0.3445 CoreferenceMetric_SPA_Entities_F1 0.2558 TypeMetricV1_SPA_Entities_F1 0.2081 CoreferenceMetric_SPA_Events_F1 0.1912 TypeMetricV1_SPA_Events_F1 0.1593 ArgumentMetricV1_SPA_Events_F1 0.0886 FrameMetric_SPA_Events_F1 0.0714 FrameMetric_SPA_Relations_F1 0.0720 TemporalMetric_SPA_Events_S 0.0217 TemporalMetric_SPA_Relations_S 0.0824 FrameMetric_SPA_F1 Russian Evaluation Results: Score Metric 0.1793 CoreferenceMetric_RUS_Entities_F1 0.1522 TypeMetricV1_RUS_Entities_F1 0.0875 CoreferenceMetric_RUS_Events_F1 0.0882 TypeMetricV1_RUS_Events_F1 0.0296 ArgumentMetricV1_RUS_Events_F1 0.0152 FrameMetric_RUS_Events_F1 0.0366 FrameMetric_RUS_Relations_F1 0.0420 TemporalMetric_RUS_Events_S 0.0000 TemporalMetric_RUS_Relations_S 0.0226 FrameMetric_RUS_F1 Tri-lingual (English, Spanish, Russian) Evaluation Results: Score Metric 0.3246 CoreferenceMetric_Entities_F1 0.2689 TypeMetricV1_Entities_F1 0.1079 CoreferenceMetric_Events_F1 0.0951 TypeMetricV1_Events_F1 0.0716 ArgumentMetricV1_Events_F1 0.0373 FrameMetric_Events_F1 0.0340 FrameMetric_Relations_F1 0.0324 TemporalMetric_Events_S 0.0066 TemporalMetric_Relations_S 0.0361 FrameMetric_F1 ******************** Run: teamisi-ev-202010180029 English Evaluation Results: Score Metric 0.4384 CoreferenceMetric_ENG_Entities_F1 0.3800 TypeMetricV1_ENG_Entities_F1 0.0000 CoreferenceMetric_ENG_Events_F1 0.0000 TypeMetricV1_ENG_Events_F1 0.0000 ArgumentMetricV1_ENG_Events_F1 0.0000 FrameMetric_ENG_Events_F1 0.1125 FrameMetric_ENG_Relations_F1 0.0000 TemporalMetric_ENG_Events_S 0.0368 TemporalMetric_ENG_Relations_S 0.0484 FrameMetric_ENG_F1 Spanish Evaluation Results: Score Metric 0.3445 CoreferenceMetric_SPA_Entities_F1 0.2558 TypeMetricV1_SPA_Entities_F1 0.2081 CoreferenceMetric_SPA_Events_F1 0.1912 TypeMetricV1_SPA_Events_F1 0.1593 ArgumentMetricV1_SPA_Events_F1 0.0886 FrameMetric_SPA_Events_F1 0.0714 FrameMetric_SPA_Relations_F1 0.0720 TemporalMetric_SPA_Events_S 0.0217 TemporalMetric_SPA_Relations_S 0.0824 FrameMetric_SPA_F1 Russian Evaluation Results: Score Metric 0.1793 CoreferenceMetric_RUS_Entities_F1 0.1522 TypeMetricV1_RUS_Entities_F1 0.0875 CoreferenceMetric_RUS_Events_F1 0.0882 TypeMetricV1_RUS_Events_F1 0.0296 ArgumentMetricV1_RUS_Events_F1 0.0152 FrameMetric_RUS_Events_F1 0.0366 FrameMetric_RUS_Relations_F1 0.0420 TemporalMetric_RUS_Events_S 0.0000 TemporalMetric_RUS_Relations_S 0.0226 FrameMetric_RUS_F1 Tri-lingual (English, Spanish, Russian) Evaluation Results: Score Metric 0.3248 CoreferenceMetric_Entities_F1 0.2691 TypeMetricV1_Entities_F1 0.1079 CoreferenceMetric_Events_F1 0.0951 TypeMetricV1_Events_F1 0.0716 ArgumentMetricV1_Events_F1 0.0373 FrameMetric_Events_F1 0.0815 FrameMetric_Relations_F1 0.0324 TemporalMetric_Events_S 0.0285 TemporalMetric_Relations_S 0.0543 FrameMetric_F1 ******************** Run: teamisi-ev-202010181629 English Evaluation Results: Score Metric 0.4384 CoreferenceMetric_ENG_Entities_F1 0.3818 TypeMetricV1_ENG_Entities_F1 0.3216 CoreferenceMetric_ENG_Events_F1 0.3095 TypeMetricV1_ENG_Events_F1 0.3198 ArgumentMetricV1_ENG_Events_F1 0.1606 FrameMetric_ENG_Events_F1 0.1125 FrameMetric_ENG_Relations_F1 0.2699 TemporalMetric_ENG_Events_S 0.0420 TemporalMetric_ENG_Relations_S 0.1423 FrameMetric_ENG_F1 Spanish Evaluation Results: Score Metric 0.3445 CoreferenceMetric_SPA_Entities_F1 0.2558 TypeMetricV1_SPA_Entities_F1 0.2081 CoreferenceMetric_SPA_Events_F1 0.1912 TypeMetricV1_SPA_Events_F1 0.1593 ArgumentMetricV1_SPA_Events_F1 0.0886 FrameMetric_SPA_Events_F1 0.0714 FrameMetric_SPA_Relations_F1 0.0720 TemporalMetric_SPA_Events_S 0.0217 TemporalMetric_SPA_Relations_S 0.0824 FrameMetric_SPA_F1 Russian Evaluation Results: Score Metric 0.1793 CoreferenceMetric_RUS_Entities_F1 0.1522 TypeMetricV1_RUS_Entities_F1 0.0875 CoreferenceMetric_RUS_Events_F1 0.0882 TypeMetricV1_RUS_Events_F1 0.0296 ArgumentMetricV1_RUS_Events_F1 0.0152 FrameMetric_RUS_Events_F1 0.0366 FrameMetric_RUS_Relations_F1 0.0420 TemporalMetric_RUS_Events_S 0.0000 TemporalMetric_RUS_Relations_S 0.0226 FrameMetric_RUS_F1 Tri-lingual (English, Spanish, Russian) Evaluation Results: Score Metric 0.3248 CoreferenceMetric_Entities_F1 0.2697 TypeMetricV1_Entities_F1 0.2076 CoreferenceMetric_Events_F1 0.2158 TypeMetricV1_Events_F1 0.1707 ArgumentMetricV1_Events_F1 0.1001 FrameMetric_Events_F1 0.0815 FrameMetric_Relations_F1 0.1622 TemporalMetric_Events_S 0.0316 TemporalMetric_Relations_S 0.0933 FrameMetric_F1