================================================================ TAC 2020 EPIDEMIC QUESTION ANSWERING EVALUATION RESULTS (TASK B) ================================================================ Team ID: h2oloo Organization(s): University of Waterloo Run: h2oloo_1 Query Metric Score Median Maximum CQ101 NDNS-Exact 0.0839 0.1503 0.2700 CQ101 NDNS-Partial 0.0748 0.1340 0.2869 CQ101 NDNS-Relaxed 0.0812 0.1456 0.2894 CQ102 NDNS-Exact 0.7355 0.4063 0.7355 CQ102 NDNS-Partial 0.6011 0.3457 0.6011 CQ102 NDNS-Relaxed 0.6194 0.3494 0.6194 CQ103 NDNS-Exact 0.1368 0.1031 0.1767 CQ103 NDNS-Partial 0.1160 0.0875 0.1499 CQ103 NDNS-Relaxed 0.1160 0.0875 0.1499 CQ104 NDNS-Exact 0.4367 0.1333 0.4982 CQ104 NDNS-Partial 0.4367 0.1333 0.4982 CQ104 NDNS-Relaxed 0.4367 0.1333 0.4982 CQ105 NDNS-Exact 0.3553 0.3401 0.4124 CQ105 NDNS-Partial 0.3318 0.3572 0.4059 CQ105 NDNS-Relaxed 0.3318 0.3254 0.3850 CQ106 NDNS-Exact 0.1934 0.1063 0.5912 CQ106 NDNS-Partial 0.1577 0.0962 0.4821 CQ106 NDNS-Relaxed 0.1577 0.0962 0.4821 CQ107 NDNS-Exact 0.2044 0.2447 0.5969 CQ107 NDNS-Partial 0.1663 0.1991 0.4857 CQ107 NDNS-Relaxed 0.1663 0.1991 0.4857 CQ108 NDNS-Exact 0.6340 0.4565 0.7288 CQ108 NDNS-Partial 0.5293 0.4417 0.6084 CQ108 NDNS-Relaxed 0.5293 0.4100 0.6084 CQ109 NDNS-Exact 0.7500 0.3804 0.7500 CQ109 NDNS-Partial 0.7500 0.3867 0.7500 CQ109 NDNS-Relaxed 0.7500 0.3867 0.7500 CQ110 NDNS-Exact 0.5033 0.5001 0.5953 CQ110 NDNS-Partial 0.3585 0.3627 0.4440 CQ110 NDNS-Relaxed 0.3640 0.3654 0.4347 CQ111 NDNS-Exact 0.5319 0.5054 0.7144 CQ111 NDNS-Partial 0.4847 0.4649 0.6510 CQ111 NDNS-Relaxed 0.4847 0.4649 0.6510 CQ112 NDNS-Exact 0.1770 0.2450 0.3337 CQ112 NDNS-Partial 0.1538 0.2175 0.3522 CQ112 NDNS-Relaxed 0.1538 0.2175 0.3522 CQ113 NDNS-Exact 0.1034 0.0793 0.2236 CQ113 NDNS-Partial 0.0869 0.0667 0.1879 CQ113 NDNS-Relaxed 0.0869 0.0667 0.1879 CQ114 NDNS-Exact 0.5026 0.3065 0.7244 CQ114 NDNS-Partial 0.4759 0.2952 0.6859 CQ114 NDNS-Relaxed 0.4759 0.2902 0.6859 CQ115 NDNS-Exact 0.6267 0.4474 0.7055 CQ115 NDNS-Partial 0.6267 0.4699 0.8956 CQ115 NDNS-Relaxed 0.6267 0.4699 0.8956 CQ116 NDNS-Exact 0.1711 0.1467 0.1711 CQ116 NDNS-Partial 0.1411 0.1210 0.1411 CQ116 NDNS-Relaxed 0.1411 0.1210 0.1411 CQ117 NDNS-Exact 0.1756 0.2244 0.5007 CQ117 NDNS-Partial 0.1756 0.2244 0.5007 CQ117 NDNS-Relaxed 0.1756 0.2244 0.5007 CQ118 NDNS-Exact 0.4144 0.4771 0.6880 CQ118 NDNS-Partial 0.2639 0.3276 0.4794 CQ118 NDNS-Relaxed 0.2637 0.3087 0.4540 CQ119 NDNS-Exact 0.3275 0.1554 0.3275 CQ119 NDNS-Partial 0.3287 0.1559 0.3287 CQ119 NDNS-Relaxed 0.3287 0.1559 0.3287 CQ120 NDNS-Exact 0.6630 0.5451 0.8264 CQ120 NDNS-Partial 0.6149 0.5149 0.7665 CQ120 NDNS-Relaxed 0.6149 0.5149 0.7665 CQ121 NDNS-Exact 0.3585 0.2090 0.4916 CQ121 NDNS-Partial 0.2743 0.1776 0.3967 CQ121 NDNS-Relaxed 0.2810 0.1819 0.3927 CQ122 NDNS-Exact 0.8128 0.4885 0.8152 CQ122 NDNS-Partial 0.8128 0.5132 0.8152 CQ122 NDNS-Relaxed 0.8128 0.5132 0.8152 CQ123 NDNS-Exact 0.7226 0.5139 0.7226 CQ123 NDNS-Partial 0.5432 0.4122 0.5432 CQ123 NDNS-Relaxed 0.5468 0.4118 0.5468 CQ124 NDNS-Exact 0.6871 0.3348 0.6871 CQ124 NDNS-Partial 0.5566 0.2711 0.5566 CQ124 NDNS-Relaxed 0.5566 0.2711 0.5566 CQ125 NDNS-Exact 0.0000 0.0000 0.0449 CQ125 NDNS-Partial 0.0000 0.0000 0.0449 CQ125 NDNS-Relaxed 0.0000 0.0000 0.0449 CQ126 NDNS-Exact 0.2870 0.2838 0.2972 CQ126 NDNS-Partial 0.2419 0.2407 0.2743 CQ126 NDNS-Relaxed 0.2419 0.2407 0.2725 CQ127 NDNS-Exact 0.1965 0.1922 0.6145 CQ127 NDNS-Partial 0.1703 0.1703 0.5328 CQ127 NDNS-Relaxed 0.1703 0.1703 0.5328 CQ128 NDNS-Exact 0.5117 0.2830 0.5150 CQ128 NDNS-Partial 0.4664 0.2661 0.4694 CQ128 NDNS-Relaxed 0.4664 0.2655 0.4694 CQ129 NDNS-Exact 0.2976 0.0387 0.3125 CQ129 NDNS-Partial 0.2450 0.0318 0.2572 CQ129 NDNS-Relaxed 0.2450 0.0318 0.2572 CQ130 NDNS-Exact 0.5951 0.3557 0.6022 CQ130 NDNS-Partial 0.5951 0.3675 0.6022 CQ130 NDNS-Relaxed 0.5951 0.3675 0.6022 MEAN NDNS-Exact 0.4065 0.3142 0.4143 MEAN NDNS-Partial 0.3593 0.2858 0.3662 MEAN NDNS-Relaxed 0.3607 0.2845 0.3675 ******************************************************* Run: h2oloo_2 Query Metric Score Median Maximum CQ101 NDNS-Exact 0.0839 0.1503 0.2700 CQ101 NDNS-Partial 0.0748 0.1340 0.2869 CQ101 NDNS-Relaxed 0.0812 0.1456 0.2894 CQ102 NDNS-Exact 0.7355 0.4063 0.7355 CQ102 NDNS-Partial 0.6011 0.3457 0.6011 CQ102 NDNS-Relaxed 0.6194 0.3494 0.6194 CQ103 NDNS-Exact 0.1368 0.1031 0.1767 CQ103 NDNS-Partial 0.1160 0.0875 0.1499 CQ103 NDNS-Relaxed 0.1160 0.0875 0.1499 CQ104 NDNS-Exact 0.4367 0.1333 0.4982 CQ104 NDNS-Partial 0.4367 0.1333 0.4982 CQ104 NDNS-Relaxed 0.4367 0.1333 0.4982 CQ105 NDNS-Exact 0.3581 0.3401 0.4124 CQ105 NDNS-Partial 0.3344 0.3572 0.4059 CQ105 NDNS-Relaxed 0.3344 0.3254 0.3850 CQ106 NDNS-Exact 0.2044 0.1063 0.5912 CQ106 NDNS-Partial 0.1667 0.0962 0.4821 CQ106 NDNS-Relaxed 0.1667 0.0962 0.4821 CQ107 NDNS-Exact 0.2044 0.2447 0.5969 CQ107 NDNS-Partial 0.1663 0.1991 0.4857 CQ107 NDNS-Relaxed 0.1663 0.1991 0.4857 CQ108 NDNS-Exact 0.7288 0.4565 0.7288 CQ108 NDNS-Partial 0.6084 0.4417 0.6084 CQ108 NDNS-Relaxed 0.6084 0.4100 0.6084 CQ109 NDNS-Exact 0.7500 0.3804 0.7500 CQ109 NDNS-Partial 0.7500 0.3867 0.7500 CQ109 NDNS-Relaxed 0.7500 0.3867 0.7500 CQ110 NDNS-Exact 0.5071 0.5001 0.5953 CQ110 NDNS-Partial 0.3612 0.3627 0.4440 CQ110 NDNS-Relaxed 0.3667 0.3654 0.4347 CQ111 NDNS-Exact 0.5850 0.5054 0.7144 CQ111 NDNS-Partial 0.5331 0.4649 0.6510 CQ111 NDNS-Relaxed 0.5331 0.4649 0.6510 CQ112 NDNS-Exact 0.1832 0.2450 0.3337 CQ112 NDNS-Partial 0.1592 0.2175 0.3522 CQ112 NDNS-Relaxed 0.1592 0.2175 0.3522 CQ113 NDNS-Exact 0.1034 0.0793 0.2236 CQ113 NDNS-Partial 0.0869 0.0667 0.1879 CQ113 NDNS-Relaxed 0.0869 0.0667 0.1879 CQ114 NDNS-Exact 0.5026 0.3065 0.7244 CQ114 NDNS-Partial 0.4759 0.2952 0.6859 CQ114 NDNS-Relaxed 0.4759 0.2902 0.6859 CQ115 NDNS-Exact 0.6433 0.4474 0.7055 CQ115 NDNS-Partial 0.6433 0.4699 0.8956 CQ115 NDNS-Relaxed 0.6433 0.4699 0.8956 CQ116 NDNS-Exact 0.1711 0.1467 0.1711 CQ116 NDNS-Partial 0.1411 0.1210 0.1411 CQ116 NDNS-Relaxed 0.1411 0.1210 0.1411 CQ117 NDNS-Exact 0.1750 0.2244 0.5007 CQ117 NDNS-Partial 0.1750 0.2244 0.5007 CQ117 NDNS-Relaxed 0.1750 0.2244 0.5007 CQ118 NDNS-Exact 0.4184 0.4771 0.6880 CQ118 NDNS-Partial 0.2665 0.3276 0.4794 CQ118 NDNS-Relaxed 0.2663 0.3087 0.4540 CQ119 NDNS-Exact 0.3275 0.1554 0.3275 CQ119 NDNS-Partial 0.3287 0.1559 0.3287 CQ119 NDNS-Relaxed 0.3287 0.1559 0.3287 CQ120 NDNS-Exact 0.6965 0.5451 0.8264 CQ120 NDNS-Partial 0.6460 0.5149 0.7665 CQ120 NDNS-Relaxed 0.6460 0.5149 0.7665 CQ121 NDNS-Exact 0.3584 0.2090 0.4916 CQ121 NDNS-Partial 0.2742 0.1776 0.3967 CQ121 NDNS-Relaxed 0.2809 0.1819 0.3927 CQ122 NDNS-Exact 0.8152 0.4885 0.8152 CQ122 NDNS-Partial 0.8152 0.5132 0.8152 CQ122 NDNS-Relaxed 0.8152 0.5132 0.8152 CQ123 NDNS-Exact 0.7226 0.5139 0.7226 CQ123 NDNS-Partial 0.5432 0.4122 0.5432 CQ123 NDNS-Relaxed 0.5468 0.4118 0.5468 CQ124 NDNS-Exact 0.6871 0.3348 0.6871 CQ124 NDNS-Partial 0.5566 0.2711 0.5566 CQ124 NDNS-Relaxed 0.5566 0.2711 0.5566 CQ125 NDNS-Exact 0.0000 0.0000 0.0449 CQ125 NDNS-Partial 0.0000 0.0000 0.0449 CQ125 NDNS-Relaxed 0.0000 0.0000 0.0449 CQ126 NDNS-Exact 0.2873 0.2838 0.2972 CQ126 NDNS-Partial 0.2422 0.2407 0.2743 CQ126 NDNS-Relaxed 0.2422 0.2407 0.2725 CQ127 NDNS-Exact 0.1965 0.1922 0.6145 CQ127 NDNS-Partial 0.1703 0.1703 0.5328 CQ127 NDNS-Relaxed 0.1703 0.1703 0.5328 CQ128 NDNS-Exact 0.5150 0.2830 0.5150 CQ128 NDNS-Partial 0.4694 0.2661 0.4694 CQ128 NDNS-Relaxed 0.4694 0.2655 0.4694 CQ129 NDNS-Exact 0.2924 0.0387 0.3125 CQ129 NDNS-Partial 0.2407 0.0318 0.2572 CQ129 NDNS-Relaxed 0.2407 0.0318 0.2572 CQ130 NDNS-Exact 0.6022 0.3557 0.6022 CQ130 NDNS-Partial 0.6022 0.3675 0.6022 CQ130 NDNS-Relaxed 0.6022 0.3675 0.6022 MEAN NDNS-Exact 0.4143 0.3142 0.4143 MEAN NDNS-Partial 0.3662 0.2858 0.3662 MEAN NDNS-Relaxed 0.3675 0.2845 0.3675 ******************************************************* Run: h2oloo_3 Query Metric Score Median Maximum CQ101 NDNS-Exact 0.0665 0.1503 0.2700 CQ101 NDNS-Partial 0.0593 0.1340 0.2869 CQ101 NDNS-Relaxed 0.0644 0.1456 0.2894 CQ102 NDNS-Exact 0.7336 0.4063 0.7355 CQ102 NDNS-Partial 0.5995 0.3457 0.6011 CQ102 NDNS-Relaxed 0.6178 0.3494 0.6194 CQ103 NDNS-Exact 0.1191 0.1031 0.1767 CQ103 NDNS-Partial 0.1010 0.0875 0.1499 CQ103 NDNS-Relaxed 0.1010 0.0875 0.1499 CQ104 NDNS-Exact 0.4982 0.1333 0.4982 CQ104 NDNS-Partial 0.4982 0.1333 0.4982 CQ104 NDNS-Relaxed 0.4982 0.1333 0.4982 CQ105 NDNS-Exact 0.2655 0.3401 0.4124 CQ105 NDNS-Partial 0.2479 0.3572 0.4059 CQ105 NDNS-Relaxed 0.2479 0.3254 0.3850 CQ106 NDNS-Exact 0.1657 0.1063 0.5912 CQ106 NDNS-Partial 0.1351 0.0962 0.4821 CQ106 NDNS-Relaxed 0.1351 0.0962 0.4821 CQ107 NDNS-Exact 0.1899 0.2447 0.5969 CQ107 NDNS-Partial 0.1545 0.1991 0.4857 CQ107 NDNS-Relaxed 0.1545 0.1991 0.4857 CQ108 NDNS-Exact 0.6135 0.4565 0.7288 CQ108 NDNS-Partial 0.5121 0.4417 0.6084 CQ108 NDNS-Relaxed 0.5121 0.4100 0.6084 CQ109 NDNS-Exact 0.6934 0.3804 0.7500 CQ109 NDNS-Partial 0.6934 0.3867 0.7500 CQ109 NDNS-Relaxed 0.6934 0.3867 0.7500 CQ110 NDNS-Exact 0.4404 0.5001 0.5953 CQ110 NDNS-Partial 0.3137 0.3627 0.4440 CQ110 NDNS-Relaxed 0.3185 0.3654 0.4347 CQ111 NDNS-Exact 0.4884 0.5054 0.7144 CQ111 NDNS-Partial 0.4450 0.4649 0.6510 CQ111 NDNS-Relaxed 0.4450 0.4649 0.6510 CQ112 NDNS-Exact 0.1694 0.2450 0.3337 CQ112 NDNS-Partial 0.1472 0.2175 0.3522 CQ112 NDNS-Relaxed 0.1472 0.2175 0.3522 CQ113 NDNS-Exact 0.1034 0.0793 0.2236 CQ113 NDNS-Partial 0.0869 0.0667 0.1879 CQ113 NDNS-Relaxed 0.0869 0.0667 0.1879 CQ114 NDNS-Exact 0.3871 0.3065 0.7244 CQ114 NDNS-Partial 0.3666 0.2952 0.6859 CQ114 NDNS-Relaxed 0.3666 0.2902 0.6859 CQ115 NDNS-Exact 0.5271 0.4474 0.7055 CQ115 NDNS-Partial 0.5271 0.4699 0.8956 CQ115 NDNS-Relaxed 0.5271 0.4699 0.8956 CQ116 NDNS-Exact 0.1626 0.1467 0.1711 CQ116 NDNS-Partial 0.1341 0.1210 0.1411 CQ116 NDNS-Relaxed 0.1341 0.1210 0.1411 CQ117 NDNS-Exact 0.1899 0.2244 0.5007 CQ117 NDNS-Partial 0.1899 0.2244 0.5007 CQ117 NDNS-Relaxed 0.1899 0.2244 0.5007 CQ118 NDNS-Exact 0.4030 0.4771 0.6880 CQ118 NDNS-Partial 0.2566 0.3276 0.4794 CQ118 NDNS-Relaxed 0.2564 0.3087 0.4540 CQ119 NDNS-Exact 0.3275 0.1554 0.3275 CQ119 NDNS-Partial 0.3287 0.1559 0.3287 CQ119 NDNS-Relaxed 0.3287 0.1559 0.3287 CQ120 NDNS-Exact 0.6002 0.5451 0.8264 CQ120 NDNS-Partial 0.5566 0.5149 0.7665 CQ120 NDNS-Relaxed 0.5566 0.5149 0.7665 CQ121 NDNS-Exact 0.3188 0.2090 0.4916 CQ121 NDNS-Partial 0.2440 0.1776 0.3967 CQ121 NDNS-Relaxed 0.2499 0.1819 0.3927 CQ122 NDNS-Exact 0.8052 0.4885 0.8152 CQ122 NDNS-Partial 0.8052 0.5132 0.8152 CQ122 NDNS-Relaxed 0.8052 0.5132 0.8152 CQ123 NDNS-Exact 0.7069 0.5139 0.7226 CQ123 NDNS-Partial 0.5314 0.4122 0.5432 CQ123 NDNS-Relaxed 0.5349 0.4118 0.5468 CQ124 NDNS-Exact 0.6113 0.3348 0.6871 CQ124 NDNS-Partial 0.4951 0.2711 0.5566 CQ124 NDNS-Relaxed 0.4951 0.2711 0.5566 CQ125 NDNS-Exact 0.0000 0.0000 0.0449 CQ125 NDNS-Partial 0.0000 0.0000 0.0449 CQ125 NDNS-Relaxed 0.0000 0.0000 0.0449 CQ126 NDNS-Exact 0.2873 0.2838 0.2972 CQ126 NDNS-Partial 0.2421 0.2407 0.2743 CQ126 NDNS-Relaxed 0.2421 0.2407 0.2725 CQ127 NDNS-Exact 0.1880 0.1922 0.6145 CQ127 NDNS-Partial 0.1630 0.1703 0.5328 CQ127 NDNS-Relaxed 0.1630 0.1703 0.5328 CQ128 NDNS-Exact 0.5036 0.2830 0.5150 CQ128 NDNS-Partial 0.4590 0.2661 0.4694 CQ128 NDNS-Relaxed 0.4590 0.2655 0.4694 CQ129 NDNS-Exact 0.3125 0.0387 0.3125 CQ129 NDNS-Partial 0.2572 0.0318 0.2572 CQ129 NDNS-Relaxed 0.2572 0.0318 0.2572 CQ130 NDNS-Exact 0.5957 0.3557 0.6022 CQ130 NDNS-Partial 0.5957 0.3675 0.6022 CQ130 NDNS-Relaxed 0.5957 0.3675 0.6022 MEAN NDNS-Exact 0.3825 0.3142 0.4143 MEAN NDNS-Partial 0.3382 0.2858 0.3662 MEAN NDNS-Relaxed 0.3395 0.2845 0.3675 *******************************************************