Grading

Basic principles

In all Science Fights, the Jury evaluates the Team performances by publicly showing integer scores called Grades or G.

The Grades G from all n Jurors in a Group are used to calculate the Average Point P. Two extreme Grades, one maximum and one minimum, are replaced with one grade equal to their arithmetic mean. In the next step, P is determined as the arithmetic mean of the new data set of n−1 grades. This procedure has the advantage of weighting outliers less heavily. P is rounded to the nearest 0.1 points. An example below (see a pdf file) illustrates this procedure with some real data from the 2nd IYNT 2014 (2014-1-A-II-Rep.)



Each Team completes three performances (Reporter, Opponent, and Reviewer) in each Science Fight and earns three Average Points P which are summed up to obtain the Sum of Points SP. In case the Team obtains a Yellow Card, the SP is consequently reduced.

The values of SP in each Group are used to calculate the Criterion of Victory V which is set to V=1 for the Team with the highest SP and for one or two Teams which have an SP that differs from the top result by no more than 2 points (SPSPmax−2.) For the Teams in the Group which have (SPmax−10)≤SP<(SPmax−2), the Criterion of Victory is set to V=½. For the Teams which have SP<SPmax−10, V=0. The Criterion of Victory is the primary parameter that determines the placing and rank of each Team. It minimizes effects of statistical noise and Juror-to-Juror differences in grading.


Distributions of G and P

During the IYNTs 2013 through 2019 taken together, 10290 Grades were delivered in 471 stages. Each of the 471×3=1413 performances therefore obtained its Grades G from an average of n=7.3 Jurors. A total of 165 round-robin meetings (Science Fights in a Group) were played with three or two Teams each. In them, the Teams collected 471 values of SP and 471 values of V.



The graph (hi-res image, raw ASCII data) shows a fitted histogram of the Grades for each type of performance (Report, Opposition, and Review.)

The spread in the G along the X-axis is broad, indicating that nearly a whole spectrum of available G is used by the Jury, however the extreme G are inherently less frequent. We encourage each Juror to stay centered around 15 points for a Report, around 10 points for an Opposition, and around 5 points for a Review, each time weighting their G against what they believe an average IYNT performance is. In reality however the actual mean and standard deviation are 17.6±5.3 for a Report, 12.2±3.5 for an Opposition, and 6.6±1.8 for a Review, indicating that an average Juror shifts all their G to the right as compared to our guidelines. The distributions are moderately asymmetric, with the median Grades of 18 for a Report, 12 for an Opposition, and 7 for a Review.

In turn, the mean and standard deviation for the Average Points P are 17.6±4.2 for a Report, 12.1±2.8 for an Opposition, and 6.6±1.4 for a Review (raw ASCII data.) Such distributions of P are narrower than the respective distributions of G.


Spread of the Grades

At this point it is important to realize that the IYNT requires a direct comparison of results from parallel Groups, whilst each Team does not play every other Team during one Science Fight. We should therefore be aware of the extent to which parallel boards of Jury can be influenced by fluctuations and what grading parameters are uniformly objective indications of the relative strengths of all IYNT contestants.

A consistent grading is extremely important for the IYNT as it must allow reliable identification of winners in each Group and the ultimate winners of the competition.

Let us consider three hypothetical and extreme scenarios, Example 1, Example 2, and Example 3.



In the Example 1, all Jurors agree with each other and there is high confidence that Average Points P reflect objective differences between the two Teams. For each Juror and each Grade, G−P=0.

The Example 2 shows a situation where both Teams obtain equal Grades from the middle of the spectrum. Although it is natural that some games can end in a tie, this scenario is less advantageous. If no Team can impress the Jury or the Jury lacks sensitivity to hidden variances between the Teams, it is difficult to rank all Teams from top to bottom.

The Example 3 depicts an undesired event where different Jurors use radically different grading criteria. Team 1 earns 0.1 points more than Team 2, however the level of confidence to this difference is low because σG−P, or the spread in G given to one performance by several Jurors, is much wider than these eventual 0.1 points. Albeit both Average Points P are nearly equal to the pair of P in the Example 2, these two P do not reflect the serious differences between the Teams that each Juror has noticed and highlighted in their grading. Concluding that Team 1 shows a better performance than Team 2, or either Team in Example 2, is inconsistent with this set of Grades G.

Our aim is that separate Jurors put very similar Grades for one performance, however each Juror puts distinctly different Grades for different performances.


Grading criteria

During each IYNT, we brief Jurors and Teams on our grading and evaluation criteria. Our guidelines have evolved since 2013, and as of now consist of four partial grading criteria. Our aim is to keep the guidelines clear and simple, and make sure that any Juror relies on the fixed, common criteria when evaluating performances across the parallel Groups. The criteria are printed directly on the individual Juror's protocols.



The Jurors are asked to add or subtract points from a starting grade (15, 10, or 5) and decide on their final Grade G. Such a decision is individual, and an upwards of 99.8% performances cause come disagreement in the Grades given by Jurors and a spread of these Grades. No Grade can be corrected retroactively, and each Juror must justify any of their Grades upon the request of Team Captains or the Chairperson. Each G is public.

Find below the blank Juror's protocols used at the 7th IYNT 2019, as well as the slides from the most recent introductory briefing for Jurors and Teams.

  • Blank individual Juror's Protocol, A4 size (2019/07/30) [pdf]
  • Briefing for Jurors and Teams, slides by Ilya Martchenko (2019/08/19) [pdf]


Distributions of G−P

It is now good to look at the real data from all previous IYNTs and analyze the spread in the G given by different Jurors to one performance in one Science Fight.



The graph (hi-res image, raw ASCII data) shows each of the 10290 Grades G given during all previous IYNTs. X-coordinate indicates which of the 30 possible Grades G was given. Y-coordinate indicates the difference between this particular G and the Average Point P that was calculated on its basis.

To interpret the spread in the Y-axis, one should remember that if Example 1 would take place in each IYNT Stage, each G would be equal to its respective P, and G−P would globally collapse to zero. Luckily, the IYNT is not a paper-and-pencil exam, and its Jurors have opinions which result in a distribution of individual G around the P in each Stage.

Despite being statistically unlikely, extreme G−P occur at the IYNT as well. The table below reflects each of the 16 rare events wherein G−P was equal or exceeded 10 points in absolute value.

StageGPG−PJurorTeamSlides
2015-F-A-II722.0-15.0Andrei KlishinCroatia[pdf]
2016-2-E-I1022.1-12.1Danko MarušićIran-Khashayar---
2015-1-A-III2614.3+11.7Dušan RadovanovićSerbia-2[pdf]
2019-1-C-I920.6-11.6Domagoj PluščecRomania-Sol. Sq.[pdf]
2016-4-A-I516.3-11.3Ilya MartchenkoIran-Amordad---
2016-3-C-II1121.9-10.9Danko MarušićIran-Maple[pdf]
2019-F-A-III1323.5-10.5Giorgi KhomerikiNew Zealand[pdf]
2019-1-G-I2918.5+10.5Hanna KarpenkaChina-PML---
2018-F-A-III1323.5-10.5Ivan SyulzhynSwitzerland[pdf]
2016-2-D-III717.5-10.5Zahra YazdgerdiCroatia---
2019-S-B-III515.4-10.4Florian KochChina-PML[pdf]
2016-4-A-III2514.6+10.4Roya RadgoharGeorgia---
2016-2-A-I1121.2-10.2Giorgi KhomerikiIran-Mehr[pdf]
2019-S-B-II2413.9+10.1Al. FalchevskayaGreece-Anat. Col.[pdf]
2019-1-G-III1222.0-10.0Mladen MatevRussia-Moonlight[pdf]
2016-1-A-I1020.0-10.0Andrei KlishinGeorgia[pdf]


The standard deviations of the distributions of G−P for the three types of performances are found as follows: 3.21 for a Report, 2.18 for an Opposition, and 1.21 for a Review. These particular values are calculated with two extreme G in each Group taken with the weight of 1, rather than ½.



Statistical significance of SF results

Since the extreme G contribute to statistics of P with the weight of ½, we can prepare the working dataset of 10290−3×471=8877 processed grades that we label g (raw ASCII data for 2013, 2014, 2015, 2016, and 2017.)

g=G if the respective G is not extreme in the Group, and g=½(Gmax+Gmin) for the pairs of extreme Gmax and Gmin in each Group. The statistical parameters of the distributions of residuals g−P now provide crucial information on the statistical significance of each P, and in turn SP, and the IYNT rankings.

In case the SF results do not permit rejecting the null hypothesis that a slightly higher SP in a round-robin Science Fight is observed by chance, more than one Team earns V=1. The Sum of Victories SV keeps track of such statistically significant cases wherin one or several SF winners step forward.

We can define the significance of V=1 as the level of statistical confidence for the interval [SP−2...60] in one Science Fight Group. This statistical significance depends only on the values of g−P and number of Jurors n in the Group, and does not directly depend on the absolute magnitudes of G. This has the advantage of placing focus on congruence between opinions of Jurors, rather than ranking of Teams.

To illustrate how to compute the confidence of [SP−2...60] and [SP−10...60], let us analyze in depth one example of a round-robin Science Fight (2016-F-A, see a hi-res pdf file.)



The standard deviations σ of the distributions of g−P for each of three types of performances in this SF are found as follows: σREP=1.91 for a Report, σOPP=1.36 for an Opposition, and σREV=0.65 for a Review (each from a sample of 27 residuals g−P.) As per the IYNT procedure, SP is calculated as a sum of Average Points P for these three performances. By assuming σ2SP=σ2REP+σ2OPP+σ2REV, we can easily find σSP=2.43 in these Finals of the 4th IYNT 2016. It is now easy to determine the root-mean-square deviation ρSP by defining ρ2SP=σ2SP/(n−1), where number of Jurors is n=10.

The value of ρSP=0.81 evaluates the standard error of the mean, i.e. the statistical uncertainty of SP earned by any Team in the SF. This standard error is inherent in estimating whatever true value of SP from limited statistics. If the difference between any two sample-based SPi and SPj is comparable to or less than ρSP, they can be assumed statistically indistinguishable.

In the next step, we can find the confidence level for interval [SP−2...60] as a function of the number of degrees of freedom in a representative sample (i.e. n−1) and Student's t-score (i.e. 2/ρSP.) This is based on assuming that Student's t-distribution is a valid approximation.

This calculation yields the confidence level of 98.2% for the interval [SP−2...60], above a two-sigma significance threshold. In the Finals of other previous IYNTs, confidence levels for the interval [SP−2...60] were 99.1% in 2019, 95.9% in 2018, 96.1% in 2017, 94.8% in 2015, 98.8% in 2014, and 98.2% in 2013. A similar calculation yields the confidence level of 99.99999% for the interval [SP−10...60] in 2017 (above five-sigma), 99.99997% in 2016 (above five-sigma), and 99.99986% in 2015, 99.99971% in 2014, and 99.99827% in 2013 (above four-sigma in each case.)

The Table below summarizes the paramateres calculated in the similar manner for all 165 round-robin Science Fights of past IYNTs. We cordially acknowledge Hieorhi Liaśnieŭski and Dmitriy Baranov for their help in processing the data. Click on the headers to have the table sorted by any desired parameter.

SFσREPσOPPσREVσSFρSPt2nV=1V
2013-1-A1.601.390.722.240.912.19796.4100.0
2013-1-B3.022.340.703.881.741.15684.999.89
2013-1-C0.991.260.471.670.842.39596.399.99
2013-1-D1.181.080.531.690.752.65697.7100.0
2013-1-E1.162.780.423.041.521.31587.199.86
2013-1-F1.231.410.351.910.852.34696.7100.0
2013-2-A1.440.770.691.770.792.52697.4100.0
2013-2-B1.670.960.772.080.932.15695.899.99
2013-2-C2.291.290.822.761.231.62691.799.98
2013-2-D1.320.850.681.710.862.34596.099.98
2013-2-E1.441.100.461.870.842.39696.9100.0
2013-2-F0.991.610.802.050.842.39797.3100.0
2013-3-A1.080.710.551.400.702.85597.799.99
2013-3-B0.980.850.301.330.673.00598.099.99
2013-3-C1.780.981.062.291.151.74592.299.95
2013-3-D0.880.660.701.300.653.07598.199.99
2013-3-E0.482.210.462.301.151.74592.199.95
2013-3-F1.460.870.771.870.842.39696.9100.0
2013-4-A1.340.950.531.730.772.59697.6100.0
2013-4-B0.200.860.350.950.474.22599.3100.0
2013-4-C1.121.230.741.820.912.20595.399.98
2013-4-D1.210.770.371.470.663.03698.6100.0
2013-4-E1.431.000.461.810.902.21595.499.98
2013-4-F1.890.770.882.231.002.01694.999.99
2013-S-A1.360.970.911.900.852.35696.7100.0
2013-S-B1.641.440.792.320.882.28897.2100.0
2013-S-C1.501.080.661.960.742.70898.5100.0
2013-F-A1.811.261.022.430.812.471098.2100.0
2014-1-A2.421.950.713.181.421.40689.099.95
2014-1-B2.550.870.672.781.241.61691.699.98
2014-2-A1.751.180.522.170.972.06695.399.99
2014-2-B1.200.840.691.620.722.76698.0100.0
2014-3-A2.321.170.352.621.171.71692.699.98
2014-3-B2.041.030.682.391.071.87694.099.99
2014-4-A1.021.150.771.720.772.60697.6100.0
2014-4-B2.041.150.452.391.071.87694.099.99
2014-F-A1.290.860.561.650.672.97798.8100.0
2015-1-A2.961.230.673.281.471.36688.599.95
2015-1-B1.370.950.971.930.862.32696.6100.0
2015-1-C1.420.940.571.800.802.49697.2100.0
2015-1-D1.060.820.581.460.732.74597.499.99
2015-2-A3.023.281.234.621.891.06783.599.91
2015-2-B1.571.010.832.050.842.39797.3100.0
2015-2-C1.771.260.752.301.031.95694.599.99
2015-2-D0.771.430.681.770.792.53697.4100.0
2015-3-A2.491.260.412.821.151.74793.499.99
2015-3-B0.901.420.591.780.802.51697.3100.0
2015-3-C2.141.160.712.541.131.76693.199.98
2015-3-D2.031.090.552.371.061.89694.199.99
2015-4-A1.121.130.891.820.812.45697.1100.0
2015-4-B1.470.990.631.880.842.38696.8100.0
2015-4-C1.951.340.622.451.091.83693.699.99
2015-4-D2.240.730.632.441.091.83693.799.99
2015-S-A2.300.951.122.731.111.79793.999.99
2015-S-B2.340.841.002.681.091.83794.1100.0
2015-F-A3.211.760.823.751.131.771294.8100.0
2016-1-A3.451.960.844.061.821.10684.099.87
2016-1-B2.661.700.763.251.451.38688.799.95
2016-1-C2.621.520.953.171.421.41689.199.96
2016-1-D1.901.580.662.561.151.75692.999.98
2016-1-E2.291.541.263.041.361.47690.099.96
2016-1-F2.402.131.153.411.521.31687.799.94
2016-2-A1.971.330.662.471.111.81693.599.99
2016-2-B0.950.690.481.270.573.52699.2100.0
2016-2-C1.791.000.382.081.041.92593.699.97
2016-2-D2.041.770.722.801.251.60691.599.98
2016-2-E2.401.450.662.881.441.39588.199.89
2016-2-F1.592.191.303.001.341.49690.299.97
2016-3-A1.991.400.602.511.021.95795.1100.0
2016-3-B3.092.551.254.191.881.07683.399.84
2016-3-C2.371.580.712.941.201.68792.799.99
2016-4-A3.681.440.694.012.001.00581.399.62
2016-4-B2.331.240.782.751.231.63691.899.98
2016-4-C2.391.730.573.011.501.33587.399.87
2016-4-D2.521.650.703.091.381.45689.699.96
2016-4-E0.761.061.051.670.742.67697.8100.0
2016-4-F1.381.460.822.171.081.84593.199.96
2016-S-A2.191.630.882.871.171.71793.199.99
2016-S-B3.482.031.224.211.721.16785.699.94
2016-S-C2.101.100.602.451.002.00795.4100.0
2016-F-A1.911.360.652.430.812.471098.2100.0
2017-1-A1.841.501.002.570.972.06896.1100.0
2017-1-B2.201.100.692.550.962.07896.2100.0
2017-1-C1.861.680.622.581.051.90794.7100.0
2017-1-D1.910.730.542.110.862.32797.0100.0
2017-1-E1.760.910.642.080.852.35797.2100.0
2017-1-F2.091.090.682.461.001.99795.3100.0
2017-2-A2.111.490.772.691.101.82794.1100.0
2017-2-B3.272.051.214.041.531.31888.499.98
2017-2-C1.491.130.902.080.852.36797.2100.0
2017-2-D1.980.900.412.210.902.21796.6100.0
2017-2-E1.160.910.581.580.603.35899.4100.0
2017-2-F2.271.070.582.581.051.90794.7100.0
2017-3-A1.560.660.991.960.802.50797.7100.0
2017-3-B2.611.201.003.041.241.61792.199.99
2017-3-C1.741.471.072.521.031.95795.0100.0
2017-3-D2.831.070.763.121.271.57791.699.99
2017-3-E1.470.840.581.780.672.96899.0100.0
2017-3-F2.141.430.802.691.101.82794.1100.0
2017-4-A1.300.800.531.620.663.02798.8100.0
2017-4-B1.660.870.852.060.842.38797.3100.0
2017-4-C1.721.630.842.521.031.95795.0100.0
2017-4-D2.050.990.692.380.972.06795.7100.0
2017-4-E1.641.010.882.120.872.31797.0100.0
2017-4-F1.861.130.872.350.962.09795.9100.0
2017-S-A1.741.550.952.520.892.25997.3100.0
2017-S-B3.141.890.933.781.341.50991.3100.0
2017-S-C2.961.541.023.491.231.62992.8100.0
2017-F-A3.221.670.993.760.972.061697.1100.0
2018-1-A2.421.230.952.870.962.091096.7100.0
2018-1-B1.431.380.802.140.762.64998.5100.0
2018-1-C3.161.860.893.771.331.50991.4100.0
2018-1-D1.961.421.202.700.952.09996.5100.0
2018-1-E2.071.250.732.530.842.371097.9100.0
2018-2-A1.781.590.982.580.912.19997.0100.0
2018-2-B2.112.091.023.141.181.69893.2100.0
2018-2-C1.531.311.102.290.872.31897.3100.0
2018-2-D1.501.100.772.010.712.81998.9100.0
2018-2-E1.751.050.652.140.812.47897.9100.0
2018-3-A1.421.350.512.020.722.80998.8100.0
2018-3-B1.721.610.602.430.862.33997.6100.0
2018-3-C2.502.010.773.301.251.61892.4100.0
2018-3-D1.961.440.882.590.922.19997.0100.0
2018-3-E2.351.000.852.690.952.10996.6100.0
2018-4-A1.831.430.752.430.862.32997.6100.0
2018-4-B1.571.110.782.080.732.73998.7100.0
2018-4-C2.000.780.662.250.802.52998.2100.0
2018-4-D1.651.250.502.130.802.49897.9100.0
2018-4-E1.931.360.752.480.882.28997.4100.0
2018-S-A2.081.680.722.770.772.601498.9100.0
2018-S-B2.412.100.963.340.932.161497.5100.0
2018-S-C2.541.801.223.350.932.161497.5100.0
2018-F-A3.651.820.774.161.071.861695.9100.0
2019-1-A2.842.170.723.641.381.45890.599.99
2019-1-B1.201.500.562.000.762.65898.3100.0
2019-1-C2.531.641.043.191.211.66893.0100.0
2019-1-D1.902.060.892.941.111.80894.3100.0
2019-1-E1.561.610.642.330.882.27897.1100.0
2019-1-F1.681.150.862.210.842.39897.6100.0
2019-1-G3.131.320.863.511.431.40789.499.98
2019-2-A2.391.780.813.091.171.72893.5100.0
2019-2-B1.670.700.771.970.802.49797.6100.0
2019-2-C1.641.480.912.390.972.05795.7100.0
2019-2-D1.831.370.582.360.962.08795.8100.0
2019-2-E1.920.880.742.240.912.19796.4100.0
2019-2-F2.041.090.522.370.972.06795.8100.0
2019-2-G2.501.510.462.961.211.66792.699.99
2019-3-A2.281.390.752.771.131.77793.699.99
2019-3-B2.271.000.472.521.031.94795.0100.0
2019-3-C1.131.130.461.670.742.69697.8100.0
2019-3-D2.271.870.643.011.351.48690.199.96
2019-3-E1.811.380.452.321.041.93694.499.99
2019-3-F1.521.840.822.521.031.94795.0100.0
2019-3-G2.751.430.903.231.441.39688.899.95
2019-4-A2.701.670.533.221.441.39688.899.95
2019-4-B2.370.970.552.621.171.70692.699.98
2019-4-C1.801.490.882.501.121.79693.399.99
2019-4-D1.910.950.712.251.011.98694.899.99
2019-4-E1.651.050.832.130.872.30797.0100.0
2019-4-F1.461.340.862.160.972.07695.399.99
2019-4-G2.221.780.953.001.231.63792.399.99
2019-S-A2.161.330.742.640.932.14996.8100.0
2019-S-B2.581.441.023.131.101.81994.6100.0
2019-S-C1.930.580.732.140.712.801099.0100.0
2019-F-A2.271.410.752.770.742.701599.1100.0

In the IYNTs 2013 through 2019, there were 203 instances of V=1, 170 instances of V=½, and 98 instances of V=0 (including the 1st IYNT 2013 retrospectively.) If compared to the total number of round-robin Science Fights (165), these figures suggest that in (203−165)/165=23% of cases, second or third Teams in a Science Fight are indistinguishable from the Team with the maximum SP and also earn their V=1 (raw ASCII data.)

In a typical Science Fight, earning a V=½ or above is a four-sigma event, with an expected confidence level of (99.98±0.04)% for the interval [SP−10...60]. Earning a much more rewarding V=1 is a two-sigma event, with an expected confidence level of (95±4)% for the interval [SP−2...60]. Earning each V is a statistically independent event, and collecting several V=1 further contributes to the confidence in the placing of top IYNT Teams.

These results justify the importance of the Criterion of Victory V and importance of the fact that no Science Fight has ever been graded by less than 5 Jurors (while 90% of SFs, a total of 148, have been graded by 6 jurors or more.) As argued below, besides having different opinions, individual Jurors may also work on different grading scales. At all times, when looking at the IYNT scores, we ask whether their difference is representative of a real difference between Teams or whether it is a statistical fluke. An especially high level of significance is demanded if grading parameters are used to resolve the placing of eventual Semi-Finalists and Finalists.

By assuming that the grading parameters of Jurors would not improve considerably before the 8th IYNT 2020, we may estimate the average expected level of confidence for [SP−2...60] as a function of the number of Jurors n randomly selected to one Group. To do so, we can re-calculate ρSP from a historically global σSP.



Grading parameters of individual Jurors

For reference purposes, this table summarizes the individual grading parameters (within one IYNT) of all existing 209 Jurors. We cordially acknowledge Dmitriy Baranov and Hieorhi Liaśnieŭski for their help in processing the data. The presented parameters give a glimpse of the Jurors' perceptions of the grading scale in the IYNT. Click on the headers to have the table sorted by any desired parameter.

As of 2019, following 9 Jurors are ex-participants: Anton Khvalyuk (2013), Giorgi Tsereteli (2013), Ekaterine Dadiani (2014), Giorgi Khomeriki (2014), Diana Sokhashvili (2015), Toma Rtveliashvili (2015), Sofia Anisimova (2016), Sofiya Guzik (2016), Ekaterina Rosnovskaia (2018.)

Yr, StNameGσGσG−PG−PnGnSFnTκ
2019 Ilya Martchenko 11.95.71.8-0.954614+0.9
2018 Ilya Martchenko 12.56.21.8-0.54559+1.1
2017 Ilya Martchenko 12.15.82.4-0.254612+0.9
2016 Ilya Martchenko 10.36.13.0-1.94258+1.9
2015 Ilya Martchenko 12.76.12.8-0.14868+0.9
2014 Ilya Martchenko 12.26.72.1-1.03955+1.7
2013 Ilya Martchenko 12.56.02.1-0.348610+0.9
2019 Mladen Matev 11.15.72.3-1.354614+1.2
2018 Mladen Matev 11.55.21.9+0.045510+0.6
2017 Mladen Matev 11.14.72.7-0.954612+0.2
2016 Mladen Matev 11.85.62.4-1.151611+0.8
2015 Mladen Matev 12.15.12.1-1.34869+0.2
2014 Mladen Matev 10.95.32.5-1.43044+0.9
2013 Mladen Matev 12.55.53.0+0.039510+0.4
2019 Evgeny Yunosov 15.17.42.1+1.42738+1.2
2018 Evgeny Yunosov 16.97.52.1+2.1913+0.6
2017 Evgeny Yunosov 14.26.72.9+2.345513+0.9
2016 Evgeny Yunosov 13.35.51.6+0.11825+0.1
2015 Evgeny Yunosov 14.36.11.8+0.43349+0.3
2014 Evgeny Yunosov 14.46.71.7+1.23955+0.8
2019 Milen Kadiyski 13.26.32.3+0.154613+0.9
2018 Milen Kadiyski 12.86.53.1+0.945511+1.3
2017 Milen Kadiyski 13.15.61.6+1.154611+0.3
2015 Milen Kadiyski 14.66.61.9+1.14869+0.6
2014 Milen Kadiyski 16.36.91.8+1.14255+0.2
2019 Andrey Kravtsov 14.15.52.0+0.93649-0.3
2018 Andrey Kravtsov 13.55.52.3+1.536411+0.0
2017 Andrey Kravtsov 14.86.01.8+1.43649-0.1
2015 Andrey Kravtsov 13.65.62.1+1.12137+0.0
2014 Andrey Kravtsov 15.16.72.1-0.43954+0.5
2018 Alena Kastenka 11.86.52.0-0.73649+1.7
2017 Alena Kastenka 10.85.81.9-0.345512+1.4
2016 Alena Kastenka 10.55.92.5-1.03048+1.6
2015 Alena Kastenka 11.55.62.2-0.53346+0.9
2019 Giorgi Khomeriki 10.64.72.5-1.854612+0.4
2018 Giorgi Khomeriki 9.64.02.9-2.61826+0.0
2017 Giorgi Khomeriki 10.04.61.6-0.645513+0.6
2016 Giorgi Khomeriki 7.73.62.4-3.12437+0.4
2017 Andrei Klishin 12.66.93.2+0.054611+1.8
2016 Andrei Klishin 10.86.03.0-1.851610+1.5
2015 Andrei Klishin 13.56.93.5-0.44869+1.4
2018 Dina Izadi 11.45.82.1-0.954612+1.2
2016 Dina Izadi 11.97.21.6+0.8913+2.3
2013 Dina Izadi 10.44.92.1-1.048511+0.7
2018 Ivan Syulzhyn 10.15.02.6-1.054612+0.9
2017 Ivan Syulzhyn 10.34.82.3-1.445512+0.6
2016 Ivan Syulzhyn 9.85.61.3-1.73347+1.6
2018 Val. Lobyshev 12.36.02.3+1.354611+0.9
2017 Val. Lobyshev 11.86.12.0+0.454612+1.3
2013 Val. Lobyshev 13.56.32.0+1.336410+0.8
2018 Stan. Krasulin 9.15.42.5-1.645512+1.7
2017 Stan. Krasulin 9.44.42.0-0.736411+0.5
2016 Stan. Krasulin 10.14.22.5-2.31826+0.1
2019 Florian Koch 10.15.83.3-0.936411+1.6
2018 Florian Koch 9.44.11.9-0.81826+0.3
2017 Florian Koch 9.23.92.3-1.745511+0.1
2019 Volha Uhnachova 13.25.62.4+0.81826+0.2
2018 Volha Uhnachova 10.95.71.4+0.62738+1.2
2017 Volha Uhnachova 10.84.91.4+0.23649+0.5
2019 Murray Chisholm 10.44.81.8-1.745511+0.5
2018 Murray Chisholm 11.85.72.0+0.154612+0.8
2017 Murray Chisholm 9.74.32.0-1.545513+0.3
2015 Gur. Mikaberidze 11.94.61.9-0.93957-0.3
2014 Gur. Mikaberidze 11.05.62.2-1.62444+1.1
2013 Gur. Mikaberidze 11.15.12.0-0.333410+0.6
2017 Danko Marušić 10.64.32.4-0.545511+0.0
2016 Danko Marušić 11.25.13.7-1.83957+0.5
2015 Danko Marušić 11.55.32.8-0.73958+0.6
2017 Dmitriy Agarkov 12.34.91.7-0.31825-0.2
2014 Dmitriy Agarkov 14.66.21.3-0.93954+0.2
2013 Dmitriy Agarkov 12.75.61.4-0.645510+0.4
2018 Klim Sladkov 8.85.42.4-2.754611+1.9
2017 Klim Sladkov 9.05.13.0-2.245513+1.5
2019 Domagoj Pluščec 10.55.52.9-1.645514+1.2
2018 Domagoj Pluščec 10.05.72.8-1.654612+1.6
2019 Giorgi Bakhtadze 11.35.52.5-1.32739+0.9
2018 Giorgi Bakhtadze 11.56.12.3+1.31826+1.4
2018 Marc Bitterli 9.24.72.0-1.245512+1.0
2017 Marc Bitterli 11.15.72.1-0.52737+1.2
2018 A. Chervinskaia 10.06.11.6-0.736411+2.0
2017 A. Chervinskaia 11.94.91.8+0.045512+0.0
2018 Alexandr Nadeev 11.05.51.8-0.145512+1.0
2017 Alexandr Nadeev 12.25.82.1+0.454614+0.9
2018 Kseniia Wang 12.16.11.5+0.445511+1.2
2017 Kseniia Wang 13.86.21.6+1.036410+0.6
2016 Dmitry Zhukalin 12.86.92.9+0.12436+1.7
2015 Dmitry Zhukalin 13.65.71.4+0.73657+0.1
2015 Aleks. Dimić 12.65.91.4-0.33347+0.8
2014 Aleks. Dimić 12.96.11.9-0.33044+0.8
2019 Usev. Gaponenko 12.15.61.7-1.036410+0.6
2018 Usev. Gaponenko 12.45.82.1+0.854612+0.8
2019 Irina Valtcheva 12.85.71.6+0.745514+0.5
2018 Irina Valtcheva 11.25.31.9+0.145512+0.8
2017 Wang Sihui 10.45.01.4-0.136410+0.7
2015 Wang Sihui 13.35.91.5-0.32746+0.5
2019 Ivan Klimenko 11.85.11.6-0.245514+0.3
2018 Ivan Klimenko 11.25.31.5+0.445511+0.7
2018 Dmitrii Dorofeev 12.65.12.1+0.645511+0.0
2016 Dmitrii Dorofeev 10.25.22.3-2.15169+1.0
2019 Nasko Stamenov 11.45.12.2-0.645513+0.4
2014 Nasko Stamenov 11.65.22.2-0.63044+0.5
2019 K. Turekhanova 12.46.01.6-0.654613+1.0
2018 K. Turekhanova 10.14.01.5-0.136410-0.1
2019 Aleks. Zubankov 12.65.12.0+0.245515+0.0
2018 Aleks. Zubankov 11.25.32.1-0.42738+0.7
2019 Nik. Karavasilev 10.64.42.4-1.045513+0.0
2018 Nik. Karavasilev 11.45.31.8-0.236410+0.7
2016 Nika Sabashvili 11.55.01.9-1.03048+0.3
2015 Nika Sabashvili 12.65.41.9-0.13347+0.3
2019 Yury Kartynnik 14.16.22.3+1.145513+0.4
2015 Yury Kartynnik 10.44.21.3-0.92738+0.0
2018 Mar. Yavahchova 10.94.71.6+0.645512+0.2
2017 Mar. Yavahchova 11.95.11.8+0.845514+0.2
2019 Gerg. Visarieva 12.04.21.5-0.345513-0.7
2018 Gerg. Visarieva 11.15.01.6-0.345512+0.5
2019 Oscar Rabinovich 13.45.21.4+0.52739-0.3
2015 Oscar Rabinovich 12.74.91.7-0.54867-0.3
2019 Sofia Anisimova 13.05.31.8-0.445513+0.0
2018 Sofia Anisimova 12.14.42.0+1.41825-0.6
2019 Li Peng 12.54.91.5-0.736411-0.2
2017 Li Peng 10.13.21.8-0.52739-0.9
2018 B. Tevdorashvili 10.46.51.9-0.21826+2.2
2016 N. Seliverstova 12.87.21.5+0.21224+2.0
2016 Nikita Datsuk 11.86.72.8-1.03348+1.9
2016 Som. Mahmoodi 12.67.11.3+0.83349+1.9
2019 Yulia Yuts 10.66.12.0-0.2913+1.8
2015 D. Radovanović 12.36.83.6-0.43958+1.8
2018 Eduard Stefanov 11.46.32.3+0.045512+1.6
2019 Z. Jelić Matošević 11.66.33.5-1.545511+1.6
2016 Jalil Sedaghat 11.76.43.1+0.22437+1.6
2015 Ivan Reznikov 11.76.43.3-1.14869+1.6
2019 Anton Golyshev 11.86.42.5+0.436411+1.6
2018 Andria Rogava 14.57.42.3+2.145511+1.5
2018 Giorgi Tsereteli 10.45.62.2-0.136410+1.4
2016 Samuel Byland 11.16.02.2-1.451610+1.4
2019 Darya Snitavets 15.47.62.4+1.72739+1.4
2013 Igor Evtodiev 10.15.42.0-1.44259+1.3
2018 Rich. Fitzpatrick 10.85.71.9-0.42738+1.3
2018 Niko Giorgadze 11.46.02.0-0.145511+1.3
2013 Alina Astakhova 11.86.12.2+0.157614+1.3
2019 Katerina Gotina 12.26.32.3-0.136412+1.3
2013 Naime Arslan 12.46.41.9+1.9913+1.3
2016 Ahmad Sheikhi 13.16.63.0+1.436410+1.3
2013 Ismail Kiran 11.76.01.8-0.14258+1.2
2016 Af. Montakhab 12.06.11.8-0.12136+1.2
2015 Aleks. Suvorova 14.27.03.0+0.84567+1.2
2016 Roya Radgohar 14.37.02.7+1.93048+1.2
2016 Azizolah Azizi 14.77.22.2+1.748610+1.2
2018 N. Tsimakuridze 10.75.51.9-0.12738+1.1
2018 Manon Geijsen 11.55.82.4+0.845511+1.1
2013 Celalettin Baykul 12.76.32.9+0.41825+1.1
2018 Z. Vardanashvili 13.46.52.0+1.054610+1.1
2013 Jeyhun Jabarov 14.67.12.5+0.739510+1.1
2019 Hanna Karpenka 15.87.52.7+2.054614+1.1
2016 Laura Guerrini 10.45.32.3-2.245510+1.0
2013 Jevhen Olijnyk 10.85.41.5-0.73958+1.0
2019 Valeria Burianova 11.05.52.1-0.32739+1.0
2018 Domagoj Gajski 11.35.62.1-0.136410+1.0
2018 Kirill Belyaev 11.55.71.5+0.545511+1.0
2018 Tsotne Dadiani 11.55.61.5-0.436411+1.0
2013 Ersin Karademir 11.85.81.6+0.33959+1.0
2019 Maria Karaveli 11.85.82.3-0.536410+1.0
2019 Konst. Vourlias 12.15.92.3+0.92736+1.0
2019 Ap. Michaloudis 12.36.01.9-0.936411+1.0
2016 Marzieh Afkhami 13.46.51.9+1.62436+1.0
2016 M. Sadat Tahami 13.56.52.7+1.142510+1.0
2019 Ivan Bosko 14.26.82.2+1.345512+1.0
2014 D. Karashanova 15.47.31.6+0.63345+1.0
2017 Gal. Onoprienko 10.25.12.7-0.545510+0.9
2018 Ekaterine Dadiani 11.65.72.0+0.82739+0.9
2017 Song Yi 11.65.61.7+0.836410+0.9
2013 Diana Kovtunova 13.06.21.2-0.142510+0.9
2013 Ahmet Çabuk 13.16.21.5+0.32739+0.9
2013 Antoan. Nikolova 13.36.31.9+0.742510+0.9
2015 Vesna Vasić 15.77.31.7+2.3913+0.9
2019 Daniil Mnevets 9.04.51.9-0.6913+0.8
2018 Vsev. Zhdanov 10.14.91.1-0.33649+0.8
2019 Nastassia Lishai 10.65.11.5+1.0913+0.8
2017 Luisa Schrempf 10.85.22.1-0.954613+0.8
2018 Daria Vetoshkina 11.55.52.0+0.72738+0.8
2017 Wang Lin 11.75.61.6+1.12739+0.8
2013 Aliaks. Mamoika 12.25.81.6-0.442510+0.8
2019 Denis Kolchanov 12.86.01.8+0.736411+0.8
2018 Ilia Lomidze 13.26.22.2+1.536410+0.8
2013 Vlad. Vanovskiy 13.26.22.2+0.054612+0.8
2019 Artem Glukharev 13.36.21.7+0.836412+0.8
2015 Dušan Dimić 14.96.92.2+1.04869+0.8
2019 Pavel Kviatko 15.47.12.1+1.345513+0.8
2018 Anton Khvalyuk 10.45.01.8-1.53649+0.7
2013 Buras Boljiev 11.45.41.7-0.33046+0.7
2018 Elena Zhurikova 12.15.71.6+0.645512+0.7
2019 Artemii Kolenko 12.25.72.6-0.154616+0.7
2017 Sergei Kozelkov 12.45.71.3+0.136411+0.7
2017 Liu Lisa 13.16.12.0+1.454614+0.7
2019 Anna Nikitina 13.26.12.0+0.445512+0.7
2016 Ban. Rastegari 13.96.41.9+1.32737+0.7
2017 Nurzada Beissen 10.95.01.5-0.13649+0.6
2019 Phyllis Barth 11.25.32.1-1.236411+0.7
2019 Zekai Wang 11.45.32.6-0.336411+0.6
2015 Jelena Vračević 11.85.42.0+0.52437+0.6
2017 Xiaobin Chen 12.65.72.0+1.336411+0.6
2015 Viktor Nechaev 13.36.01.9+0.12748+0.6
2016 Sed. Forootan 13.96.31.2+2.8913+0.6
2013 Ek. Mendeleeva 14.36.41.4+0.748511+0.6
2016 Som. Haj. Gooki 15.16.72.3+2.44559+0.6
2013 Timothy Timur 9.84.52.1+0.31525+0.5
2018 Grigol Peradze 10.44.82.9-0.52739+0.5
2017 Michelle De Kock 10.84.91.9-1.045511+0.5
2017 Chrisy Xiyu Du 10.84.92.3-0.345510+0.5
2017 Polina Deviatova 11.05.02.1-0.745513 +0.5
2016 Tatiana Fursova 11.25.22.2-0.62737+0.5
2019 Ek. Rosnovskaia 11.35.12.2-0.936410+0.5
2013 Alexander Sigeev 11.75.31.7-0.95469+0.5
2018 Diana Sokhashvili 11.85.31.6-0.245510+0.5
2013 Emel Alğin 11.95.42.0-0.357613+0.5
2013 Sergey Sabaev 12.05.41.7-0.1913+0.5
2019 Vasileios Nousis 12.05.42.7+0.63649+0.5
2016 Jaf. Vatanparast 12.35.61.6+1.13648+0.5
2013 Özge Özşen 12.45.62.0+0.236410+0.5
2019 Effimia Mavidou 12.65.62.4+0.32738+0.5
2019 Svet. Izraileva 12.75.71.7+0.336410+0.5
2013 Ayset Yurt Ece 13.05.82.6+0.73349+0.5
2019 Tatiana Lyalina 13.56.02.3+1.036410+0.5
2016 Afshan Mohajeri 14.16.22.8+1.42737+0.5
2019 Elena Koida 14.46.42.5+1.136411+0.5
2013 Siarhei Seniuk 11.25.01.6-0.842513+0.4
2019 M.-N. Parpalea 12.25.42.3+0.42738+0.4
2016 Roya Pournejati 12.55.51.5+0.233410+0.4
2013 Dursun Eser 12.85.61.2+0.636410+0.4
2016 Zahra Yazdgerdi 12.85.62.9-0.22437+0.4
2013 Louis D. Heyns 13.25.81.6-0.342512+0.4
2013 Hakan Dal 16.77.22.3+2.3612+0.4
2018 Y. M. Sefidkhani 10.14.42.4-1.236410+0.3
2018 Nat. Semenikhina 10.64.61.0+0.636410+0.3
2019 H. Chubarova 11.55.02.3-0.42739+0.3
2017 D. Permatasari 12.05.22.3+0.754614+0.3
2019 Al. Falchevskaya 12.45.32.9+1.12739+0.3
2019 Todor Todorov 12.65.41.6+0.345512+0.3
2013 Sergey Zelenin 12.85.51.5+0.14259+0.3
2015 Kirill Volosnikov 12.95.62.3-0.54569+0.3
2013 Ebru Ataşlar 14.96.42.3+1.73649+0.3
2014 Vasilka Krasteva 17.17.31.8+1.53954+0.3
2017 Ch.-Eung Ahn 9.94.22.5-1.654611+0.2
2017 Li Ying 10.24.41.5-0.71826+0.2
2018 Pavel Kostryukov 11.44.81.8+0.23648+0.2
2015 Tatiana Besedina 12.35.21.9-0.53957+0.2
2017 Qu Yanfu 13.15.61.7+0.736411+0.2
2015 Elena Chernova 13.45.71.5+0.93648+0.2
2019 Kats. Korsak 13.85.91.1+2.1913+0.2
2019 Aksana Hubich 14.05.91.9+0.71826+0.2
2019 Bauyr. Smailov 14.36.11.4+0.61826+0.2
2016 Maryam Bahrami 14.46.12.1+1.845510+0.2
2016 Mas. Tor. Azad 13.35.62.2-0.151612+0.1
2016 Milad Zangiabadi 13.35.52.0+0.62437+0.1
2019 C.-F. Tanasescu 10.44.31.8-0.12739+0.0
2016 M. R. Moghadam 11.24.61.2+0.41524+0.0
2013 Fatih Akay 12.85.21.7-0.342510+0.0
2016 M. D. Aseman 13.25.41.7+1.12738+0.0
2013 Vladimir Shiltsev 13.45.52.6-0.51214+0.0
2015 Drag. Jovković 15.46.31.9+2.12747+0.0
2017 T. Gachechiladze 10.64.22.2-0.136410-0.1
2018 Ramaz Khomeriki 11.24.51.5+1.1913-0.1
2017 Bi Jun 12.04.81.4+0.83649-0.1
2013 Marina Sergeeva 13.55.41.5+0.351612-0.1
2016 Hassan Eslahi 13.65.41.8+0.83649-0.1
2018 Saba Kharabadze 13.65.41.5+0.8913-0.1
2014 Elena Trufanova 15.36.12.0+0.53954-0.1
2019 Vera Somova 17.37.01.0+1.8913-0.1
2017 Chen Xi 10.23.91.9-0.136411-0.2
2017 Cao Xuewei 10.64.11.7+0.52738-0.2
2018 Sergei Yudin 11.34.42.0+0.21826-0.2
2019 Sofiya Guzik 13.05.21.9+0.22739-0.2
2015 Lucija Papa 13.15.11.3+0.12135-0.2
2013 Pınar Aytar 13.55.31.3+0.230410-0.2
2017 Wang Jin 13.85.41.9+1.654614-0.2
2016 A. Poostforush 14.85.92.0+2.13957-0.2
2017 Thomas Broger 9.83.71.5-1.31826-0.3
2017 Li Bin 11.14.32.0-0.42738-0.3
2019 T. Rtveliashvili 12.14.652.0-0.82739-0.3
2019 Artsiom Bury 10.13.72.1-0.2913-0.4
2017 Song Feng 11.84.41.3+0.32739-0.4
2013 Natalia Borodina 12.34.61.1+0.03048-0.4
2017 Li Hong 12.54.61.5+0.736411-0.5
2013 Necmettin Caner 12.84.71.2-0.3913-0.5
2019 Hanna Bareika 14.45.31.6+2.6913-0.6
2013 Sertaç Eroğlu 11.73.91.9+0.51525-0.9
2013 Sabahattin Esen 15.04.81.1+1.6612-1.3
---Average12.25.62.0+0.13649+0.6
---Best records10.07.61.0+0.057616+2.3

  • Color-coded status tags reflect various roles of the IYNT Juror: violet tag is the Juror who acted as Chairperson while green tag is the Juror who did not act as Chairperson; blue tag is the independent Juror while red tag is the Team Leader who acted as Juror;
  • G⟩ is arithmetic mean of all Grades delivered during the IYNT;G⟩=(5+10+15)/3=10 is our target for any Juror to allow for uniform and equal weight grading scales throughout the parallel Groups; a majority of Jurors go above the taget;
  • σG is standard deviation of all Grades delivered during the IYNT; if σG is large, the Juror uses a broader spectrum of Grades and has more differentiation within their evaluation scale; note that σG does not necessarily reflect clearer separation of the Teams, cf. two notable hypothetical extremes of σG=13.7 for the set {30; 1; 1; 30; 1; 1} and σG=11.1 for the set {30; 1; 1; 1; 20; 10} in a two-Team SF; various limits for three types of performances result in a trend that a higher σG is more likely to appear for Jurors with a higher ⟨G⟩; theoretical slope of the trend, or baseline, is σG/⟨G⟩ for the set {30; 20; 10} or 0.40825; parameter κ corrects for this baseline;
  • σG−P is standard deviation of all residuals G−P; if σG−P is small, the Grades are less scattered respective to the Grades of other Jurors and contribute to a smaller ρ; an implausible theoretical minimum is σG−P=0; real-life maximum and minimum records are 3.7 and 1.0;
  • G−P⟩ is arithmetic mean of all residuals G−P; if ⟨G−P⟩ is close to zero, the grading scale is less shifted respective to the individual scales of other Jurors; it is easy to notice moderate statistical noise that hinders the inherent correlation between ⟨G−P⟩ and ⟨G⟩;
  • nG is the number of Grades given; greater nG means better statistics; theoretical caps depend on exact tournament brackets and were nG=60 in 2013, nG=45 in 2014, and nG=54 in 2015...2019, though often not accessible even for the Jurors working in each Science Fight due to such constraints as distribution of two-Team Groups;
  • nSF is the number of Science Fights judged; greater nSF means better statistics; a theoretical cap with Semi-Finals is nSF=6;
  • nT is the number of Teams judged; greater nT means more opportunities to observe stronger and weaker Teams and thus have a more comparative judgment; a theoretical cap is number of Teams N but no more than 18; note that some Teams are judged more than once by the same Juror within one IYNT;
  • κ is standard deviation of all delivered Grades corrected for the average Grade ⟨G⟩ via κ=σG−0.40825×⟨G⟩; κ reflects relative width of the spectrum of Grades used by the Juror and can be more suitable for comparison of Jurors with distinctly different ⟨G⟩; note that κ is linearly proportional to the relative value of σG/⟨G⟩.

The names are initially sorted by number of IYNTs judged, then by κ, then by ⟨G⟩. Click on the headers to have the table sorted by any desired parameter.

There is a complex interplay between each of the calculated parameters, and some of the crucial parameters depend not only on what G each Juror gives, but also on what G other Jurors in the same Group give. Other parameters depend on the lot, tournament brackets, or appointing decisions of the General Council, and are beyond control of the individual Juror. Persisting regularities seen for Jurors who worked at more that one IYNT suggest that any shifts in σG or ⟨G⟩, observed consistently in several Jurors, may reflect objective differences between separate IYNTs, viz. in diversity or average strength of participants.

Particular grading preferences for a Juror are sometimes clearly recognizable in separate IYNTs and not obscured by limited statistics. These differences in particular explain why Jurors and Teams rotate between the Groups, and V is a more representative derivative grading parameter than SP.

Whilst the Criterion of Victory V already alleviates scaling differences, it would allow for extracting further fine-grained data if each future IYNT Juror

  • is comfortable with lower Grades for weaker performances, and therefore stays centered closer to ⟨G⟩=10, with preferably ⟨G⟩<13;
  • at the same time, works in a broader spectrum of high and low Grades, and thus has a larger σG, with preferably σG>6;
  • at the same time, is balanced to have a moderate σG−P, with preferably σG−P<3.

These three goals can naturally clash with each other. At this point it is important to realize that each Juror must be focused only on assessing immediate performances and sticking to uniform, scientific, merit-based grading criteria, and that furthermore each G must be given independently.



Composition of Jury

YearInt'lLocalTLsTotalChairsExperienced% Experienced
201391517411100%
2014435125433%
2015105122710726%
201610219409923%
201720101848161429%
201825122057122137%
201919172662102337%
All97831072877378---


Juror retention and Zipf's law

nIYNTs2013201420152016201720182019
141456186115141166
2---451091425
3------327911
4---------2132
5------------212
6---------------21
7------------------2
Σ414969100134170209


Effects of individual grading parameters

To illustrate the potential consequences of the spread in these grading parameters, let us consider a Gedankenexperiment with three Teams competing in one Science Fight.

Team 1 shows a relatively strong performance and receives the Grades which sit on the upper end of the [⟨G⟩−½σG…⟨G⟩+½σG] interval. In other words, should G be distributed normally for each selected Juror, the performances of Team 1 would be better than Ф(⟨G⟩+½σG)=0.69 of all performances the Juror grades in the IYNT. Such a Team would potentially end up as a Finalist.

Team 2 shows an average performance and receives average Grades ⟨G⟩ from each Juror.

Team 3 shows a relatively weak performance and receives the Grades which sit on the lower end of the [⟨G⟩−½σG…⟨G⟩+½σG] interval. In other words, their performances are weaker than very approximately Ф=0.69 of all IYNT performances graded by the Juror. Such a Team would potentially not qualify for Semi-Finals.

These three Teams are graded simultaneously by two boards of selected Jurors. One board is composed of six Jurors with some of the lowest observed ⟨G⟩, while the other board is composed of six Jurors with some of the highest observed ⟨G⟩. It is easy to determine the results of this hypothetical Science Fight because ⟨G⟩ and σG are publicly known for each Juror.



These results illustrate the level of tolerance of the Criterion of Victory V and Sum of Points SP to the most severe effects of improbably unbalanced boards of Jurors. As seen from this calculation, the strong Team 1 graded by low-⟨G⟩ Jurors obtains less points than the average Team 2 graded by high-⟨G⟩ Jurors and ties with the weak Team 3. The weak Team 3 graded by high-⟨G⟩ Jurors, respectively, earns more points than the average Team 2 graded by low-⟨G⟩ Jurors. An artificial selection of Jurors in this test leads to unrealistically small σg−P and ρSP in both boards of Jurors.

There is however no negative effect on the Criterion of Victory V and consequently the results of the Science Fight.

In the next Gedankenexperiment, let us rotate and evenly distribute the same Jurors as routinely made before each real Science Fight.



Though the Jurors give the same Grades G as in the first experiment, their balanced distribution now mitigates the effects on SP of Juror-to-Juror differences in grading. Note that this happens at the cost of increased σg−P and ρSP, which both now fall in the range of typical IYNT values despite an articifial, bimodal distribution of ⟨G⟩. Although we cannot generalize from one example, the respective Sums of Points SP from both boards of Jurors now differ by only 1.5, 0.6, and 0.6 points.

In this extreme value analysis, we test a statistically improbable scenario which would have some of the worst impacts on the stability of Science Fight results. We test extreme values of ⟨G⟩ and the most unrealistic distribution of Jurors, and observe the amplitude of fluctuations in SP which always fall within a 2 points threshold.


Grading parameters of separate IYNTs

The table below provides an overview of statistical parameters for the Grades given by all Jurors within one IYNT, and the overall statistics for all previous IYNTs.

YearGσGσJσG−PnGnstnTnJnchV=1V
201312.55.81.332.014227816411196±399.98±0.03
201414.16.72.022.242323512595±399.99±0.01
201513.06.01.242.49694910271094±399.99±0.02
201612.36.21.602.81284691640991±599.94±0.08
201711.45.41.332.319448418481696±2100.0±0.00
201811.35.71.362.321157215571297±2100.0±0.00
201912.45.91.562.421339621621095±399.99±0.01
All12.15.91.612.3102904711012093795±499.98±0.04

  • G⟩ is arithmetic mean of all Grades delivered during the IYNT by all Jurors;
  • σG is standard deviation of all Grades delivered during the IYNT by all Jurors;
  • σJ=σGJ is standard deviation of all averages ⟨G⟩ for all Jurors; lower values correspond to less Juror-to-Juror scaling differences;
  • σG−P is standard deviation of all residuals G−P during the IYNT;
  • nG is the number of Grades given;
  • nst is the number of Stages in the IYNT;
  • nT is the number of Teams in the IYNT;
  • nJ is the number of individual Jurors in the IYNT (totals for unique Jurors);
  • nch is the number of individual Jury Chairpersons in the IYNT (totals for unique Chairpersons);
  • V=1 is the mean and standard deviation of the confidence levels for the interval [SP−2...60] in all SFs, in %;
  • V=½ is the mean and standard deviation of the confidence levels for the interval [SP−10...60] in all SFs, in %.


Grading parameters of separate SFs

Teams and Jurors alike rapidly learn from SF to SF. One may argue that the Teams may start showing less diverge performances, or the Jurors may start giving less diverge Grades G.

Confidence levels for the interval [SP−2...60] in each SF, as well as distributions of G, G−P, and SP, are of interest to assess the importance of these effects.



Summary

Overall, the presented results define the extent to which the results of single paired comparisons of SP are not yet obscured by statistical noise. With the available data, we can conclude that the IYNT procedures and in particular the Criterion of Victory V alleviate Group-to-Group and Juror-to-Juror scaling differences, and allow separation of each Team in the IYNT with a well-defined significance threshold.

The Criterion of Victory prevents a melting pot effect in TSP where Total Sums of Points can converge to rather similar values for many Teams.



Comparative results of real IYNT Teams

Click on the headers to have the table sorted by any desired parameter. The Teams are initially sorted by Criterion of Victory in the Finals (VF), then by Sum of Points in the Finals (SPF), then by Criterion of Victory in the Semi-Finals (VsF), then by Sum of Points in the Semi-Finals (SPsF), then by Sum of Victories after Selective SF 4 (SV4), then by Total Sum of Points after Selective SF 4 (TSP4). Final Rank (RF) and the type of Medal (M) reflect the results of each IYNT according to the regulations valid at the time.

Year Team name SV4 TSP4 VsF SPsF VF SPF RF M
2013Belarus-Universum4177.1143.3150.81
2014Georgia "Georgians"3190.6150.51
2014Bulgaria-Sofia4206.1149.32
2019New Zealand4188.5145.1147.91
2018Switzerland4174.2147.8147.31
2016Georgia-Georgians4172.4145.0146.71
2019Croatia4169.0142.7145.92
2017New Zealand-Wellington155.4142.8145.81
2015China3163.0140.6145.31
2015Georgia-Georgians165.4141.4145.12
2017Switzerland4177.8146.7144.62
2015Croatia175.8140.7144.23
2014Serbia184.3½46.53
2019Switzerland4194.0145.5½45.63
2018Bulgaria-Sofia3162.5143.9½43.42
2013Georgia-Raveko3174.4145.1½42.82
2016Belarus-Pahonia4165.1142.5½42.42
2016Croatia4173.5140.1½41.13
2018Georgia-Komarovi138.6135.4½38.53
2013Turkey-Bahçeşehir PES4173.8½38.3037.23
2013Moldova-Eco Generation154.6141.7036.44
2017China-NFLS LaplaceW/169.2137.1034.13
2018Croatia3159.0146.55
2019Bulgaria-Sofia A3175.8145.55
2013Russia-TMOLimpiycy167.5140.07
2016Iran-Gifted133.3138.84
2018Georgia-National2125.2136.78
2019Georgia165.8½43.46
2018New Zealand175.1½40.54
2013Bulgaria-Bulgaria3170.0½39.16
2015Serbia-1174.3½39.04
2013Russia-MG 123160.1½39.08
2018Belarus-Pahonia3155.3½38.77
2019Romania-Sol/ Sq/4173.2½37.64
2017Georgia-Georgians4153.6½37.54
2016Iran-Khashayar2115.5½37.48
2013Russia-RLC3173.0½37.15
2017Croatia3153.8½36.56
2015Serbia-2154.4½36.06
2013Afghanistan-Ariana3155.5½35.89
2016Iran-Mehr155.6½35.75
2018Russia-12 FM150.4½34.66
2017Bulgaria3139.9½33.97
2019Greece-Pinewood3149.6½33.48
2015Belarus-Spectrum3174.4½32.95
2016Iran-Maple145.3½31.47
2017China-Beijing RDFZ157.0½28.85
2017Indonesia-Labsky3155.5034.08
2013Bulgaria-Science Girls3153.4032.510
2017China-NFLS Unique3154.2032.39
2019China-PML3160.8031.49
2016Russia-Voronezh3147.9030.46
2016Iran-Besat 1113.0030.29
2019Greece-Anatolia C/168.3023.47
2018Iran-AYIMI 23127.4018.79
2014Russia "Vinegret"146.44
2014Bulgaria-Kyustendil "R/"½113.45
2016Iran-Black Intelligence3161.910
2019Belarus-Ecliptic148.210
2013Turkey-Fatih Eskişehir146.911
2017China-Qingdao No. 2143.610
2019Russia-Element143.111
2015Bulgaria140.97
2017Belarus-Pahonia133.511
2013Kazakhstan-NIS D/team130.315
2015Russia-Voronezh-12161.68
2019Bulgaria-Penultimate2160.912
2019Bulgaria-B2148.413
2015Russia-Voronezh-22143.39
2017Russia-Novosib/ 12FM2126.212
2017China-Shenzhen 2 M/S2121.313
2019Greece-Anatolia HS154.314
2013Iran-Iran145.912
2019Kazakhstan-RFMS139.215
2019Kazakhstan-1137.916
2017Russia-Voron/ Izolenta130.314
2017Russia-Dolgoprudny 5th129.715
2017China-Shenzhen 1 M/S119.816
2016Iran-Besat 2116.311
2013Kyrgyzstan-Alatoo1137.613
2015Russia-MG 121137.410
2013Ukraine-Richelieu1137.514
2019Russia-Moonlight1133.717
2018Russia-12 kids of sci/1113.610
2017Iran-AYIMI1112.417
2016Iran-Free Thought188.512
2019Greece-A.F.S.½129.418
2019Greece-E.F.I.½128.919
2019China-CSLD½118.520
2016Iran-Amordad½118.013
2018Russia-Voronezh Kvant/½114.611
2018Iran-AYIMI 1½114.112
2016Iran-Paramount Notion½106.414
2019Russia-13 Element½102.321
2016Iran-Farhang½85.215
2016Iran-Velayat½76.216
2018Russia-Easy Science0113.013
2013Turkey-Samanyolu0107.916
2018Kazakhstan095.614
2018Bulgaria-Awkw/ Turtles091.015
2017Kazakhstan-Bobek.kz068.318