09 Sec. 10 Annex 4 Slalom Scoring

Proposal from

Barney Townsend (GBR)

Proposal title

09 - Sec 10 A4 Slalom Scoring Formulae

Existing text

S10 A4 3.C5
PRECISION CIRCUIT IN THE SHORTEST TIME (‘Clover leaf slalom’)
Scoring

tpen: = t_pil + m * v_pen

Q: = Ln(3 * t_best / (t_pen – t_best + 3))

Where

tpil = the measured pilots time (seconds)
m = the number of missed targets
vpen = the time penalty for each missed target (seconds)
tpen = the pilots time (after penalties for missed targets)
tbest = the best time (after penalties for missed targets)
Q = the task value before normalization

Note: Spreadsheet formulas:
tpen: = _t_pil + m * v_pen_
Q: = LOG(3 * t_best / (t_pen – t_best +3))

And same in S10 A4 3.C6, S10 A4 3.C7, S10 A4 3.C9, S10 A4 3.C10

New text

Reason

Summary of Reasons:

1. Pilot safety is being put deliberately at risk in order to provide an exciting spectacle for the few spectators who may attend this amateur competition by deliberately incentivising risk-taking by competitors. Our proposal removes the incentive to take risks, introduced in 2009.
2. The current formula denies points to pilots who successfully complete the course later than some arbitrary multiple of the fastest time. All Pilots who complete a task should receive points. Our proposal ensures all pilots who complete the slalom course without incurring penalties will receive some points.
3. It is the fundamental nature of the classic paramotor competition that it should be composed of many tasks. And that these tasks be diverse in nature and that a parity of scoring be applied across all that diversity. The current formula for scoring slalom is unique in the way it deliberately distorts the points distribution and so does not match the principle under which all other tasks in precision, navigation and economy are scored. The proposal remedies this distortion of the slalom task scores.

Detail of Reasons: 

Reason 1:

In the interests of Pilot Safety

The reason for bringing in the log based formula in 2009 was given as:

At WPC 2009 we discovered a fundamental flaw in current slalom scoring when there is a small number of competitors in class.

“This formula generates an asymptotic curve which:

a) Encourages pilots to fly for the fastest time rather than be conservative; not excessive risk to miss a stick.

b) Works equally well with a large class or a small class.

For a full explanation see Option 6 in the attachment slalom_scoring_options.xls ”

The proposed formula is also asymptotic, as can be seen from the graph of its Task Score distribution (figure 1 below) The proposed formula is not “linear” as has been claimed. The asymptote simply is removed from the fastest time and placed at zero. This is a reasonable value for the asymptote because a zero time is the logical place for an infinite score. The current formula places the asymptote one second faster than the fastest score. This is a deliberate distortion put in place to disproportionately reward marginally better times with exaggerated points to encourage life-threatening risk-taking. The 2009 formula gives an infinite score at one second faster than the fastest time. Why one second? Why not two seconds or half a second? The choice of asymptote is completely arbitrary. It was placed at one seconds faster than the fastest time to give an exaggerated score to the fastest pilots as an incentive to take risks. It was modified to three seconds in 2010 without any further justification.

The supporting philosophy given by the Chairman of CIMA can be summarised as being:
“If a linear scoring system is applied to a task which is part of a competition composed of many tasks, then the reward does NOT increase with the risk, pilots are therefore incentivised to fly conservatively.”

It is the fundamental nature of the classic competition that it should be composed of “many tasks”. And that these tasks be diverse in nature and that a parity of scoring be applied across all that diversity.

We propose that the organisers' desire for pilots to take risks in tasks flown, in close proximity to the ground, in order to maximise their advantage under the log based formula, is now in direct conflict with the imperative to encourage safe flying and with parity of scoring across all diverse tasks..

The current scoring formula was introduced in 2010, brought as a proposal by ESP. This merely changed the arbitrary constant +1 to a new arbitrary constant +3.  The current formula remains deliberately constructed to encourage risk-taking by pilots flying in close proximity to the ground. This encouragement of risk taking is leading to and has lead to serious injury and fatality to pilots engaged in performing and practising for this task.

This proposal is intended to reduce the incentive to take such risks and at the same time return parity of points distribution across all diverse tasks in a competition.

One supporting argument to introduce the Log based formula for deriving Q was given as

“there was a flaw discovered in the existing formula when there is a small number of competitors.”

No explanation was given of what that flaw was and this assertion was not challenged at CIMA sub-committee or plenary. The formula in this proposal works equally well for any number of competitors and is consistent with the scoring principles of all other tasks.

A second supporting argument for the introduction of the current formula was to introduce an incentive for pilots to take risks. This is a dangerous stance for competition organisers to adopt in the light of recent serious injuries and fatalities to pilots engaged in competing in and practising for this task. It should be at the discretion of pilots whether to fly conservatively in any task and points should not be awarded to encourage risk taking particularly in flight in close proximity to the ground.

A third argument that was given was that the current formula

“Encourages pilots to fly for the fastest time rather than be conservative; not excessive risk to miss a stick.”

Now that most slalom tasks do not involve kicking sticks but use timing gates and pylons, the element of precision of striking a stick is no longer present to moderate the risk-taking incentive. For this reason alone the current formula (and its entire supporting philosophy) is flawed and a new system must be implemented for pilot safety. The proposed formula allows the pilot to determine the level of risk to take without undue reward for high risk-taking but gives proportionate reward for faster times.

This policy of the CIMA committee of deliberate encouragement to pilots to take risks is justified by its author as follows:

“For reasoning, best to look back at the proposals which introduced them; it has changed several times, but the basic philosophy is as follows: 

After watching a few people doing a slalom almost anyone can tell the difference between a ‘hot-shot’ performance and a ‘mediocre’ one. The original scoring (up to about 2005) was linear, but it was becoming rather clear that the difference in ‘risk’ between flying a clover-leaf in, say, 45 seconds and 46 seconds was not reflected in the scoring. People (especially team leaders) were looking at their global score and deciding that there was nearly no loss, and much to gain, by flying these tasks relatively conservatively. Since these tasks are interesting to spectators more than any other, then you can say they are the single most important element in any effort to expose our activities to a wider audience. This is considered a desirable objective for our sport in general, so it is important to have a scoring system which encourages ‘hot-shot’ performances. 

And that comes down to somehow measuring, or simulating that ‘risk’ I already mentioned, and inserting it in the scoring so it can be advantageous for pilots to attempt ‘hot-shot’ performances, and various complicated ways of doing this have been tried.

My argument is not whether a particular mathematical formula is fair or not, or is an accurate replication of that risk thing, but much more simply: the sheer complication and obscurity of it is self-defeating because no spectator understands it, which is the original purpose of it. I then go on to argue that the whole way of traditionally scoring tasks is completely spectator unfriendly, so I produced an alternative look at the whole way of doing it in the form of the ABG rules which de-couples a ‘task score’ (which is what a spectator is interested in) from the global score, and puts the incentive in the latter in a very simplified form which even the thickest team leader can understand. And actually it works quite well.”

Richard

We propose that seeking to make slalom into a “spectacle” by deliberately encouraging risk-taking is using the competition for the wrong purpose. Competitions attract few spectators anyway and, as an amateur competition, pilots are entering for their own entertainment and not to excite and titillate spectators.

Slalom is now a discipline in its own right with specialised equipment and dedicated competitions, which themselves do not even use log-based scoring formulae. There is therefore now even less reason to promote slalom above navigation or economy in classic competition. To do so distorts the competition and encourages the designers of wings to place more emphasis on slalom favouring designs for general purpose wings. This distorts the wing designs that general pilots will be offered and takes the encouragement of risk taking beyond competition to the general sports pilot.

We propose that this entire philosophy is completely wrong and damaging to classic competition and the sport.

Reason 2:

In the interests of equity in participation for all competitors in all three elements of classic competition; Navigation, Economy and Precision.

In 2009 the formula  Q: = LOG(3 * t_best / (t_pen – t_best +1)) was introduced.

This formula denies a proportion of pilots any score at all even if they complete the course with no penalties.

In 2010 ESP proposed an amendment to the 2009 formula to change the constant to +3. This formula narrows the advantage to the top few pilots by a bit less than half but still gives zero points to pilots whose times are slower than an arbitrary multiple of the fastest time.

The reasons given by ESP for the introduction of the +3 constant were:

This scoring formula was introduced in 2009 but, unfortunately, there has been no opportunity to use it in any international championship.

During the last Nationals in Spain the formula was applied and a long discussion followed. The conclusions were:

• ·Replace +1 by +3 in the formula
• ·The recommended penalty for missing a target (Vpen) is 5 seconds

A typographical error in the spreadsheet formula is also corrected.

No detail was given of that discussion, held by a national team of pilots, only the conclusion that +1 be replaced by +3 was given. It is not appropriate to introduce changes to a rule merely because one nation's pilots want it. A full explanation of the imagined benefits and discussion of the disadvantages should be had before an arbitrary constant is altered in an already arbitrary function. No explanation has ever been proposed that shows the correlation between the risk taken and the points awarded. No study has ever been undertaken to demonstrate any such correlation exists. Indeed the original proposer of this scoring system states:

“My argument is not whether a particular mathematical formula is fair or not, or is an accurate replication of that risk thing, but much more simply: the sheer complication and obscurity of it is self-defeating because no spectator understands it, which is the original purpose of it.”

Richard Meredith-Hardy
acknowledging the arbitrary nature of the formula and the lack of any connection with the reality of risk!

It is not equitable to deny points to pilots who have successfully completed a task with no penalties. If there is to be a cut off point for points distribution it should be a time limit within which a task must be completed not an arbitrary constant in the scoring formula. A time limit will depend on the length of the course and should be set for each course by the meet director, it cannot be set in the section 10 rules and remain fair where courses are of variable length.

The use of an arbitrary constant in this formula is unjust. Pilots whose skill or equipment is more tailored to Navigation or Economy or other Precision tasks may be denied scores in this single Precision element even if they complete the course, whereas pilots whose skill and equipment is more suited to slalom are never denied points in the Navigation or Economy or other Precision task scoring provided they complete those tasks. This situation is unjust.

Below is a graph derived from the spreadsheet, (linked to above), created by the proposer of the current system of scoring in 2009. To it has been added the formula of this proposal to demonstrate the inequity of the arbitrary application of this logarithmic function to the scoring of this task. 

The modified spreadsheet can be found here. slalom_scoring_optionsFBR2.xls

http://www.jfdiuk.com/pilotNotes/slalom_scoring_optionsFBR2.xls
Figure 1.
The graph shows that the log based formulae, which uses an arbitrary asymptote value close to the the fastest time, has a similar curve to the proposed Q=Tbest/Tpen The proposed formula also has provides an increasing gap between points for the pilots as times decrease.

The proposed improvements to the scoring formula are:

1. the spread of the points across the field of competitors is consistent with all other task scoring.
2. the reward of points is given to all pilots who successfully complete the task.
3. the asymptote is placed in its logical (real-world) location at “infinite points for zero time”.
   

Reason 3:

In the interests of parity of point distribution across all three elements of the competition.

Section 10, 4.29.3 states that

"Tasks shall, as far as practicable, conform to the following guidelines in standard championships:
For Paramotor aircraft classes PF and PL:
A) Navigation: 33% of the total value of the tasks flown.
B) Economy: 33% of the total value of the tasks flown.
C) Precision: 33% of the total value of the tasks flown."

This rule is correctly in place to ensure complete fairness to all pilots across the range of equipment choice and skill level and to encourage the development of good “all-round” wing and motor designs. A balance of skill in differing flying situations and a balance of capabilities of wings and motors, speed range, agility, weight, fuel efficiency etc. A range of conflicting requirements balanced to produce the best general purpose flying machine.

The current scoring formula for slalom tasks is too punitive to all but the top few pilots, giving them a massive disadvantage in the overall competition rankings. We claim this is in direct contravention of the spirit of rule 4.29.3. In order to be consistent in the method of spread of points we are proposing that the task is scored in fair proportion to the times recorded as it is with all other tasks which are scored using this principle. In the overall competition score Slalom scoring should not contain special advantages to pilots who do better at slalom flying than other Precision tasks, Navigation tasks or Economy tasks.

A number of different formulae have been tried , but none have been ideal, and the current one is directly unfair. We therefore propose a formula that is demonstrably fairer to all pilots.

The best or simplest way to compare the scores distribution  between two different types of task, e.g. navigation and slalom, is graphically, by plotting the scores against pilot ranking in the task.

The following graph (Figure 2)  shows the current log-based formula scores from the Japanese Slalom, task 5 at the WPC2014, directly compared against what the pilots would have scored if our proposed Q=Tmin/Tpen formula had been used instead.

It also shows the distribution of points by pilot ranking for a pure navigation task - in this case the turn-point hunt from task 3 in WPC2014. It can be clearly seen that our proposed formula removes the disproportionate advantage that the top few slalom pilots gained under the current log based formula, and makes the distribution of slalom points comparable to those that can be gained from other precision and navigation tasks.