Test problems from past UKLO competitions

• Explanation
• ‘Relative difficulty’
• The problems

=============================================

Explanation

This page gives you a spreadsheet listing all the test problems that have been used in UK Olympiads since 2010; each problem’s name includes a link to the downloadable problem. The spreadsheet is not locked, so once you’ve saved it locally, you can easily search it, edit it or re-sort the data.  The same problems are also listed below.

Each problem is listed with the following information:

• diff: its relative difficulty, as explained below
• year: the year in which it was used in UKLO
• number, name: its number in that year’s problem set, and its name
  • this cell is hyperlinked to a file containing the problem concerned, a separate answer sheet (where relevant), and the correct answers, plus a mark scheme and detailed directions for marking (as used by the original markers).
• language: the language(s) from which its data are taken
• extras?: any information beyond the problem and its solution.
  • expanded: an extended essay-length discussion of the problem and how to solve it, as well as ideas for building on it in discussing more general ideas about language and linguistic analysis.
  • notes: brief notes on the problem data, possibly as a separate file from the problem file.
  • problem, solution, commentary and marking scheme: a single file containing all relevant information.

Relative difficulty

The problems are also classified for difficulty. Here’s how this was calculated:

• For problems from 2012 and later, the grading is objective and reflects the average marks of hundreds of competitors. Informally, the difficulty is the difference between the maximum possible mark and the average mark; so if everyone scores 100%, the difficulty is 1, but if the average is just 10%, the difficulty is 10. But as you’ll see below, it has to be more complicated than that!
  • Mathematically, the difficulty (diff) of a problem is the maximum score for that problem divided by the average score;
    • e.g. if the maximum was 20 and the average was 10, diff would be 2.0.
    • If everyone achieved the maximum, diff would be 1.0.
    • If the average was very low, say 1/20, diff would be 20.
  • However, difficulty depends not only on the problem but also on the competitor, so a problem which would be very hard for a Foundation competitor might be quite easy for an Advanced one. The figure for diff is therefore standardised for a typical Advanced competitor:
    • Where a problem was part of the Advanced competition in Round 1, diff is calculated as above.
    • Problems which were only included at lower levels (Breakthrough, Foundation or Intermediate) had their diff reduced by a figure based on the problems which were taken both at these levels and at higher levels, including Advance levels.
      • E.g. if an I-level problem’s average score was 5/10, giving an I-level diff of 10/5 = 2,
      • and the same competitors scored half (0.5) as well as the Advanced competitors did on the problems that they both did, giving an ‘I-adjustment’ for I and A (the mean I and A scores for those problems) of I/A = 0.5
      • then its diff (for a typical A-level candidate) was 2 x 0.5 = 1.
      • Similarly for lower levels: so the diff of a Breakthrough problem is given by B x F x I where
        • B = the diff for Breakthrough candidates
        • F is the F-adjustment figure (F/I) for problems taken at both F and I level
        • I is the I-adjustment figure (I/A) for problems taken at both I and A level.
      • This standardization calculation explains why some diff figures are below 1.0 (which would otherwise be the lowest possible).
    • Problems from Round 2 had their diff increased by a figure based on the difference between the Round 1 figures for the winners (who eventually took Round 2) and the overall average.
      • E.g. if the R2 diff for a problem was 2,
      • and the R1 Advanced winners’ average score was twice the average for all Advanced competitors,
      • then its diff for a typical A-level candidate was 2 x 2 = 4.
• For problems from 2010 and 2011, unfortunately we have no data on scores for individual problems, so diff is copied from the grading in the two books of problems. This five-point grading is partly based on test scores in the American competition and partly estimated by test-setters; unfortunately there are no problems graded by both systems, so they are hard to match. For that reason, the American gradings are distinguished from the others by *.
• The diff rating for the 2011 problem 2, ‘Japaneasy’, is a guess because it is not covered by either of the above systems. It’s highly recommended as a starting point for exploring the problems!

The problems