Test problems from past UKLO competitions

• Explanation
• ‘Relative difficulty’
• The problems

=============================================

Explanation

Below you will find a list of all the test problems that have been used in UK Olympiads since 2010. You can also download the same list as an Excel spreadsheet; this is not locked, so once you’ve saved it locally, you can easily search it, edit it or re-sort the data. In both lists, each problem’s name has a link to the downloadable problem.  

Each problem is listed with the following information:

• diff: its relative difficulty, as explained below
• (Note: for problems set in 2010 and 2011 we have to rely on an informal grading in the two books of problems. This five-point grading is partly based on test scores in the American competition and partly estimated by test-setters; unfortunately there are no problems graded by both systems, so they are hard to match. The Japanese problem from 2011 isn’t included in these books so its grading is just a guess.)
• year: the year in which it was used in UKLO
• number, name: its number in that year’s problem set, and its name
  • this cell is hyperlinked to a file containing the problem concerned, a separate answer sheet (where relevant), and the correct answers, plus a mark scheme and detailed directions for marking (as used by the original markers).
• language: the language(s) from which its data are taken
• extras?: any information beyond the problem and its solution.
  • expanded: an extended essay-length discussion of the problem and how to solve it, as well as ideas for building on it in discussing more general ideas about language and linguistic analysis.
  • notes: brief notes on the problem data, possibly as a separate file from the problem file.
  • problem, solution, commentary and marking scheme: a single file containing all relevant information.

The problems

The problems are listed below in increasing order of difficulty.Note that:

• the table is divided into a number of pages which you can step through by pressing the button on the bottom right, but you can also choose how many rows to display in each page.
• in spite of this division into pages, you can reorder the rows by clicking on the column headers; e.g. instead of the default ordering by difficulty you can order by year, starting either with the oldest or the youngest.
• there’s a useful search box in the top right of the table, so (for instance) you can search for all the problems based on English by typing “English” into that box.

The problems are also listed in a downloadable spreadsheet which you can edit in Excel like any other spreadsheet. Mroeover, unlike the list below, this spreadsheet also includes clickable links to the downloadable problem files, so if you want to download the files, you need the spreadsheet.

The table of problems

Relative difficulty

The problems are classified for difficulty. Here’s how this was calculated:

• For problems from 2012 and later, the grading is objective and reflects the average marks of hundreds of competitors. Informally, the difficulty is the difference between the maximum possible mark and the average mark; so if everyone scores 100%, the difficulty is 1, but if the average is just 10%, the difficulty is 10. But as you’ll see below, it has to be more complicated than that!
  • Mathematically, the difficulty (diff) of a problem is the maximum score for that problem divided by the average score;
    • e.g. if the maximum was 20 and the average was 10, diff would be 2.0.
    • If everyone achieved the maximum, diff would be 1.0.
    • If the average was very low, say 1/20, diff would be 20.
  • However, difficulty depends not only on the problem but also on the competitor, so a problem which would be very hard for a Foundation competitor might be quite easy for an Advanced one. The figure for diff is therefore standardised for a typical Advanced competitor:
    • Where a problem was part of the Advanced competition in Round 1, diff is calculated as above.
    • Problems which were only included at lower levels (Breakthrough, Foundation or Intermediate) had their diff reduced by a figure based on the problems which were taken both at these levels and at higher levels, including Advance levels.
      • E.g. if an I-level problem’s average score was 5/10, giving an I-level diff of 10/5 = 2,
      • and the same competitors scored half (0.5) as well as the Advanced competitors did on the problems that they both did, giving an ‘I-adjustment’ for I and A (the mean I and A scores for those problems) of I/A = 0.5
      • then its diff (for a typical A-level candidate) was 2 x 0.5 = 1.
      • Similarly for lower levels: so the diff of a Breakthrough problem is given by B x F x I where
        • B = the diff for Breakthrough candidates
        • F is the F-adjustment figure (F/I) for problems taken at both F and I level
        • I is the I-adjustment figure (I/A) for problems taken at both I and A level.
      • This standardization calculation explains why some diff figures are below 1.0 (which would otherwise be the lowest possible).
    • Problems from Round 2 had their diff increased by a figure based on the difference between the Round 1 figures for the winners (who eventually took Round 2) and the overall average.
      • E.g. if the R2 diff for a problem was 2,
      • and the R1 Advanced winners’ average score was twice the average for all Advanced competitors,
      • then its diff for a typical A-level candidate was 2 x 2 = 4.