## Test problems from past UKLO competitions

=============================================

## Explanation

**Below** you will find a list of all the test problems that have been used in UK Olympiads since 2010. You can also download the same list as an Excel spreadsheet; this is not locked, so once you’ve saved it locally, you can easily search it, edit it or re-sort the data. In both lists, each problem’s name has a link to the downloadable problem.

Each problem is listed with the following information:

- diff: its relative difficulty, as explained below
- (Note: for problems set in 2010 and 2011 we have to rely on an informal grading in the two books of problems. This five-point grading is partly based on test scores in the American competition and partly estimated by test-setters; unfortunately there are no problems graded by both systems, so they are hard to match. The Japanese problem from 2011 isn’t included in these books so its grading is just a guess.)
- year: the year in which it was used in UKLO
- number, name: its number in that year’s problem set, and its name
- this cell is hyperlinked to a file containing the problem concerned, a separate answer sheet (where relevant), and the correct answers, plus a mark scheme and detailed directions for marking (as used by the original markers).

- language: the language(s) from which its data are taken
- extras?: any information beyond the problem and its solution.
- expanded: an extended essay-length discussion of the problem and how to solve it, as well as ideas for building on it in discussing more general ideas about language and linguistic analysis.
- notes: brief notes on the problem data, possibly as a separate file from the problem file.
- problem, solution, commentary and marking scheme: a single file containing all relevant information.

## The problems

The problems are listed below in increasing order of difficulty.Note that:

- the table is divided into a number of pages which you can step through by pressing the button on the bottom right, but you can also choose how many rows to display in each page.
- in spite of this division into pages, you can reorder the rows by clicking on the column headers; e.g. instead of the default ordering by difficulty you can order by year, starting either with the oldest or the youngest.
- there’s a useful search box in the top right of the table, so (for instance) you can search for all the problems based on English by typing “English” into that box.

The problems are also listed in a downloadable spreadsheet which you can edit in Excel like any other spreadsheet. Moreover, unlike the list below, this spreadsheet also includes clickable links to the downloadable problem files, so if you want to download the files, you need the spreadsheet.

#### The table of problems

diff | year | number, name | language | extras? | area | author | ||
---|---|---|---|---|---|---|---|---|

0.5 | 2011 | 2. Japaneasy | Japanese | expanded | script | Harold Somers | ||

0.54 | 2014 | 3. Mokilese | Mokilese | expanded | general | Jeff Siegel | ||

0.67 | 2014 | 1. What time is it in Tallinn? | Estonian | expanded | general | Babette Newsome | ||

0.71 | 2018 | 18.1 (In)definitely Romanian | Romanian | notes and solution | general | Julia Barron | ||

0.81 | 2018 | 18.2 Lithuanian road trip | Lithuanian | notes and solution | general | Babette Newsome | ||

0.83721279460816 | 2019 | P2 Japanese | Japanese | notes | script | Babette Newsome | ||

0.85840027993281 | 2016 | 2. Foreign | French, German, Spanish | problem, solution, commentary and marking scheme | comparative | Dick Hudson | ||

0.89 | 2012 | 1. Being with it in Yolmo | Yolmo | expanded | general | Lauren Gawne | ||

0.9 | 2017 | R1.3. We are all European | 9 languages | notes and solution | comparative | Babette Newsome | ||

0.92 | 2012 | 2. Danish numbers | Danish | expanded | numbers | Mike Swan | ||

0.9487586661846 | 2016 | 4. Beijing subway | Chinese | problem, solution, commentary and marking scheme | script | Catherine Sheard | ||

1 | 2010 | 1. Sorry we have no red cucumbers | French | syntax | Dragomir Radev | |||

1 | 2010 | 2. Gelda’s House of Gelbergarg | English | general | Cindy Schneider | |||

1 | 2010 | 3. Say it in Abma | Abma | general | Luda Kedova & Rachel Nordlinger | |||

1 | 2010 | R2.3. F u cn rd ths | English | computing | Richard Sproat | |||

1.0039859930683 | 2016 | 1. Alphabet | Greek | problem, solution, commentary and marking scheme | script | Dick Hudson | ||

1.0163011897517 | 2019 | P3 Jahai | Jahai | notes | meaning | Babette Newsome | ||

1.05 | 2017 | R1.1. Our Italian family | Italian | notes and solution | morphosyntax | Dick Hudson | ||

1.06 | 2015 | 1. Counting in Karelian | Karelian | numbers | Graeme Trousdale | |||

1.08 | 2014 | 2. Maori loanwords | Maori | notes | general | Pat Littell | ||

1.17 | 2012 | 6. Crocodile Bardi | Bardi | general | Catherine Sheard | |||

1.18 | 2017 | R1.5. Basquing in the sun | Basque | notes and solution | morphosyntax | Aleka Blackwell | ||

1.19 | 2018 | 18.5 At Ease in Gilbertese | Gilbertese | notes and solution | general | Michael Salter | ||

1.2 | 2013 | 8. A little Dutch problem | Dutch | general | Harold Somers | |||

1.2516202260259 | 2016 | 9. Nhanda | Nhanda | problem, solution, commentary and marking scheme | general | Babette Verhoeven | ||

1.26 | 2014 | 6. Kairak verbs | Kairak | notes | general | Cindy Schneider | ||

1.26 | 2015 | 2. Georgian places | Georgian | script | Daniel Rucki | |||

1.28 | 2018 | 18.3 Finding your way in Sofia | Bulgarian | notes and solution | script | Babette Newsome | ||

1.2931995540691 | 2016 | 8. Somali | Somali | problem, solution, commentary and marking scheme | morphology | Harold Somers | ||

1.3064235294237 | 2019 | P4 Welsh | Welsh | notes | morphosyntax | Babette Newsome | ||

1.32 | 2015 | 3. Polish your Polish | Polish | general | Daniel Rucki | |||

1.3479711739734 | 2019 | P5 Pitjantjatjara | Pitjantjatjara | notes | general | Rebecca Dafina and Wilmoth | ||

1.36 | 2012 | 3w. Welsh libraries | Welsh | notes | general | Dick Hudson | ||

1.3675213675214 | 2016 | 6. Kaqchikel | Kaqchikel | problem, solution, commentary and marking scheme | general | Michael Yoshitaka Erlewine | ||

1.38 | 2014 | 4. Running speech | English | notes | phonetics, script | Sue Barry & Dick Hudson | ||

1.39 | 2017 | R1.4. A little Tshiluba | Tshiluba | notes and solution | morphosyntax | Tom McCoy | ||

1.4004954694488 | 2016 | 3. Watsonium | Irish | problem, solution, commentary and marking scheme | spelling of loans | Jane D’Altuin and Harold Somers | ||

1.42 | 2017 | R1.2. Inuktitut script | Inuktitut | notes and solution | script | Ollie Sayeed | ||

1.4368067378375 | 2016 | 5. Amele | Amele | problem, solution, commentary and marking scheme | general | Babette Verhoeven | ||

1.44 | 2013 | 5. Bulgarian | Bulgarian | syntax | Bozhidar Bozhanov | |||

1.488004927245 | 2019 | P1 Ladin | Ladin | notes | general | Julia Barron | ||

1.4997332461288 | 2016 | R2.1 Malay | Malay | Problem, solution and marking | general | Bozhidar Bozhanov | ||

1.51 | 2015 | 4. Old English | English | general | Graeme Trousdale | |||

1.5574825174825 | 2016 | 7. The strange case of the Estonian language | Estonian | problem, solution, commentary and marking scheme | general | Praveen Venkataramana | ||

1.56 | 2013 | 1. Yodaspeak | English | syntax | Harold Somers | |||

1.59 | 2012 | 8. Arcturan intergalactic peace message | fictitious language | general | Simon Zwarts | |||

1.59 | 2015 | 5. Elder Futhark | Norse | script | Catherine Sheard | |||

1.69 | 2012 | 7. Waorani numbers | Waorani | numbers | Drago Radev | |||

1.72 | 2012 | R2.2. Who is good? | Luiseño | general | Ronald Langacker (via Dick Hudson) | |||

1.76 | 2012 | 4. Haitian | Haitian | notes | general | Ivaylo Youmerski | ||

1.76 | 2015 | 8. Can you finish Finnish? | Finnish | general | Babette Newsome | |||

1.77 | 2017 | R1.6. Maori for the beach | Maori | notes and solution | grammar | Aleka Blackwell | ||

1.77 | 2017 | R1.8. Choctaw | Choctaw | notes and solution | morphosyntax | Babette Newsome | ||

1.79 | 2013 | 6. The long and short of English verbs | English | notes | general | Dick Hudson | ||

1.8156050669302 | 2019 | P7 Harmongolian | Mongolian | notes | morphophonology | Ethan Chi | ||

1.8348570361662 | 2016 | R2.2. Tocharian | Tocharian | Problem, solution and marking | historical | Ollie Sayeed | ||

1.93 | 2012 | 5. Esperanto | Esperanto | general | Alexey Pegushev | |||

1.95 | 2018 | 18.6 Are you OK with N’ko? | Nko | notes and solution | script | Babette Newsome, Harold Somers | ||

1.96 | 2018 | 18.4 What's yours is mine (for eating) | Fijian | notes and solution | general | ?? | ||

2 | 2010 | R2.1. Ardhay Uzzlepay | Minangkabau | general | Diego Molla-Aliod | |||

2 | 2011 | 7. The war of the dots | English (braille) | script | Patrick Littell | |||

2 | 2011 | R2.1. Stopping and flapping in Warlpiri | Warlpiri | general | Mary Laughren | |||

2 | 2011 | R2.2. Counting in Irish | Irish | notes | number | Tom Payne | ||

2 | 2011 | 1. Running on MT | English | meaning | Harold Somers | |||

2 | 2011 | 3. Doubling in Caterpillar Country | Arrernte | notes | general | Mark Dras & Mary Laughren | ||

2 | 2011 | 5. Mix up on the farm | Papago | general | Lori Levin (data from Ken Hale) | |||

2 | 2011 | 6. Tiger tale | Indonesian | general | Dragomir Radev | |||

2.0370607540223 | 2019 | P9 Ndebele | Ndebele | notes | general | Michael Salter | ||

2.0804226432579 | 2016 | R2.5. Get edumacated! | English | Problem, solution and marking | phonology | Patrick Littell | ||

2.12 | 2013 | 3. Pali | Pali | general | Babette Newsome | |||

2.13 | 2015 | 7. Murrinhpatha | Murrinhpatha | general | Rachel Nordlinger | |||

2.1689005481638 | 2016 | 10. Don't sell the house! | Nung | problem, solution, commentary and marking scheme | general | Alex Wade | ||

2.22 | 2012 | R2.1. A fox among the h | English | computing | Patrick Littell | |||

2.35 | 2012 | R2.3. The little engine that could… read | English | logic | James Pustejovsky, Patrick Littell | |||

2.4 | 2012 | 9. Waanyi | Waanyi | general | Mary Laughren | |||

2.46 | 2015 | R2.3 Do-this-do-that | Hmong | notes | general | David Mortensen | ||

2.56 | 2015 | 9. Zoink! | English | semantics | Dragomir Radev with Christiane Fellbaum and Jonathan May | |||

2.6353137770436 | 2019 | R2.2 Lepcha | Lepcha | notes | script | Ethan Chi | ||

2.65 | 2017 | R1.9. The goat, the mother and the Abkhaz wardrobe | Abkhaz | notes and solution | grammar | Samuel Andersson, Oliver Sayeed, and Elysia Warner | ||

2.7 | 2018 | 18.7 Icelandic Relations | Icelandic | notes and solution | general | Babette Newsome | ||

2.74 | 2015 | 6. Japanese places | Japanese | general | Harold Somers | |||

2.7549447673453 | 2016 | R2.4. Devanagari | Devanagari | Problem, solution and marking | script | Lauren Gawne | ||

2.766399839143 | 2016 | R2.3. Huli | Huli | Problem, solution and marking | numbers | Bill Huang | ||

2.84 | 2017 | R2.1. Nepali | Nepali | notes and solution | grammar | Babette Newsome | ||

2.88 | 2017 | R2.3. Viet | Vietnamese | notes and solution | grammar | James Hyett | ||

2.9369437275088 | 2019 | P10 Braille | English | notes | script | Babette Newsome | ||

2.99 | 2014 | 5. Turkish | Turkish | notes | general | David Palfreyman | ||

3 | 2010 | 4. Lost in Yerevan | Armenian | script | Dragomir Radev | |||

3 | 2010 | 7. Texting, texting, one two three | English | code | Patrick Littell | |||

3 | 2010 | R2.2. Be Cree-ative | Cree | script | Patrick Littell & Julia Workman | |||

3 | 2010 | 6. Tangkhul tangle | Tangkhul | general | David Mortensen | |||

3 | 2010 | R2.4. Tale of Kieu | Vietnamese | script | David Mortensen | |||

3 | 2010 | R2.5. Possessed in Vanuatu | Vanuatu | general | Jane Simpson & Jeremy Hammond | |||

3 | 2011 | 4. Ulwa possessives | Ulwa | notes | general | Richard Sproat | ||

3.0144721003143 | 2019 | R2.1 Afrihili | Afrihili | notes | general | Michael Salter | ||

3.06 | 2012 | R2.5. Catalan plurals | Catalan | morphophonology | Boris Iomdin | |||

3.2 | 2017 | R1.10. Kaytetye kinship | Kaytetye | notes and solution | semantics | Myfany Turpin | ||

3.24 | 2014 | 7. Ilokano | Ilokano | notes | general | Bozhidar Bozhanov | ||

3.29 | 2013 | 9. Bengali | Bengali | notes | general | Bozhidar Bozhanov | ||

3.32 | 2017 | R2.2. Do you see what I see? | Proto-Algonquian | notes and solution | morphosyntax | Daniel Lovsted | ||

3.39 | 2012 | 3d. Dutch past participles | Dutch | notes | general | Babette Newsome | ||

3.49 | 2013 | 4a. Arabic script | Arabic | notes | script | David Palfreyman | ||

3.63 | 2015 | R2.2 Malagasy crossnumber | Malagasy | notes | numbers | Tom McCoy | ||

3.6410701876303 | 2019 | P6 CippusAbellanus | Oscan | notes | script | Michael Salter | ||

3.98 | 2014 | 9. Lontara | Buginese | notes | script | Chelsea Voss | ||

3.9822519297547 | 2019 | R2.4 Cupeno | Cupeno | notes | general | Daniel Lovsted | ||

4 | 2011 | R2.5. Swallow the salt | Tadaksahak | general | Bozhidar Bozhanov | |||

4.02 | 2017 | R2.4. Muddled hieroglyphs | Egyptian | notes and solution | script | Dick Hudson | ||

4.23 | 2017 | R1.7. On the right Tamil track | Tamil | notes and solution | script | Kai Low | ||

4.35 | 2017 | R2.5. Magic Yup’ik | Yup'ik | notes and solution | number | Kai Low | ||

4.61 | 2014 | R2.1. Kiswahili verbs | Kiswahili | general | Catherine | |||

4.64 | 2018 | 18.R2.4 It’s true: The truth about Chalcatongo Mixtec | Mixtec | notes and solution | general | Babette Newsome | ||

4.6765261148764 | 2019 | R2.5 Witsuwit'en | Witsuwit'en | notes | general | Daniel Lovsted, Sam Ahmed, Ellie Warner | ||

4.8459522655128 | 2019 | R2.3 Polish | Polish | notes | morphophonology | Ms Veneva | ||

5 | 2010 | 5. Turkish delight | Turkish | general | Bozhidar Bozhanov | |||

5 | 2011 | R2.3. Axolotl in the water | Nahuatl | general | John Berman | |||

5 | 2011 | R2.3. A script for the Ndyuka | Ndyuka | script | Patrick Littell | |||

5.07 | 2018 | 18.R2.2 Nivkh | Nivkh | notes and solution | general | Heather Newell | ||

5.3 | 2018 | 18.R2.3 Menya | Menya | solution | general | Aleka Blackwell | ||

5.63 | 2013 | R2.4. Stockholms Tunnelbana | Swedish | general | Patrick Littell | |||

5.74 | 2013 | R2.1. Deer Father | Quechua | general | Patrick Littell | |||

5.93 | 2018 | 18.8 Vietnamese Matching | Vietnamese | notes and solution | general | Tom McCoy, Pat Littell, and Lori Levin | ||

6.06 | 2013 | 4s. The Shavian Alphabet | English – Shavian | notes | script | Babette Newsome | ||

6.1 | 2018 | 18.10 Tirana tourist | Albanian | notes and solution | general | Ali Sharman | ||

6.48 | 2014 | R2.2. Untangling Tanghulic | Kachai, Tusom, Ukhrul | comparison | David Mortensen | |||

6.5 | 2015 | R2.4 Aymara | Aymara | phonology | Josh Falk | |||

6.68 | 2014 | R2.4. Navajo gophers | Navajo | general | Babette Newsome | |||

8.11 | 2014 | R2.3. Learning Yidiny | Yidiny | general | Mary Laughren | |||

8.58 | 2015 | 5 Romance | Latin, French, Catalan, Romanian | historical comparison | David Palfreyman | |||

8.72 | 2013 | 7. Phoenician | Phoenician | script | Harold Somers | |||

9.01 | 2014 | R2.5. Hungarians in a field | Hungarian | puzzle | Adam Hesterberg | |||

9.43 | 2015 | R2.1 Maxakali | Maxakali | general | Alex Wade | |||

9.48 | 2015 | 10. Georgian Era | Georgian | script, general | Dorottya Demszky | |||

9.99 | 2013 | R2.5. Playing the cognate game | Indonesian, Swahili | etymology | Catherine Sheard | |||

12.67 | 2013 | R2.3. Beja | Beja | general | Dick Hudson | |||

15.26 | 2018 | 18.9 Central Pamerican sums | Pame | notes and solution | number | Milena Veneva | ||

16.99 | 2014 | 8. Musical code | English | notes | code | Harold Somers | ||

17.581347150259 | 2019 | P8 Gumatj | Gumatj | notes | numbers | Ethan Chi | ||

32.94 | 2013 | R2.2. Putting the books in order | Georgian, Armenian | script | Drago Radev | |||

33.33 | 2012 | R2.4. 100 surnames | Phags-pa | script | Patrick Littell |

## Relative difficulty

The problems are classified for **difficulty**. Here’s how this was calculated:

- For problems from 2012 and later, the grading is objective and reflects the average marks of hundreds of competitors. Informally, the difficulty is the difference between the maximum possible mark and the average mark; so if everyone scores 100%, the difficulty is 1, but if the average is just 10%, the difficulty is 10. But as you’ll see below, it has to be more complicated than that!
- Mathematically, the difficulty (
**diff**) of a problem is the maximum score for that problem divided by the average score;- e.g. if the maximum was 20 and the average was 10, diff would be 2.0.
- If everyone achieved the maximum, diff would be 1.0.
- If the average was very low, say 1/20, diff would be 20.

- However, difficulty depends not only on the problem but also on the competitor, so a problem which would be very hard for a Foundation competitor might be quite easy for an Advanced one. The figure for diff is therefore standardised for a
**typical Advanced competitor:**- Where a problem was part of the Advanced competition in Round 1, diff is calculated as above.
- Problems which were only included at lower levels (Breakthrough, Foundation or Intermediate) had their diff
**reduced**by a figure based on the problems which were taken both at these levels and at higher levels, including Advance levels.- E.g. if an I-level problem’s average score was 5/10, giving an I-level diff of 10/5 = 2,
- and the same competitors scored half (0.5) as well as the Advanced competitors did on the problems that they both did, giving an ‘I-adjustment’ for I and A (the mean I and A scores for those problems) of I/A = 0.5
- then its diff (for a typical A-level candidate) was 2 x 0.5 = 1.
- Similarly for lower levels: so the diff of a Breakthrough problem is given by B x F x I where
- B = the diff for Breakthrough candidates
- F is the F-adjustment figure (F/I) for problems taken at both F and I level
- I is the I-adjustment figure (I/A) for problems taken at both I and A level.

- This standardization calculation explains why some diff figures are below 1.0 (which would otherwise be the lowest possible).

- Problems from Round 2 had their diff
**increased**by a figure based on the difference between the Round 1 figures for the winners (who eventually took Round 2) and the overall average.- E.g. if the R2 diff for a problem was 2,
- and the R1 Advanced winners’ average score was twice the average for all Advanced competitors,
- then its diff for a typical A-level candidate was 2 x 2 = 4.

- Mathematically, the difficulty (