Test problems from past UKLO competitions

=============================================

Explanation

Below you will find a list of all the test problems that have been used in UK Olympiads since 2010. You can also download the same list as an Excel spreadsheet; this is not locked, so once you’ve saved it locally, you can easily search it, edit it or re-sort the data. In both lists, each problem’s name has a link to the downloadable problem.  

Each problem is listed with the following information:

  • diff: its relative difficulty, as explained below
  • (Note: for problems set in 2010 and 2011 we have to rely on an informal grading in the two books of problems. This five-point grading is partly based on test scores in the American competition and partly estimated by test-setters; unfortunately there are no problems graded by both systems, so they are hard to match. The Japanese problem from 2011 isn’t included in these books so its grading is just a guess.)
  • year: the year in which it was used in UKLO
  • number, name: its number in that year’s problem set, and its name
    • this cell is hyperlinked to a file containing the problem concerned, a separate answer sheet (where relevant), and the correct answers, plus a mark scheme and detailed directions for marking (as used by the original markers).
  • language: the language(s) from which its data are taken
  • extras?: any information beyond the problem and its solution.
    • expanded: an extended essay-length discussion of the problem and how to solve it, as well as ideas for building on it in discussing more general ideas about language and linguistic analysis.
    • notes: brief notes on the problem data, possibly as a separate file from the problem file.
    • problem, solution, commentary and marking scheme: a single file containing all relevant information.

 

The problems

The problems are listed below in increasing order of difficulty.Note that:

  • the table is divided into a number of pages which you can step through by pressing the button on the bottom right, but you can also choose how many rows to display in each page.
  • in spite of this division into pages, you can reorder the rows by clicking on the column headers; e.g. instead of the default ordering by difficulty you can order by year, starting either with the oldest or the youngest.
  • there’s a useful search box in the top right of the table, so (for instance) you can search for all the problems based on English by typing “English” into that box.

The problems are also listed in a downloadable spreadsheet which you can edit in Excel like any other spreadsheet. Mroeover, unlike the list below, this spreadsheet also includes clickable links to the downloadable problem files, so if you want to download the files, you need the spreadsheet.

The table of problems

diffyearnumber, namelanguageextras?areaauthor diff
0.520112. JapaneasyJapaneseexpandedscriptHarold Somers0.5
0.5420143. MokileseMokileseexpandedgeneralJeff Siegel0.54
0.6720141. What time is it in Tallinn?EstonianexpandedgeneralBabette Newsome0.67
0.71201818.1 (In)definitely RomanianRomaniannotes and solutiongeneralJulia Barron0.71
0.81201818.2 Lithuanian road tripLithuaniannotes and solutiongeneralBabette Newsome0.81
0.832019P2 JapaneseJapanesenotesscriptBabette Newsome0.83
0.8520162. ForeignFrench, German, Spanishproblem, solution, commentary and marking schemecomparativeDick Hudson0.85
0.8920121. Being with it in YolmoYolmoexpandedgeneralLauren Gawne0.89
0.92017R1.3. We are all European9 languagesnotes and solutioncomparativeBabette Newsome0.9
0.9220122. Danish numbersDanishexpandednumbersMike Swan0.92
0.9420164. Beijing subwayChineseproblem, solution, commentary and marking schemescriptCatherine Sheard0.94
120101. Sorry we have no red cucumbersFrenchsyntaxDragomir Radev1
120102. Gelda’s House of GelbergargEnglishgeneralCindy Schneider1
120103. Say it in AbmaAbmageneralLuda Kedova & Rachel Nordlinger1
12010R2.3. F u cn rd thsEnglishcomputingRichard Sproat1
120161. AlphabetGreekproblem, solution, commentary and marking schemescriptDick Hudson1.0039859930683
1.012019P3 JahaiJahainotesmeaningBabette Newsome1.01
1.052017R1.1. Our Italian familyItaliannotes and solutionmorphosyntaxDick Hudson1.05
1.0620151. Counting in KarelianKareliannumbersGraeme Trousdale1.06
1.0820142. Maori loanwordsMaorinotesgeneralPat Littell1.08
1.1720126. Crocodile BardiBardigeneralCatherine Sheard1.17
1.182017R1.5. Basquing in the sunBasquenotes and solutionmorphosyntaxAleka Blackwell 1.18
1.19201818.5 At Ease in GilberteseGilbertesenotes and solutiongeneralMichael Salter1.19
1.220138. A little Dutch problemDutchgeneralHarold Somers1.2
1.2520169. NhandaNhandaproblem, solution, commentary and marking schemegeneralBabette Verhoeven1.25
1.2620146. Kairak verbsKairaknotesgeneralCindy Schneider1.26
1.2620152. Georgian placesGeorgianscriptDaniel Rucki1.26
1.28201818.3 Finding your way in SofiaBulgariannotes and solutionscriptBabette Newsome1.28
1.2920168. SomaliSomaliproblem, solution, commentary and marking schememorphologyHarold Somers1.2931995540691
1.32019P4 WelshWelshnotesmorphosyntaxBabette Newsome1.3064235294237
1.3220153. Polish your PolishPolishgeneralDaniel Rucki1.32
1.342019P5 PitjantjatjaraPitjantjatjaranotesgeneralRebecca Dafina and Wilmoth1.3479711739734
1.3620123w. Welsh librariesWelshnotesgeneralDick Hudson1.36
1.3620166. KaqchikelKaqchikelproblem, solution, commentary and marking schemegeneralMichael Yoshitaka Erlewine1.3675213675214
1.3820144. Running speechEnglishnotesphonetics, scriptSue Barry & Dick Hudson1.38
1.392017R1.4. A little TshilubaTshilubanotes and solutionmorphosyntaxTom McCoy1.39
1.420163. WatsoniumIrishproblem, solution, commentary and marking schemespelling of loansJane D’Altuin and Harold Somers1.4004954694488
1.422017R1.2. Inuktitut scriptInuktitutnotes and solutionscriptOllie Sayeed1.42
1.4320165. AmeleAmeleproblem, solution, commentary and marking schemegeneralBabette Verhoeven1.4368067378375
1.4420135. BulgarianBulgariansyntaxBozhidar Bozhanov1.44
1.482019P1 LadinLadinnotesgeneralJulia Barron1.488004927245
1.492016R2.1 MalayMalayProblem, solution and markinggeneralBozhidar Bozhanov1.4997332461288
1.5120154. Old EnglishEnglishgeneralGraeme Trousdale1.51
1.5520167. The strange case of the Estonian languageEstonianproblem, solution, commentary and marking schemegeneralPraveen Venkataramana1.5574825174825
1.5620131. YodaspeakEnglishsyntaxHarold Somers1.56
1.5920128. Arcturan intergalactic peace messagefictitious languagegeneralSimon Zwarts1.59
1.5920155. Elder FutharkNorsescriptCatherine Sheard1.59
1.6920127. Waorani numbersWaoraninumbersDrago Radev1.69
1.722012R2.2. Who is good?LuiseñogeneralRonald Langacker (via Dick Hudson)1.72
1.7620124. HaitianHaitian notesgeneralIvaylo Youmerski1.76
1.7620158. Can you finish Finnish?FinnishgeneralBabette Newsome1.76
1.772017R1.6. Maori for the beachMaorinotes and solutiongrammarAleka Blackwell 1.77
1.772017R1.8. ChoctawChoctawnotes and solutionmorphosyntaxBabette Newsome1.77
1.7920136. The long and short of English verbsEnglishnotesgeneralDick Hudson1.79
1.812019P7 HarmongolianMongoliannotesmorphophonologyEthan Chi1.8156050669302
1.832016R2.2. TocharianTocharianProblem, solution and markinghistoricalOllie Sayeed1.8348570361662
1.9320125. EsperantoEsperantogeneralAlexey Pegushev1.93
1.95201818.6 Are you OK with N’ko?Nkonotes and solutionscriptBabette Newsome, Harold Somers1.95
1.96201818.4 What's yours is mine (for eating)Fijiannotes and solutiongeneralVica Papp1.96
22010R2.1. Ardhay UzzlepayMinangkabaugeneralDiego Molla-Aliod2
220117. The war of the dotsEnglish (braille)scriptPatrick Littell2
22011R2.1. Stopping and flapping in WarlpiriWarlpirigeneralMary Laughren2
22011R2.2. Counting in IrishIrishnotesnumberTom Payne2
220111. Running on MTEnglishmeaningHarold Somers2
220113. Doubling in Caterpillar CountryArrerntenotesgeneralMark Dras & Mary Laughren2
220115. Mix up on the farmPapagogeneralLori Levin (data from Ken Hale)2
220116. Tiger taleIndonesiangeneralDragomir Radev2
2.032019P9 NdebeleNdebelenotesgeneralMichael Salter2.0370607540223
2.082016R2.5. Get edumacated!EnglishProblem, solution and markingphonologyPatrick Littell2.0804226432579
2.1220133. PaliPaligeneralBabette Newsome2.12
2.1320157. MurrinhpathaMurrinhpathageneralRachel Nordlinger2.13
2.16201610. Don't sell the house!Nungproblem, solution, commentary and marking schemegeneralAlex Wade2.1689005481638
2.222012R2.1. A fox among the hEnglishcomputingPatrick Littell2.22
2.352012R2.3. The little engine that could… readEnglishlogicJames Pustejovsky, Patrick Littell2.35
2.420129. WaanyiWaanyigeneralMary Laughren2.4
2.462015R2.3 Do-this-do-thatHmong notesgeneralDavid Mortensen2.46
2.5620159. Zoink!EnglishsemanticsDragomir Radev with Christiane Fellbaum and Jonathan May2.56
2.632019R2.2 LepchaLepchanotesscriptEthan Chi2.6353137770436
2.652017R1.9. The goat, the mother and the Abkhaz wardrobeAbkhaznotes and solutiongrammarSamuel Andersson, Oliver Sayeed, and Elysia Warner2.65
2.7201818.7 Icelandic RelationsIcelandicnotes and solutiongeneralBabette Newsome2.7
2.7420156. Japanese placesJapanesegeneralHarold Somers2.74
2.752016R2.4. DevanagariDevanagariProblem, solution and markingscriptLauren Gawne2.7549447673453
2.762016R2.3. HuliHuliProblem, solution and markingnumbersBill Huang2.766399839143
2.842017R2.1. NepaliNepalinotes and solutiongrammarBabette Newsome2.84
2.882017R2.3. VietVietnamesenotes and solutiongrammarJames Hyett2.88
2.932019P10 BrailleEnglishnotesscriptBabette Newsome2.9369437275088
2.9920145. TurkishTurkishnotesgeneralDavid Palfreyman2.99
320104. Lost in YerevanArmenianscriptDragomir Radev3
320107. Texting, texting, one two threeEnglishcodePatrick Littell3
32010R2.2. Be Cree-ativeCreescriptPatrick Littell & Julia Workman3
320106. Tangkhul tangleTangkhulgeneralDavid Mortensen3
32010R2.4. Tale of KieuVietnamesescriptDavid Mortensen3
32010R2.5. Possessed in VanuatuVanuatugeneralJane Simpson & Jeremy Hammond3
320114. Ulwa possessivesUlwanotesgeneralRichard Sproat3
3.012019R2.1 AfrihiliAfrihilinotesgeneralMichael Salter3.0144721003143
3.062012R2.5. Catalan pluralsCatalanmorphophonologyBoris Iomdin3.06
3.22017R1.10. Kaytetye kinshipKaytetyenotes and solutionsemanticsMyfany Turpin3.2
3.2420147. IlokanoIlokanonotesgeneralBozhidar Bozhanov3.24
3.2920139. BengaliBengalinotesgeneralBozhidar Bozhanov3.29
3.322017R2.2. Do you see what I see?Proto-Algonquian notes and solutionmorphosyntaxDaniel Lovsted3.32
3.3920123d. Dutch past participles DutchnotesgeneralBabette Newsome3.39
3.4920134a. Arabic scriptArabicnotesscriptDavid Palfreyman3.49
3.632015R2.2 Malagasy crossnumberMalagasy notesnumbersTom McCoy3.63
3.642019P6 CippusAbellanusOscannotesscriptMichael Salter3.6410701876303
3.9820149. LontaraBuginesenotesscriptChelsea Voss3.98
3.982019R2.4 CupenoCupenonotesgeneralDaniel Lovsted3.9822519297547
42011R2.5. Swallow the saltTadaksahakgeneralBozhidar Bozhanov4
4.022017R2.4. Muddled hieroglyphsEgyptiannotes and solutionscriptDick Hudson4.02
4.232017R1.7. On the right Tamil trackTamilnotes and solutionscriptKai Low 4.23
4.352017R2.5. Magic Yup’ikYup'iknotes and solutionnumberKai Low 4.35
4.612014R2.1. Kiswahili verbsKiswahiligeneralCatherine4.61
4.64201818.R2.4 It’s true: The truth about Chalcatongo Mixtec Mixtecnotes and solutiongeneralBabette Newsome4.64
4.672019R2.5 Witsuwit'enWitsuwit'ennotesgeneralDaniel Lovsted, Sam Ahmed, Ellie Warner4.6765261148764
4.842019R2.3 PolishPolishnotesmorphophonologyMs Veneva 4.8459522655128
520105. Turkish delightTurkishgeneralBozhidar Bozhanov5
52011R2.3. Axolotl in the waterNahuatlgeneralJohn Berman5
52011R2.3. A script for the NdyukaNdyukascriptPatrick Littell5
5.07201818.R2.2 NivkhNivkhnotes and solutiongeneralHeather Newell5.07
5.3201818.R2.3 MenyaMenyasolutiongeneralAleka Blackwell5.3
5.632013R2.4. Stockholms TunnelbanaSwedishgeneralPatrick Littell5.63
5.742013R2.1. Deer FatherQuechuageneralPatrick Littell5.74
5.93201818.8 Vietnamese MatchingVietnamesenotes and solutiongeneralTom McCoy, Pat Littell, and Lori Levin5.93
6.0620134s. The Shavian AlphabetEnglish – ShaviannotesscriptBabette Newsome6.06
6.1201818.10 Tirana touristAlbaniannotes and solutiongeneralAli Sharman6.1
6.482014R2.2. Untangling Tanghulic Kachai, Tusom, UkhrulcomparisonDavid Mortensen6.48
6.52015R2.4 AymaraAymaraphonologyJosh Falk6.5
6.682014R2.4. Navajo gophersNavajogeneralBabette Newsome6.68
8.112014R2.3. Learning YidinyYidinygeneralMary Laughren8.11
8.5820155 RomanceLatin, French, Catalan, Romanianhistorical comparisonDavid Palfreyman8.58
8.7220137. PhoenicianPhoenicianscriptHarold Somers8.72
9.012014R2.5. Hungarians in a fieldHungarianpuzzleAdam Hesterberg9.01
9.432015R2.1 MaxakaliMaxakaligeneralAlex Wade9.43
9.48201510. Georgian EraGeorgianscript, generalDorottya Demszky9.48
9.992013R2.5. Playing the cognate gameIndonesian, SwahilietymologyCatherine Sheard9.99
12.672013R2.3. BejaBejageneralDick Hudson12.67
15.26201818.9 Central Pamerican sumsPamenotes and solutionnumberMilena Veneva15.26
16.9920148. Musical codeEnglishnotescodeHarold Somers16.99
17.582019P8 GumatjGumatjnotesnumbersEthan Chi17.581347150259
32.942013R2.2. Putting the books in order Georgian, ArmenianscriptDrago Radev32.94
33.332012R2.4. 100 surnamesPhags-pascriptPatrick Littell33.33

 

Relative difficulty

The problems are classified for difficulty. Here’s how this was calculated:

  • For problems from 2012 and later, the grading is objective and reflects the average marks of hundreds of competitors. Informally, the difficulty is the difference between the maximum possible mark and the average mark; so if everyone scores 100%, the difficulty is 1, but if the average is just 10%, the difficulty is 10. But as you’ll see below, it has to be more complicated than that!
    • Mathematically, the difficulty (diff) of a problem is the maximum score for that problem divided by the average score;
      • e.g. if the maximum was 20 and the average was 10, diff would be 2.0.
      • If everyone achieved the maximum, diff would be 1.0.
      • If the average was very low, say 1/20, diff would be 20.
    • However, difficulty depends not only on the problem but also on the competitor, so a problem which would be very hard for a Foundation competitor might be quite easy for an Advanced one. The figure for diff is therefore standardised for a typical Advanced competitor:
      • Where a problem was part of the Advanced competition in Round 1, diff is calculated as above.
      • Problems which were only included at lower levels (Breakthrough, Foundation or Intermediate) had their diff reduced by a figure based on the problems which were taken both at these levels and at higher levels, including Advance levels.
        • E.g. if an I-level problem’s average score was 5/10, giving an I-level diff of 10/5 = 2,
        • and the same competitors scored half (0.5) as well as the Advanced competitors did on the problems that they both did, giving an ‘I-adjustment’ for I and A (the mean I and A scores for those problems) of I/A = 0.5
        • then its diff (for a typical A-level candidate) was 2 x 0.5 = 1.
        • Similarly for lower levels: so the diff of a Breakthrough problem is given by B x F x I where
          • B = the diff for Breakthrough candidates
          • F is the F-adjustment figure (F/I) for problems taken at both F and I level
          • I is the I-adjustment figure (I/A) for problems taken at both I and A level.
        • This standardization calculation explains why some diff figures are below 1.0 (which would otherwise be the lowest possible).
      • Problems from Round 2 had their diff increased by a figure based on the difference between the Round 1 figures for the winners (who eventually took Round 2) and the overall average.
        • E.g. if the R2 diff for a problem was 2,
        • and the R1 Advanced winners’ average score was twice the average for all Advanced competitors,
        • then its diff for a typical A-level candidate was 2 x 2 = 4.
 

Comments are closed.