Once again the Lemmings were not playing but asking the questions this time. For some weeks this blog has been an excellent forum for the exchange of ideas about balance, obscurity etc. and I feel it proper to say something about the Park Taverners’ questions. To guard against imbalance and so on, they had drawn up a spreadsheet identifying for each question where the player would be sitting (so they don’t get questions on the same subject twice), wh
ether the questions would be classified as history or whatever (for subject balance) and a rating as to how difficult the questions were. I think this was a remarkable effort and is to be applauded – it certainly showed at the Ox-Fford where the scores were very close. Against this however, they should bear in mind the advanced years of some of the question masters and print the questions in a larger font than last night.
Nick was at the Ox-Fford where there was a battle at the top of the League with the Weaver; the specialist did not suit the Weaver at all but the GK was a much closer battle; the pub has greatly improved with a wider selection of ales and a much brighter ambience – and they make excellent sandwiches.
Wendy was saved a long drive to the Royal Oak which was closed for maintenance, and her battle was held at the Old Millstone – once the centre of the league.
Any reports of something interesting in the other two games will be added later.
8 comments:
Our QM may well contribute in due course, but I've beaten him to it.
The Robin faced the Dragons in one of those "just about winnable" fixtures. Sadly, 'twas not to be. A interval deficit of 20 (it looks as if first first may have been the road to hell) was clawed back a little bit in the second half, but to no avail.
Credit to the Park - we enjoyed a close fought and productive GK section, which strode the boundary between answerable ( by someone) and a bit too easy well. I noted that my first five qs in the GK were very current - only one could have been asked a year ago, but I got them, so was happy. They were a welcome antidote to a painful specialist round, certainly edging towards difficult, but in that frustrating territory of questions that you could have known, rather than armadillos. 20/20 was a bit clumsily worded and (uncharacteristically for the GKs) the one about Relativity was a bit too precise for the 8 non- physicists in the room - we revelled against the scoresheet and played a supp.
After last week's discussion, I must make reference to the very convivial way in which this potentially crucial and (eventually) closely fought match was played - humour, generosity and co-operation reigned and the Dragons certainly offer diverse charm and humour and one certainly wasn't anticipating an anecdote about being knifed by Steve Marriott in a London playground. The night meandered along gently, never quite stalling to form an oxbow lake, with Messrs Langstaff, Turner and Moss uniting to offer a comedy trio that had me pining for Des o'connor. I will choose not to relay the one about the suicidal steam train enthusiast, those who want the punchline can seek out Duncan Disorderly.
(GK 15) Ukulele [which fell to me] had the scent of armadillo about it. (GK 71) I covered all bases with the answer "maps, cartography, drawing places which have recently been discovered" [I also got the SPECIFIC Einstein question]. (GK 80) I thought the answer was Jill!
Since it was the Park Timers' week off and I haven't seen our sister team's questions, I thought this might be a good opportunity to say how we in the Park Timers have gone about setting questions for several years now, without it looking like I'm pointing fingers at anyone (I'm not). Our entertainment scores this season suggest we're getting better at it so some of this must be working. I've boiled it down to 8 points, but the main thing seems to be starting early and involving the whole team at each stage. There's a bit of time involved but it's only once a season and it makes quiz mastering on the night more enjoyable.
1. We also use a spreadsheet approach, which sounds similar to the Taverners. Here's the (previously secret) link: https://docs.google.com/spreadsheets/d/1tFHe6WZnAGkZOZnNE9PXQs_xHcHChsYX5MSYHzLDan8/edit?usp=sharing
Essentially it's a long list of all the questions next to their answers, but with easy ability to then look at which questions and answers are landing on each of the 8 players and across the two teams. We might not get the balance perfect but we can try, and have caught several problems along the lines of one player getting two rugby questions or one team getting more "name the year" questions than the other etc. We can also check that the balance of hard/easy questions within each round and the specialist/GK as a whole looks about right, both for the players and the teams. This is more subjective but, again, at least we can try.
The spreadsheet also enables me to "mail merge" its contents into numbered questions and answers, consistently formatted, into a Word document I've set up (happy to share).
In principle we can share the spreadsheet with the vetting teams, although not all have had good enough web browsers in the past for it to work well for them, so sometimes we just send the merged documents for comment. I don't know how vetters can really do a thorough job of checking balance without an approach like this. It is harder though to ensure full balance across the entire game, when the two "halves" are set by different teams - but we do try and look.
2. All of our team, including regular subs, are invited to set questions. This stops "our" rounds from being the work of one person and helps mix things up a bit. I think some other teams still have all their questions set by one person.
3. We started setting these and discussing them over a month before we had to read them. It meant we weren't rushed and didn't have problems when people were busy or away for a particular weekend.
[Parts 4-8 follow because I'm limited to 4096 characters per post]
[part 2]
4. Before we send them for vetting we thoroughly vet each others within the team. I imagine how the most awkward players in the League (no names) could possibly argue over a question/answer and then make sure we have phrased our question and answer to avoid that. We look up most of each others' questions on the internet and give them a good kicking - it's better we kick them ourselves than the players on the night. It's amazing how many questions out quiz books are wrong or arguable (I try and avoid copying whole rounds out of books anyway).
5. If my fellow team players tell me my questions are too hard or obscure I pay attention, even if I think they are "interesting" or "easy". Players who say "they were alright but there were a few easy ones in there" seem to enjoy themselves a lot lot more than those who have to say "they were alright but there was few really obscure and difficult ones in there and I hated that question".
6. We send them to all the available vetting teams about two weeks ahead, and then make sure we alter any questions that sound too similar to the ones that come up in the game or two before we are due to set. Not every potential vetter replies. We might not agree with all the vetters' feedback but we consider it carefully and try to accommodate the majority of it. If our vetters thought something, chances are some of the players on the night would say the same thing, whatever we might think.
7. All our team vet the questions of the other team setting on the night. Again, I don't think every team takes this approach, but my colleagues have different knowledge to me and will offer different views. They will be reading out the questions on the night and the questions shouldn't come as a surprise to them, or without the opportunity for them to offer feedback to the other setters.
8. Once the last game before ours is over we check for any late alterations needed, and run the mail merge. I usually circulate a PDF version so that it won't change the page breaks or formatting when someone else prints it out. I get the rest of the team to cast their eye over for any typos that have escaped us. We've hopefully got a few days left at this point to get on with the actual printing.
I hope some of this is useful and I'd be happy to explain my Excel/Word approach to anyone who would find that helpful.
Excellent work, Glyn. I'm sure that I speak for several in saying that there is a lot of useful advice in there. There is a bit of "IT Lingo" that has flown beyond my skill set, but nonetheless very useful. It wouldn't be a bad idea if Nick or Mark put a permanent link on their blog to the spreadsheet.
Balance is, from experience, not enough to guarantee satisfaction - the overall difficulty level is vital. My gut feel is that the "perfect quiz" would allow scores of between about 18 and 54 - wouldn't want people to get 60s without genuinely knowing stuff that is beyond what can normally be expected. A sad reality is that there are some players who need to be scoring below 24/27ish every week if the "top-enders" are to get stretched a bit. However, can vouch for the idea that it is better to err towards ease than difficulty.
The crucial elements coming through (and they go hand in hand) is the period of time before the quiz that you have started at and the number of people involved in the process. I'm sure that every one of us could set a quiz that we thought was easy, but is almost universally unanswerable - I certainly did so this year - and would have valued realising this well before the quiz itself!
I was asking at The Waters and the only questions I thought would cause real trouble were the ones about gravitational waves and special relativity. I was prepared to explain that gravity waves are something quite, quite different, and pick out the difference between special, general and other relativities, but all passed without incident. In fact, the class was so well behaved, we were all eating butties and doing supps by 21:40. Hurrah!
The Ox-fford played the Weaver this week which normally guarantees a very close contest. Most unusually, the Ox-fford won the Specialist round 75 – 42, which is a pretty extraordinary margin in a game between two sides that historically score reasonably well most weeks, no matter what questions are on offer.
In contrast, the GK was much closer with the Ox-fford winning 100 – 96. The GK included the Weaver throwing in some guesses for “3s” in an effort to reduce the overall deficit, so the GK would have been even closer or indeed the Weaver would most likely have won that round on a different night.
I thought the GK was very even most of the way through although I did think on the night that we might have had the better of the Specialists. The Weaver won the toss and went “first first”, so all the odd numbered questions were theirs and the evens were ours.
Only 7 Specialist questions actually went unanswered by either side, 5 of those being the Weavers and 2 of them ours. In Round 1 (Blue is the Colour), I was particularly delighted as a fourth-generation Manchester City supporter to get off the mark with the “Blue Moon” question and the Weaver had the only unanswered question in the rest of that set with South Korea’s “Blue House” passing us all by.
In Round 6 (A&E), no one knew or cared who Mark Stinchcombe was (Ox-fford Q) or had read any of Clive Cussler’s books (Weaver Q).
In Round 7, 20/20 (Weaver Q) left us all wanting another drink and then unusually, there were 3 questions in Round 8 (Sport) that no one got – Oliver McCall (my question), the Lakeside Hammers Speedway merchants (Weaver) and David Beckham’s 3 consecutive World Cup scoring efforts (Weaver again).
An even closer look at the scoresheets in our game shows that 29 out of the 33 points advantage on the Specialists came in just 3 rounds – Round 2 (Geography) finished 9 – 1 (one “3” + three conferred + three passed over against one conferred) Round 4 (Advertising Slogans) finished 14 – 2 (four “3s” + two passed over against two conferred) and Round 8 (Sport) finished 9 – 0 (two “3s” + one conferred + two passed over).
The Geography and Slogans rounds look reasonably balanced to me whilst the odd-numbered Sports questions were probably harder than the evens.
I think we had one of those nights when we were lucky to be asked questions we knew the answers to and picked up some passed overs that we wouldn’t do normally. I haven’t yet seen any other scores so it will be interesting to see how other matches turned out.
Looking more widely at Nick’s comments and Glyn’s example spreadsheet etc, The Park Taverners did a good job and Glyn makes some very good points about how to assemble questions and how long it does take to put a reasonable set together. I will certainly look to put a link to the Google documents page in a more prominent place for anyone who wants it.
I tend to use Excel these days when the Ox-fford set questions. It works just as well for the Cup / Plate too, indeed it is even easier in that the Cup / Plate is all “General Knowledge” so everything can be used and the only thing to try and do is make the questions of similar difficulty for both teams as far as possible. One team gets all the odd-numbered questions and one the even-numbered questions.
To try and avoid too much obvious “pairing” (not that you have to), I assemble the questions in blocks of 40, so Questions 1 and 40 are on a similar subject and of similar difficulty (as best as can be judged), as are 2 and 39, 3 and 38 and so on – or at least I think they are by the end! I do the same for each group of 40, so 41 and 80 are similar, 42 and 79, 43 and 78 and so on.
It isn’t surprising that the longer you spend doing them, the better they usually are…
I have used a similar spread sheet for years, thanks to my husband's obsession with the pleasures of spread sheets, with a balancing tool, and everything. It is occasionally temperamental, but is invaluable for balance.
People still complain though...
Liz Horrocks
If anyone would like a template of said balancing tool, I can help!
Post a Comment