Little Golem#1 · darse
21 Jan 2012
I've enjoyed my first few games of Empathy, and have enjoyed some success (winning on my first two tries, then 3 out of 5, and coming very close on a couple of others).
I didn't see a good way of finding out who the best players are, so i wrote a program to gather and parse all the game results, and calculate a few different metrics. The next couple of posts will provide the results of that analysis.
Please critique or comment on my methods, or suggest other measures that you might like to see.
Here are the three scoring methods i used:
1. The shared win - shared loss score.
All the winners share in the win, and all the non-winners share in the loss. For example, if there were 22 players, and two of them tied for first, then the winners would each earn 1/2 = 0.5 wins, and the non-winners would each get -1/20 = -0.05 losses (making it a zero-sum metric). Over many games, we sum a player's win/loss shares, and divide by the number of games played to get an average.
2. The number of "chips" won.
This model is like a poker game: everyone antes one chip to play, and the winners split the pot equally. In the previous example, each of the non-winners would net -1 chip, while the winners would net +10 chips each (they gross 11 chips, but it cost them 1 for the ante). That makes it a zero-sum total again, and winning a game with many players counts for more than winning a smaller game. Over many games, we sum the chip totals, and divide by the number of games played to get an average (or "return on investment", ROI).
3. The fraction of the best possible score.
Here we compare the player's score to the maximum score possible (i.e. the sum of the top 10 answers). [1] Since 10 unique answers still earns 10 points, i subtract 10 from both totals to get a fraction in the range 0.0 - 1.0. [2] Over many games, we sum the fractions, and divide by the number of games played to get an average.
I did not sort the player rankings by average fraction, because that number depends a lot on which games were played. Some wins will have a 1.0 fraction, while other games have tough categories where the winning fraction might be only 0.75, or less. The average winning fraction was 0.9037 (~90%) over the 65 games in the archive so far. [3]
Footnotes:
[1] If no one had a perfect score, then you could argue that the best possible score was actually a little higher. I ignore that and just use the sum of the top 10 answers as the denominator.
[2] Some players somehow managed to score less than 10 (e.g. an incomplete entry), but they do not score below 0.0.
[3] I've read some forum posts indicating that many past games were lost as a result of a server crash. Unfortunately, i have no way of accessing those game results.
For the averages, a minimum number of games is required, so that statistically meaningless results don't pollute the top of the list. If a player has played less than the minimum number of games, then their totals are divided by the minimum instead of by the actual number of games (i.e. a true Bayesian average). I would normally choose a 10-game or 20-game minimum, but i chose 7 here because Richard's Empathy results page gives a table for the last 7 games. [No, i didn't choose 7 just because that's the number of games i've played under "drbot" -- and i'd still be on top of the list with a 20-game minimum anyway, so there. :-p]
For the player rankings, i've only shown the top 27 out of 121 players who have played at least one game of Empathy (roughly the top 22%). There was a natural break at around the zero expected value point, just above the many players who have played only one game.
I haven't shown the whole table because i don't want to risk causing any embarrassment to anyone. If you're not on the list, and would like to know your ranking and stats, then send me a message and i'll send you the row.
Following the player rankings will be a summary of all the games played so far (since the crash). Included in that table is the number of unique answers given for each game (and average number of uniques per player), which can be viewed as a measure of how disperse the answers were (or perhaps how hard the category was, in terms of reaching common consensus choices).
I'm new to LG, and not yet sure how to do proper formatting, so forgive me if the first attempts get messed up.