View source for Lojban Rating System


This page is an answer to the suggestion (1) of adopting a kind of chess-rating system for Lojban speakers:

"One way of doing this, that was suggested on IRC yesterday, is to set up a 

system based on the ELO ranking system, which was originally designed 

for the game of chess.<br />The way we are thinking of adapting this to Lojban is roughly as follows:<br />Each "match" is a conversation. After the conversation, both parties throw a secret ballot which indicates whether they thought themselves better or 

worse than their "opponent". If the ballots agree, a winner is designated 

appropriately. If the ballots disagree, the conversation is treated as a 

draw for the purposes of the ELO system."

Many replies came to this suggestion, some concerning moral/ethical side, some concerning psychological side, more - technical.<br />

I think that the main factor in taking decision on creating such system must be the benefit of the language. Will Lojban benefit from it, or not? It is dubious that the "players" will benefit from such "matches", either, since only they will be left to judge the winner of the talk. Besides, such conversations will probably be quite unvaried, as typical talks from ICQ's and chats are.

The main need of Lojban today (my opinion, of course!) is a large textual base, gathered in one place, each sentence/piece of text properly classified as "Correct Lojbanic" or "Wrong Lojbanic". This can be easily combined with rating system allowing constant increase of ratings - a feature that so attracts many RPG players! (:wink:) 

This could be at the same time a rating system for estimating the competence, activity and contribution to the language of those who write in Lojban. 

An outline of such measuring system may be like this:

# '''Portions''', each consisting of one or more sentences in English or other non-Lojban language, are posted and stored on a server, categorized by topics. 

# The goal is to have a number of correct translations of the portions into Lojban.

# Everyone is free to add new portions on the server.

# Everyone is free to add his/her translation of the portions.

# Everyone is free to mark any translation (by other people) of any portion as ''correct'' or ''incorrect'', with optional comments.

# The weight of a translation varies between 0 and 1. It is 0, IF no one has approved it, and 1, IF (no one has disapproved it AND at least one person with non-zero level of contribution has approved it), OTHERWISE it is the special function, definition below.

# The level of contribution to the language is the sum of weights of all translations made by that person.

The definition of the special function:<br />

Suppose ''N'' persons have marked a translation of a portion.<br />

Let ''Fi'' be equal 1, if the i-th person has approved the translation, and 0, if he/she has disapproved the translation.<br />

Let ''Ri'' be the rating (level of contribution) of the ''i''-th person.<br />

Let ''T = SUM(i=1..N, Ri)'', i. e. the sum of ratings (levels of contribution) of all markers (those who mark, I mean).<br />

Then the weight of the translation, ''x'', equals

''x = SUM(i=1..N, Fi * Ri / T)''.

It can be easily seen that the 'special function' works for 0- and 1- cases as well. :)

Since only posting translations will be leading to increase of the rating, some lojbanists may (although I don’t really believe in this) participate only in translating. So there must be formal means of encouraging not only translating, but also posting new portions (texts to translate) and reviewing / marking translations of others. For example, everyone may be able to make 10 translations from the beginning, but in order to make each next translation after those 10, he/she will have to post 0.5 portions and review / mark 4 translations of others.

Besides, people could estimate the translations of others by giving not only 1-s and 0-es, but also values in between:

||1|'Absolutely agree with this translation'

0.9|'The translation is correct, although I would have put this in other way'

0.5 – 0.7|'The translation would be correct after minor fixes'

0.3|'The translation is incorrect in general, but some moments were interesting / OK'

0|'Absolutely disagree with this translation / Written Nonsense / A copy of translation made before'||

Portions to translate won’t be equal in difficulty (compare “Me invent wheel” with an excerpt from Homer). So it would be reasonable to use a difficulty multiplier. The poster of a portion may specify its difficulty between 0.1 and 5, or leave it 1 by default. Of course estimating difficulty is also very subjective, so it may also be computed the same way as weights of translations were:

Multiplier is 1, if no one with non-zero rating has estimated it.

Otherwise, it is a value between 0.5 and 5, rating-averaged from estimates.

It will be great if newbies could filter out only easiest portions, with multiplier < 1, to translate. Vice versa, experienced Lojbanists could be interested in hard ones.

---

(1) Lojban-beginners mailing list, 17 November 2005.

'''A realization of such system, called "jboselkei", is being written by me in PHP. It will be finished (and I hope, available) in mid-late March 2006.'''

Update: Yanis unveiled jboselkei to the Lojban community on March 15. [http://www.lojban.org/jsk/ lick here to sign up and play.]