Changes to Voting System

Suggestions and comments about SummitPost's features, policies, and procedures. Post bugs here.
User Avatar
lcarreau

 
Posts: 4226
Joined: Thu Sep 06, 2007 10:27 pm
Thanked: 1898 times in 1415 posts

Re: Changes to Voting System

by lcarreau » Mon Jan 28, 2013 1:00 am

yatsek wrote: ... Now I’m beginning to understand why my “power” has dropped much less than expected. I hoped that being an admin of over a dozen of others’ pages was behind that. But a couple of years ago, when I had lots of unwanted free time, I made quite a few comments. :oops:


I thought your comments were truly entertaining ... it brought my "power points" up just replying to all your comments.

I've never heard of "unwanted" free time ... I wish I had more free time and less "having to get up early to go to stupid work" time .... :wink:
"Turkey Vultures always vomit when they get nervous."

The following user would like to thank lcarreau for this post
yatsek

User Avatar
Redwic

 
Posts: 98
Joined: Tue Feb 13, 2007 5:16 am
Thanked: 23 times in 12 posts

Re: Changes to Voting System

by Redwic » Mon Jan 28, 2013 1:40 am

Technically speaking, I guess making comments on SP is making a "contribution" to SP. Just a different type of contribution.

User Avatar
Redwic

 
Posts: 98
Joined: Tue Feb 13, 2007 5:16 am
Thanked: 23 times in 12 posts

Re: Changes to Voting System

by Redwic » Mon Jan 28, 2013 1:59 am

Not to sound political, but that post by Mr. Chad showing a photo of Rick Perry was perfect timing! lol

Certain things are not mentioned (or at least I did not see them) in the FAQ section, in regards to votes or scores...

Do pages for Lists, Fact Sheets, and Logistical Centers fall under the "Custom Objects" category?

User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Re: Changes to Voting System

by mrchad9 » Mon Jan 28, 2013 2:06 am

That's in the FAQ Redwic. Not in the little tables, but in the paragraph above it. Everything you mentioned is counted like trip reports, which is the same as it was before vs how a mountain page is counted. So they count more than custom objects (we lowered what a custom object was worth relative to everything else).

Any other questions just ask.

User Avatar
lcarreau

 
Posts: 4226
Joined: Thu Sep 06, 2007 10:27 pm
Thanked: 1898 times in 1415 posts

Re: Changes to Voting System

by lcarreau » Mon Jan 28, 2013 2:36 am

Not a question, just a comment ... thanks for making "Parofes" picture photo-of-the-week.

He's got the "devils horns" thing goin' on, and that's just bitchin' ... all the best for the dude.
"Turkey Vultures always vomit when they get nervous."

The following user would like to thank lcarreau for this post
Buz Groshong, Marcsoltan

User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Re: Changes to Voting System

by mrchad9 » Mon Jan 28, 2013 2:39 am

POTW and all the others are automatically selected by the system.

User Avatar
Scott
Forum Moderator
 
Posts: 8548
Joined: Thu Aug 21, 2003 1:03 pm
Thanked: 1211 times in 649 posts

Re: Changes to Voting System

by Scott » Mon Jan 28, 2013 4:41 am

Hmmm, the scoring is "interesting". I've been messing around with it.

If you take a nearly blank page (this is a random example), here is how the scoring changes when I vote:

http://www.summitpost.org/medetsiz-tepe/780188

Current score 69.51%.

The score changes to the following when I vote:

1/10: 15.02%
2/10: 68.61%
3/10: 68.88%
4/10: 69.14%
5/10: 69.39%
6/10: 69.65%
7/10: 69.90%
8/10: 70.16%
9/10: 70.40%
10/10: 70.65%

Is this intentional because the 10/10 score system is going away?

It appears the only way one could really change the score is to vote 1/10.

It appears that unless a page has 1/10 votes, the scoring is too uniform. If you look arrange the pages by score, ones with a 1/10 score 17.xx%, but this immediately jumps up to ~70%.

http://www.summitpost.org/object_list.p ... e&page=245

Voting wise, it seems that only a 1/10 vote is important on a page that still needs work. I was going to vote the page a 2 or 3, but it won't do much. If so, we'll have to vote harsher on pages such as in the example.

User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Re: Changes to Voting System

by mrchad9 » Mon Jan 28, 2013 4:58 am

Scott a single 10/10 vote increase the score by more than 1%!!!

If you compare that to the previous system, I think you will see that a single 10/10 vote is MORE significant now than it was before.

Note that before 200 10/10 votes would raise a score from 86% to 94% or so, depending on user weights. Now 200 10/10s can raise a score from 70% to above 98%. A much bigger change.

And of course votes less than 10 have a smaller impact. Each vote definitely counts, and more than previously. We could weight them more (or less) but if we weighed them more we'd have a lot of 100% pages.

What you are seeing with the 1/10 is a different mechanism kicking in when a certain number of votes are really low (different percentages of 1s 2s or 3s can do this). He page must have only a few votes, or already a couple low votes. This mechanism allows users to bring a page to zero if it is really worthless. Otherwise it would take 20-30 folks to get it way down there (it was a suggestion by Bob Burd and Chugach Mtn Boy that I think was a great idea to add on).

User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Re: Changes to Voting System

by mrchad9 » Mon Jan 28, 2013 5:01 am

Also, note that multiple votes in the 4-6 range are going to leave the score at about 70%, whereas with 18 or so 8s a page will increase above 80%. If a page is poor, anything from 1-3 can eventually start cratering the score.

User Avatar
Scott
Forum Moderator
 
Posts: 8548
Joined: Thu Aug 21, 2003 1:03 pm
Thanked: 1211 times in 649 posts

Re: Changes to Voting System

by Scott » Mon Jan 28, 2013 5:34 am

Here's a demonstration of what I mean. Here are two pages with three votes each.

One 1/10 vote and two 10/10 votes (average vote = 7.0; score = 17.62%):

http://www.summitpost.org/mt-haku/749087

One 2/10 vote, one 4/5 vote and one 5/10 vote (average vote = 3.7; score = 68.61%):

http://www.summitpost.org/medetsiz-tepe/780188

Not a complaint, but it just doesn't seem quite right to me (check out the two pages themselves as well).

User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Re: Changes to Voting System

by mrchad9 » Mon Jan 28, 2013 6:16 am

I understand Scott. To be honest I didn't spend a lot of time trying to perfect exactly when the dramatic dropping off in scores begins. I think in most scenarios it will kick in eventually when it is warranted. Note that if I go put an additional 2 vote on the second page, or even a 3 as I just did, the score is now below 20%. With 2s and 3s it takes more time to kick in if a page already has votes.

The reason for this is the system is based on the total number of votes... It doesn't matter that the first page had 10s and the second had 5s.

Also how often does this apply? And what is the real world situation where this applies? Would anyone really put a 1 on that first page? No... so the system isn't designed for that. It can't prevent someone from being able to downvote a page with only 3 votes (neither did the last).

In a real situation folks shouldn't be placing a 1 on a page with 10s unless those who put the 10s there were ignorant voters who slap a 10 on just about anything. This is how the system was designed... so a thoughtful voter could put a 1 on a page and bring the score down if reckless people out 10s on it. It's been my observation that people are more thoughtful about very low votes than high ones. If not, there are other ways of dealing with the person unfairly placing a 1.

User Avatar
rgg
Forum Moderator
 
Posts: 859
Joined: Sat Oct 02, 2010 7:15 pm
Thanked: 192 times in 154 posts

Re: Changes to Voting System

by rgg » Mon Jan 28, 2013 1:43 pm

I have a proposal for a new algorithm to select the POTM.

Why? At the moment, the selection, although random, only selects recent images, and the same ones keep coming back pretty frequently. I would like to see older images have a chance as well, and reduce the frequency of repeating the same ones.

The basic idea? Select an image from all images above a certain minimum score, but such that the probability of selection decreases depending on how old the image is. For practical reasons, the algorithm uses the object_id, not the time of submission.

Here it is.

Determine the object_id of the latest posted image

Let's call this number U. It's an integer, and this number is steadily approaching the 1 000 000 mark.

Repeat
Select a random number
R between 1 and U (see below).
Until object
R refers to an image and has a score that exceeds some predetermined threshold value


The key question is how to select that random number R between 1 and U, such that the probability of R decreases somehow for lower values. There are actually many ways to do that, but after a bit of trial and error I got one that I believe works quite well.

Let X be a random number from the interval [0,1> taken from a uniform distribution. Most programming environments have an easy way to get such a number. In MS Excel, which I used for my trials and errors, it's the RAND() function.

Now, calculate R with the formula

U + 1 - floor(exp(X^0.7 * ln(U + 1))

For the record, the floor function rounds down to the nearest integer.
The exponent 0.7 in the formula is more or less arbitrary. I tried a few numbers, and this one worked satisfactory. To increase the probability for selecting older images, simply use a somewhat lower value.

Now how does this work out in practice?

With the highest object id currently around 835 000, and before applying the litmus test to see if it is an image with a high enough score:
  • 25% of the numbers are from the 174 most recent objects
  • 25% from the next 4244
  • 25% from the next 65056
  • 25% from the oldest 766091

A different way to look at it is to calculate the probabilities for a given object to be selected (again, before applying the test).
  • The 10th most recent object has a probability of 0.5%
  • The 100th one has a probability of 0.06%
  • The 1 000th one has a probability of 0.008%
  • The 10 000th one has a probability of 0.009%
  • The 100 000th one has a probability of 0.001%

As for what threshold value to use, that's a matter of personal opinion as well as trial and error to see what works well. I think anything between 75% and 80% will work. With a low threshold we get to see more images that do not have a really high score yet, regardless of whether they have been overlooked or are simply not good enough for a higher score. With a high threshold we only get to see images that have already gathered a lot of (high) votes. It's just a matter of deciding what kind of images we want as POTM. I prefer a somewhat lower threshold, so good images that somehow have been overlooked by many can still pass the test. But we could start somewhere in the middle and see how it goes.


I expect that this would be be real easy to implement. So, the question is, do we want this?

User Avatar
yatsek

 
Posts: 919
Joined: Wed Aug 27, 2008 4:04 pm
Thanked: 65 times in 50 posts

Re: Changes to Voting System

by yatsek » Mon Jan 28, 2013 4:21 pm

Montana Matt wrote:
Redwic wrote:Technically speaking, I guess making comments on SP is making a "contribution" to SP. Just a different type of contribution.

Exactly. Members who comment regularly (and provide good/helpful/critical feedback) are extremely valuable.

OK but this kind of contribution doesn't need to be rewarded with power points. Note that one can downvote a worthless page whereas one cannot downvote a worthless comment.

User Avatar
rgg
Forum Moderator
 
Posts: 859
Joined: Sat Oct 02, 2010 7:15 pm
Thanked: 192 times in 154 posts

Re: Changes to Voting System

by rgg » Mon Jan 28, 2013 8:50 pm

Montana Matt wrote:I like the new idea for POTM rgg. There is one small problem with the algorithm though. Repeatedly querying the database while looking for the following:
Until object R refers to an image and has a score that exceeds some predetermined threshold value
Would be extremely costly in terms of time. Putting a mysql inside of a loop is a big "no no"


I thought about that right after I posted, but then I decided not to go into the performance of the algorithm just yet - the idea might not catch on in the first place.

What I thought was to have an independent process on the server that runs periodically to select an image, put it somewhere handy and then use that same image for all clients requesting a front page, until the process runs again. That way it's much less important if the process takes a few seconds because it's completely separate from loading the front page. The faster the process proves to be, the more often it can be run. That would be a simple matter of experimenting a bit.


Montana Matt wrote:The other option is to select out 1000 or 10,000 images all at once (between two, evenly-spaced, random numbers) and loop through the images until one is found that meets the criteria. This would still be somewhat costly, and would slow down the load time of the front page, but far less so than repeatedly querying the database.


I would have to think about how that would change the probability distribution. My gut feeling is that the change would be pretty substantial, and not necessarily in a good way.


But to avoid going back to the database over and over again, how about this:

Start by selecting the object_id's for all images with a score above the threshold, in ascending order.

Now, consider all the rows in this result set numbered 1 through U. The big difference is that every row number represents a valid image this time, so after using the same random sampling formula as before to determine R (but using this lower value U!) there is no need for looping anymore!

Again this could be done in an independent process that runs periodically and stores the selected image, ready whenever a client requests a front page. I expect this to be fast enough, but if not, this could simply be done a little bit less frequently.

If this is still not good enough (which would really surprise me), a further option might be to make a background process that keeps the result set of the select statement in memory, and periodically produces a randomly selected object_id using that same set. Restart the background process occasionally, say every few hours, to reflect the changes in picture scores (in practical terms that's only relevant for recent images).

Just a few ideas...

PreviousNext

Return to Site Feedback

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 0 guests