Welcome to SP!  -
Areas & RangesMountains & RocksRoutesImagesArticlesTrip ReportsGearOtherPeoplePlans & PartnersWhat's NewForum

Changes to Voting System

Suggestions and comments about SummitPost's features, policies, and procedures. Post bugs here.
 

Re: Changes to Voting System

Postby Josh Lewis » Sun Jan 13, 2013 6:33 pm

Bubba Suess wrote:If the desire is there to simplify the voting then a like/dislike system could work if a third option was 'abstain' or something like that. Obviously that may not be the best name for it but it works for the discussion. Voting 'abstain' would not change the score but communicate to the page builder that someone has the opinion the page is coming along but needs to be improved. One could even install a mechanism that would notify someone to review their 'abstain' vote after so much time has elapsed. If the page has not improved then one can dislike it. If it has improved, one can like it.


Great point. This is in the direction I was proposing earlier. A system to push folks in the right direction when a page is alright or of lesser quality. There are a lot of great contributions to SP, however I've seen some good mountains that deserve a lot better of a job done to them. I wouldn't hesitate to use the button Bubba proposed (I'd think about it of course).

MarkDidier wrote:Page quality improves via communication with the submitter. I would recommend that for any Dislike, the voter is forced to submit a comment (or PM). Giving a submitter constructive criticism and recommendations is what will improve page quality. If the submitter does nothing, then the issue could easily be escalated to the elves for action from there. This could easily be accomplished by having the Comment box pop up with a Dislike.


Great post Mark! You hit the nail on this one. I wouldn't mind publicly saying why I thought a page needed improvements. 8) If I was worried about my reputation, I wouldn't get involved with the politics of SP. :wink:

Montana Matt wrote:Again, if someone can provide me with a mathematical equation for a system where you count a 10 vote and calculate a total score based on it, then you count a 7 vote and calculate a total score based on the 7 and the 10, and that total score isn't lower than it was when there was only a 10, I'd be happy to try implementing and testing it.


I'm pretty sure there is a way to deal with this. Make 5/10 neutral, less than 5/10 negative, and everything above 5/10 positive. In some sense TrekNature pulled this off. In this example someone voted 1/2 on my photo. This did not bring down the score. They don't have any negative points, but I'm sure they could have. With a 6/10 you could have it designed so that it's not the actual vote weight that is given, but is the one that we (SP users) see. Behind the scenes it could add the appropriate number based on your vote weight (it would be minor in this case). If this kind of a system was made, a 4/10 would do little to bring down the score. That would be a good thing! This would make it possible to down vote without being "too mean". And of course if the content is actually of poor quality you would vote less than 4/10 which would bring down the score to what it should be according to the votes.
My Websites: Alpine Josh · Alpine Ascent · AceMaps
User Avatar
Josh Lewis

 
Posts: 2275
Joined: Thu Jan 06, 2011 11:12 pm
Location: Lynnwood, Washington, The Cloudiest Place on Earth, United States
Thanked: 472 times in 323 posts

Re: Changes to Voting System

Postby Bubba Suess » Sun Jan 13, 2013 6:56 pm

MarkDidier wrote:And as for page scores, I take no stock in them and personally do not think they are necessary. The value of the beta submitted lies in mind of the user!

The value of the beta on a page is subjective whether the page is good or bad. The quality of a page, on the other hand, is an objective standard. Consequently there are good reasons to maintain the use of the page score. As long as only one page is permitted per peak (a good thing) the score helps measure whether the pages are good or bad. If SP is to have any quality standards there must be a mechanism for that standard to be sustained. To do this, one can either have admin nuke bad pages on their own, which is labor intensive on the elves part or permit the SP community to sink bad pages. The score permits the latter and when something sinks low enough then it is brought to the elves attention and then, usually, deleted. The mechanism for the voting is obviously something that can be improved, hence this thread, but doing away with the page score dismantles the means by which the general SP community polices the site.
User Avatar
Bubba Suess

 
Posts: 697
Joined: Tue Oct 04, 2005 9:15 pm
Location: Mount Shasta, California, United States
Thanked: 173 times in 97 posts

The following user would like to thank Bubba Suess for this post
Josh Lewis, yatsek

Re: Changes to Voting System

Postby Marmaduke » Sun Jan 13, 2013 6:59 pm

It seems like this is something that can be fixed without making it into solving a world peace isssue. Why should there be a dislike option, that would just bring on problems. If there is a photo or a page that needs work a vote of 2 or 3 let's say should still give a positive increase to the score but is sending a messsage that more work is needed. There are many pages that are good pages but not great, those pages deserve a 6 or 7 vote let's say but again the page score should go up not down. Keep it a 1 - 10 with even a 1 vote increasing the score but obviouslsy just ever so slighting with each number being weighted with a gradual increase of the percentage.
User Avatar
Marmaduke

 
Posts: 1325
Joined: Tue Feb 02, 2010 1:08 am
Location: Sonoma, California, United States
Thanked: 578 times in 444 posts

Re: Changes to Voting System

Postby Josh Lewis » Sun Jan 13, 2013 7:06 pm

@Marmaduke: Problem with that idea is that popular mountains would for sure be the top rated pages. That's some what the case right now, but at least well made pages on obscure places have a chance. In the system your proposing, a bad quality but famous mountain would have a much greater score than some of the best work posted here on SP. Trust me, this would happen.

On the bright side that system would create far less complains and less drama. :wink: But it does not have enough score integrity. That is the entire reason for having negative votes.
My Websites: Alpine Josh · Alpine Ascent · AceMaps
User Avatar
Josh Lewis

 
Posts: 2275
Joined: Thu Jan 06, 2011 11:12 pm
Location: Lynnwood, Washington, The Cloudiest Place on Earth, United States
Thanked: 472 times in 323 posts

Re: Changes to Voting System

Postby Scott » Sun Jan 13, 2013 7:22 pm

PERSONALLY I VOTE ONLY WHEN I LIKE, WHEN I DON'T LIKE I RATHER PREFER TO GIVE NO VOTES,


Sorry, but that's the whole problem. Most people only vote 10/10 on everything and everyone is afraid to give a "bad" vote on a page that needs work. It shouldn't be this way. Bad pages should be voted on, hopefully fixed by the maintainer (at which time the vote is raised), and if still poor quality after a long period of time, deleted.
User Avatar
Scott

 
Posts: 7452
Joined: Thu Aug 21, 2003 1:03 pm
Location: Craig, Colorado, United States
Thanked: 552 times in 308 posts

The following user would like to thank Scott for this post
Josh Lewis

Re: Changes to Voting System

Postby Bob Sihler » Sun Jan 13, 2013 8:55 pm

Montana Matt wrote:But the problem is still the following: Someone votes a 10 on a page (that's all anyone votes now). The score is calculated based on that 10. If someone else comes along and votes a 7, how could that NOT lower the score? Again, if someone can provide me with a mathematical equation for a system where you count a 10 vote and calculate a total score based on it, then you count a 7 vote and calculate a total score based on the 7 and the 10, and that total score isn't lower than it was when there was only a 10, I'd be happy to try implementing and testing it.


Matt, it isn't so much that a 7 lowers the score but that it lowers it far out of proportion to how voting 10 raises it.

Many of us are used to the 10-point academic scale, so when we think of 8, we think of 80%, a B or B-, for example. But I just went and test-voted 8 on an image with no votes (and then canceled the vote), and the score was 63.52%, what many of us think of as a D or below average. I think the voting system would have worked much better had votes corresponded to the 10-point scale and page scores been calculated as averages. I realize the point of weighting vote power based on power points was a means to combat avatars and trolls, but it also had the effect of producing what many of us are accustomed to thinking are bad scores when we voted numbers that we normally would expect to produce good scores. There are many pages that are indeed decent or pretty good that deserve a 7 or 10, but I just don't vote on them at all because the number that results looks pretty brutal.

Since that system is hopelessly broken, maybe the best thing now really is to try the like/dislike idea and see how it goes, though I agree it might be nice to consider something like an "Okay" option as well.
Bob's Law: The rationality of a person is inversely proportional to the number of political and religious bumper stickers on his car. Around bumper-sticker warriors, smile, speak calmly, and get the hell away as fast as possible.
User Avatar
Bob Sihler
Forum Moderator
 
Posts: 7295
Joined: Mon Aug 23, 2004 3:20 pm
Location: Herndon, Virginia
Thanked: 1818 times in 1029 posts

Re: Changes to Voting System

Postby Montana Matt » Sun Jan 13, 2013 9:56 pm

awilsondc wrote:I think this is what needs to happen. Either we go with the like/dislike system, or someone comes up with a new algorithm for the 0-10 system. The reason you don't get more 5-9 votes is that even though the person is trying to say hey, your work is pretty good but not perfect, the page score gets lowered. Lowering a page score is seen as a negative thing even though you were trying to say something positive. The new algorithm should improve a score if if the vote is 6-10 (by varying degrees) and lower the score if the vote is 1-5 (by varying degrees).

Josh Lewis wrote:I'm pretty sure there is a way to deal with this. Make 5/10 neutral, less than 5/10 negative, and everything above 5/10 positive.

I want to say that I hear what many of you desire to see with the voting system. But someone needs to provide an algorithm or an equation for calculating the total score where a 6-10 vote ALWAYS improves the score and 1-5 vote ALWAYS lowers the score REGARDLESS of what the current score is...if that's even possible. I actually have a masters degree in Math, so I'm burned out on that sort of thinking :) But I'd be happy to implement any algorithm or equation someone comes up with and test it out on the new server SP will be migrating to.

For argument's sake, suppose we do come up with some sort of calculation that would function as people desire with a voting scale from 1 to 3 or 1 to 5 or 1 to 10. Would people really use the entire scale? I doubt it since no one did with the transition to SPv2 (if they had, we wouldn't be having this discussion). And what do we do with the 4.1 million votes in the system that are 10 if we transition to a new system. Assuming people actually did use the scale as intended, voting the full range from 1 to 10, all of the pages that have so many historical votes of 10 would have a significantly skewed score if the votes were pulled into the new system. Do you delete all of the historical votes and start over? Doesn't sound like a good idea to me...
Bruno wrote:Personally, I would almost never vote "Dislike"

AlbertoRampini wrote:PERSONALLY I VOTE ONLY WHEN I LIKE, WHEN I DON'T LIKE I RATHER PREFER TO GIVE NO VOTES

I think that is the way over 96% of the people vote with the current system (10 if they like it, abstain if they don't). Perhaps all we need is a like button. And if one thinks the page needs work, then they should submit a comment detailing what they think needs to be added/corrected and abstain from "liking" the page until the person improves it to the point where you think it is worth "liking."

Then we could do away with score completely and just sort pages by the number of likes instead of a complex score calculation. That's probably not too far off from the results generated by the extremely complicated equation currently implemented for calculating scores right now since nearly all of the votes are 10. In which case, is any change warranted? Perhaps everyone can just keep voting 10 if they like a page and not voting if they don't ;)
Josh Lewis wrote:In some sense TrekNature pulled this off. In this example someone voted 1/2 on my photo. This did not bring down the score.

I can't see any votes or scores on that page. Can you point me to exactly where those things are? Maybe I need to have an account and be logged in to see them?...Just created an account and still can't see the score or votes...

Anyway, without knowing how their algorithm works, you can't assume that they have something along the lines of what you're proposing. They may have implemented something similar to what SP currently does where outlying votes (votes too far away from the standard deviation) are thrown out or they have some other way of calculating the score.
Scott wrote:Most people only vote 10/10 on everything and everyone is afraid to give a "bad" vote on a page that needs work. It shouldn't be this way. Bad pages should be voted on, hopefully fixed by the maintainer (at which time the vote is raised), and if still poor quality after a long period of time, deleted.

Right! That's they way the SPv2 voting system was intended to work. But the problem (as I understand it, at least) is that almost no one is comfortable voting anything other than a 10. The current voting system would work perfectly if people actually voted on a scale from 1 to 10, but they don't. Since almost no one (less than 3% of voters) is voting less than 10, I don't see the point in having anything other than a like button.

I'm fairly certain I can come up with a way to calculate some sort of "score" based on the number of likes a page has in comparison to other pages. And factor in the power of the user who "likes" the page to make more a powerful user's "like" count for more than newly created, less powerful users (as is the case right now). I think I'll set up such a system on the new server and see if it yields reasonable results.
Bob Sihler wrote:Many of us are used to the 10-point academic scale, so when we think of 8, we think of 80%, a B or B-, for example. But I just went and test-voted 8 on an image with no votes (and then canceled the vote), and the score was 63.52%, what many of us think of as a D or below average. I think the voting system would have worked much better had votes corresponded to the 10-point scale and page scores been calculated as averages.

The 10-point academic scale thinking probably is the root of the caused failure (if you want to call it that. I'd argue that it still works in some capacity) of the current SP voting system. People believe that a page score of 50% is horrible or failing, when in reality it should be representative of an average (on order with C) page. 40% is just below average (on order with C-), 60% would be above average (on order with a C+). Of course, that's assuming that if one took the 4.3 million votes in the system and averaged them, they'd average out to be about a 5. But the average of them is 9.88 with a standard deviation of only 0.7!
Bob Sihler wrote:I realize the point of weighting vote power based on power points was a means to combat avatars and trolls, but it also had the effect of producing what many of us are accustomed to thinking are bad scores when we voted numbers that we normally would expect to produce good scores.

Matt, it isn't so much that a 7 lowers the score but that it lowers it far out of proportion to how voting 10 raises it.

I just went through the voting code and reacquainted myself with the algorithm. The weighting of the vote power doesn't really have a significant effect on the value you see for the score. If you were Dow (the user with the most power points in the system at 5623 and 2600+ points more than you), and voted an 8 on that image, the score would have been 63.31% instead of your 63.52%. When I vote an 8 on an image with no votes, it generates a score of 66.37% and I have only 89 power points. When the system first came online, a vote of 8 would have yielded vastly different results; probably yielding a score closer to 80% or 90%.

PAY CLOSE ATTENTION HERE BECAUSE THIS IS WHAT MOST PEOPLE AREN'T UNDERSTANDING: But now that the average vote in the system is 9.88 and the standard deviation is only 0.7, a vote of 8 looks pretty bad in comparison, so a lower score is generated. If the average vote in the system were a 5 and the standard deviation were 2 (which I'd guess to be somewhat accurate if people voted fairly), a vote of 8 would yield a score of 92.27%.

Most people's voting weight is very, very similar, especially once they get 10 or more power points. The "lower-than-expected" score is due to the way people are expecting the scores to be calculated. Scores were intended to spread evenly from 0% to near 100% and to be used primarily for the purpose of ranking pages (not making people feel bad). A score of 50% isn't a "bad" score. It just indicates that the page is average and could be improved. In hindsight, it would have been better to base the "score" on some other sort of scale. Maybe have called something like "Power Score" and spread it from 0 to 1000 or something would have prevented this from happening. But maybe not...

Anyway, as I mentioned above, I think I'll try to implement a "like" button, change all 10 votes to "like", drop the others and come up with some sort of equation to calculate score based on vote weight and number of "likes."
User Avatar
Montana Matt
Site Admin
 
Posts: 1135
Joined: Sun Apr 11, 2004 10:06 pm
Location: Ashland, Oregon, United States
Thanked: 340 times in 184 posts

The following user would like to thank Montana Matt for this post
Josh Lewis, Matt Lemke

Re: Changes to Voting System

Postby Bob Burd » Sun Jan 13, 2013 10:11 pm

Perhaps what people are looking for is an open-ended scoring system, rather than the current 0-100%. Score would then be something like: sum(weight1*vote1, weight2*vote2,etc)
If a like/dislike were used, a like vote might be +1, a dislike vote -1.
But perhaps the 0-10 scale could be retained since any vote will now add to the score.
The goal is still the same - trying to sort good from bad. Currently, the page with the most 10 votes pops up on top. The above scoring would pretty much do the same, I think.
It's obvious when a page isn't good even if people don't vote because it has only a few votes. So it would have a lower score in either system.
I'm not sure what the advantage is of keeping scores in the 0-100% range. Does it do a better job of sorting for the very best somehow?

btw, I think weighting should be a significant factor. It offers some sort of quality check and keeps the creation of fake avatars for the sole purpose of voting. Having a waiting period on new members weighting may only delay this. I think some sort of participation (it could even be in the forums) should be a factor, aside from just voting.
User Avatar
Bob Burd
Forum Moderator
 
Posts: 4098
Joined: Sun Jul 22, 2001 10:42 pm
Location: San Jose, California, United States
Thanked: 458 times in 237 posts

Re: Changes to Voting System

Postby Bubba Suess » Sun Jan 13, 2013 10:20 pm

I don't know if my comments are helpful here, but I will go ahead and reiterate them anyway. I think the like/dislike option would work if there was a middle of the road choice. The middle choice need not be used to determine the score, but would simply be a marker indicating that the page needs to be improved. That is why I called it "abstain". In a sense, someone is not voting per se but indicating an opinion regarding quality that is in a specific point in a page's development.

There seems to be a consensus that some mid-level vote is desired, be it a third option or the current 10 point scale that exists, so finding some system that incorporates this would be key.

The 'abstain' (or whatever you want to call it) choice would simplify things since it would not have to be included in calculating the page score but would still give people the option of expressing a desire to see the page improved.
User Avatar
Bubba Suess

 
Posts: 697
Joined: Tue Oct 04, 2005 9:15 pm
Location: Mount Shasta, California, United States
Thanked: 173 times in 97 posts

The following user would like to thank Bubba Suess for this post
chugach mtn boy, Silvia Mazzani, yatsek

Re: Changes to Voting System

Postby Bubba Suess » Sun Jan 13, 2013 10:21 pm

Bob Burd wrote:I think weighting should be a significant factor. It offers some sort of quality check and keeps the creation of fake avatars for the sole purpose of voting. Having a waiting period on new members weighting may only delay this. I think some sort of participation (it could even be in the forums) should be a factor, aside from just voting.

I concur.
User Avatar
Bubba Suess

 
Posts: 697
Joined: Tue Oct 04, 2005 9:15 pm
Location: Mount Shasta, California, United States
Thanked: 173 times in 97 posts

The following user would like to thank Bubba Suess for this post
Silvia Mazzani

Re: Changes to Voting System

Postby Scott » Sun Jan 13, 2013 10:25 pm

I think weighting should be a significant factor. It offers some sort of quality check and keeps the creation of fake avatars for the sole purpose of voting. Having a waiting period on new members weighting may only delay this. I think some sort of participation (it could even be in the forums) should be a factor, aside from just voting.


I agree too.
User Avatar
Scott

 
Posts: 7452
Joined: Thu Aug 21, 2003 1:03 pm
Location: Craig, Colorado, United States
Thanked: 552 times in 308 posts

Re: Changes to Voting System

Postby Montana Matt » Sun Jan 13, 2013 10:39 pm

Bob Burd wrote:Perhaps what people are looking for is an open-ended scoring system, rather than the current 0-100%. Score would then be something like: sum(weight1*vote1, weight2*vote2,etc)

I think you may be right Bob. And we already do that calculation on the way to calculating the score (that sum of weighted votes is used in the calculation), so it wouldn't be hard to go that route. Then the score of a page would be similar to the way a user's power works now, with no finite end.
Bob Burd wrote:The goal is still the same - trying to sort good from bad. Currently, the page with the most 10 votes pops up on top. The above scoring would pretty much do the same, I think.

Yes, at this point it would be pretty much identical. As you say, the page with the most 10 votes is on top, followed by the page with the next most 10 votes, etc. So maybe the "score" of the page could simply be the sum of the weighted votes...that would be easy enough to implement and wouldn't require as significant of a change to the database, code and HTML.
Bob Burd wrote:I'm not sure what the advantage is of keeping scores in the 0-100% range. Does it do a better job of sorting for the very best somehow?

No. I'm not sure why we decided on the % at the end, but we decided to map it to 0-100. The score calculation yields a number between 0 and 1. I guess we assumed that most people would rather see a whole number than a decimal, so we multiply the score by 100 to get something between 0 and 100.
Bob Burd wrote:btw, I think weighting should be a significant factor. It offers some sort of quality check and keeps the creation of fake avatars for the sole purpose of voting. Having a waiting period on new members weighting may only delay this. I think some sort of participation (it could even be in the forums) should be a factor, aside from just voting.

I agree as well. I intend to keep the weighting in place as it is now.
User Avatar
Montana Matt
Site Admin
 
Posts: 1135
Joined: Sun Apr 11, 2004 10:06 pm
Location: Ashland, Oregon, United States
Thanked: 340 times in 184 posts

Re: Changes to Voting System

Postby Josh Lewis » Sun Jan 13, 2013 10:42 pm

Montana Matt wrote:I can't see any votes or scores on that page. Can you point me to exactly where those things are? Maybe I need to have an account and be logged in to see them?...Just created an account and still can't see the score or votes...

Anyway, without knowing how their algorithm works, you can't assume that they have something along the lines of what you're proposing. They may have implemented something similar to what SP currently does where outlying votes (votes too far away from the standard deviation) are thrown out or they have some other way of calculating the score.


So on that photo example there is a box on the left below the photo called "Photo Information". At the very bottom of that box it says "Points: 5". Most folks vote 2/2, but every now and then someone votes a 1/2. This is great! That means folks are being completely honest with their votes. Odd votes are not thrown out on TrekNature. That site is a great role model not just in the vote system, but how comments are supposed to be conducted. The site is built on being a critique. This was beneficial to me personally, people were willing to approaching me and tell me exactly what was wrong with my photos. They were nice about it too. We need more of that going on in SP. I personally don't settle for "the page is good enough". :wink:

Regarding Bubba's "abstain" idea: I think it's a great idea. I personally would like to see it. It may effect the how people vote, but it wouldn't actually be apart of the voting system. Right? (if this was actually added)
My Websites: Alpine Josh · Alpine Ascent · AceMaps
User Avatar
Josh Lewis

 
Posts: 2275
Joined: Thu Jan 06, 2011 11:12 pm
Location: Lynnwood, Washington, The Cloudiest Place on Earth, United States
Thanked: 472 times in 323 posts

Re: Changes to Voting System

Postby Bob Sihler » Sun Jan 13, 2013 10:47 pm

Scott wrote:
I think weighting should be a significant factor. It offers some sort of quality check and keeps the creation of fake avatars for the sole purpose of voting. Having a waiting period on new members weighting may only delay this. I think some sort of participation (it could even be in the forums) should be a factor, aside from just voting.


I agree too.


And I.
Bob's Law: The rationality of a person is inversely proportional to the number of political and religious bumper stickers on his car. Around bumper-sticker warriors, smile, speak calmly, and get the hell away as fast as possible.
User Avatar
Bob Sihler
Forum Moderator
 
Posts: 7295
Joined: Mon Aug 23, 2004 3:20 pm
Location: Herndon, Virginia
Thanked: 1818 times in 1029 posts

Re: Changes to Voting System

Postby Bob Sihler » Sun Jan 13, 2013 10:50 pm

Matt, Dow's 8 is not much different from my 8, but compare my 1 to a new member's 1. It is a massive difference. You're right that the academic scale that most of us are used to is a big part of the dissatisfaction with the way SP's voting system works. You either have to fix it to align with what people are accustomed to or throw it out and try something else. And since it seems you and most others here want to try something else, let's enjoy the ride!
Bob's Law: The rationality of a person is inversely proportional to the number of political and religious bumper stickers on his car. Around bumper-sticker warriors, smile, speak calmly, and get the hell away as fast as possible.
User Avatar
Bob Sihler
Forum Moderator
 
Posts: 7295
Joined: Mon Aug 23, 2004 3:20 pm
Location: Herndon, Virginia
Thanked: 1818 times in 1029 posts

The following user would like to thank Bob Sihler for this post
Josh Lewis, Montana Matt

PreviousNext

Return to Site Feedback

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 0 guests

© 2006-2013 SummitPost.org. All Rights Reserved.