Welcome to SP!  -
Areas & RangesMountains & RocksRoutesImagesArticlesTrip ReportsGearOtherPeoplePlans & PartnersWhat's NewForum

Changes to Voting System

Suggestions and comments about SummitPost's features, policies, and procedures. Post bugs here.
 

Re: Changes to Voting System

Postby Josh Lewis » Sun Jan 13, 2013 11:33 pm

For those who have seen my profile, they would know that I don't care about my power points. After all I have "Unlimited". :lol: :wink: Anyways, I like most of the ideas presented in this thread. The like one seems to be the best suit for everyone. Lets roll. 8)
My Websites: Personal Website · PeakBeta · AceMaps
User Avatar
Josh Lewis

 
Posts: 2776
Joined: Thu Jan 06, 2011 11:12 pm
Location: Lynnwood, Washington, The Cloudiest Place on Earth, United States
Thanked: 673 times in 434 posts

Re: Changes to Voting System

Postby rgg » Sun Jan 13, 2013 11:46 pm

Almost immediately after I joined I saw that there was rarely a vote other than 10. Rather than trying to use the whole scale, I followed suit, because despite the flaws of the system, I saw the distorting effect that votes of 8 and 9 had on some good pages.

I think it would be an improvement to discard the scale in favor of Like/Dislike. People are familiar with that. I still expect very few dislikes, as most people will simply refrain from voting if they don't like something, but I think that the option of saying that a page is bad would be useful. I see no merit in having more than two options. Too many choices leads to the same problems as with the 1-10 scale; most people won't use them.

As for calculating page scores, I'm against an open scale. As pointed out earlier, that would simply lead to popular peaks and routes having a higher rating, but it wouldn't reflect the quality of the page itself. Even now, popular peaks typically get more votes and hence higher scores than unknown ones, but with a 0-100% scale, the effect is not dramatic.

I would suggest all low (1 thru 5) votes are converted to Dislike, votes 9 and 10 to Likes and the others discarded. After that, the current page score calculation, wouldn't even have to be changed significantly: just substitute a 10 again for every Like, and a 1 (or perhaps a 2 or 3) for every Dislike and keep everything else the same.
While I believe this will lower the score for almost everything, I don't see any problem with that. Scores were supposed to help the best stuff rise to the top, so they are relative anyway, right?
User Avatar
rgg

 
Posts: 553
Joined: Sat Oct 02, 2010 7:15 pm
Location: Amsterdam, Netherlands
Thanked: 135 times in 105 posts

Re: Changes to Voting System

Postby Bob Burd » Mon Jan 14, 2013 12:20 am

Regarding comments about an "abstain" option.
I figured not voting at all would be the same (score-wise, it would), but I can see that there is some additional feedback provided to the submitter - it shows that someone has looked on it with at least some criticality (page hits would give you an idea as well, but doesn't mean that someone actually spent any time looking at it). Maybe instead of "dislike", it's just a "needs work" vote which has zero score effect instead of negative. I don't think having dislike negative scores will change the over rankings much anyway - something with no likes is in the same boat as a page with a few dislikes.

So, maybe "like/needs work" as the two options.

Secondly, it might be good if the "needs work" votes sort of dissolved over time, maybe 6-12 months. That way, no one has to go begging for old votes to be removed/changed. It wouldn't change the rankings much since pages still need lots of "like" votes to score high. I imagine some folks might like a "please revisit" button that can be used to ping "needs work" voters to reconsider if work has been done to improve it. I could see this being abused, but it might be worth a try.

Thirdly, things garner more votes when sitting in the "What's New" page, so it becomes important to some, *very* important to others. I think votes that come later after some one has stumbled upon a page or picture and taken the time to vote are more valuable as they're not just reacting to the newness factor. The voter's not just piling on when a page first gets submitted. I think there might be more weighting given to votes that come at a later time. Maybe cut it off at some point, like a year or two to keep the very oldest pages from having an advantage with this.

Ok, lastly (for now), perhaps add an inverse weighting based on the number of summit logs (or perhaps page hits), with the intent to keep pages like Rainier and Everest from popping to the top based solely on popularity.
User Avatar
Bob Burd
Forum Moderator
 
Posts: 4161
Joined: Sun Jul 22, 2001 10:42 pm
Location: San Jose, California, United States
Thanked: 487 times in 251 posts

Re: Changes to Voting System

Postby lcarreau » Mon Jan 14, 2013 12:30 am

So ..... when's all this supposed to go into effect :?:
User Avatar
lcarreau

 
Posts: 4059
Joined: Thu Sep 06, 2007 10:27 pm
Location: Court of the Crimson King, Arizona, United States
Thanked: 1084 times in 812 posts

Re: Changes to Voting System

Postby Marmaduke » Mon Jan 14, 2013 1:04 am

The Chief wrote:Why am I not allowed to see/participate in this Voting Thread?

"You are not authorised to read this forum."


You are registered as a Dem and a Repub, so they banned you from voting. Cheater! :wink: :wink:
User Avatar
Marmaduke

 
Posts: 1378
Joined: Tue Feb 02, 2010 1:08 am
Location: Sonoma, California, United States
Thanked: 622 times in 478 posts

Re: Changes to Voting System

Postby Bubba Suess » Mon Jan 14, 2013 1:05 am

Bob Burd wrote:Regarding comments about an "abstain" option.
I figured not voting at all would be the same (score-wise, it would), but I can see that there is some additional feedback provided to the submitter - it shows that someone has looked on it with at least some criticality (page hits would give you an idea as well, but doesn't mean that someone actually spent any time looking at it). Maybe instead of "dislike", it's just a "needs work" vote which has zero score effect instead of negative. I don't think having dislike negative scores will change the over rankings much anyway - something with no likes is in the same boat as a page with a few dislikes.

So, maybe "like/needs work" as the two options.

I disagree. I think that there needs to be some mechanism for a negative vote to be registered. Pages that are just bad need to just get bad votes. If the 'initial construction period' passes and the page is still bad, it needs to get dinged.

Secondly, it might be good if the "needs work" votes sort of dissolved over time, maybe 6-12 months. That way, no one has to go begging for old votes to be removed/changed. It wouldn't change the rankings much since pages still need lots of "like" votes to score high. I imagine some folks might like a "please revisit" button that can be used to ping "needs work" voters to reconsider if work has been done to improve it. I could see this being abused, but it might be worth a try.

In my model with the 'abstain' choice, those that have pressed this button ought, once having done so, receive a reminder to revisit the page after a certain amount of time has elapsed. I think we have the something similar in mind here.

Thirdly, things garner more votes when sitting in the "What's New" page, so it becomes important to some, *very* important to others. I think votes that come later after some one has stumbled upon a page or picture and taken the time to vote are more valuable as they're not just reacting to the newness factor. The voter's not just piling on when a page first gets submitted. I think there might be more weighting given to votes that come at a later time. Maybe cut it off at some point, like a year or two to keep the very oldest pages from having an advantage with this.

I think this is a really good idea. Could this be installed retroactively, i.e for votes cast in 2011 on pages created in 2007?
User Avatar
Bubba Suess

 
Posts: 715
Joined: Tue Oct 04, 2005 9:15 pm
Location: Mount Shasta, California, United States
Thanked: 181 times in 103 posts

Re: Changes to Voting System

Postby Bob Burd » Mon Jan 14, 2013 1:17 am

Montana Matt wrote:
Bob Burd wrote:So, maybe "like/needs work" as the two options. Secondly, it might be good if the "needs work" votes sort of dissolved over time, maybe 6-12 months. That way, no one has to go begging for old votes to be removed/changed.

If I understand you correctly, the "needs work" wouldn't actually affect the score, right? Is there really a need for this "needs work" button. Couldn't people just leave a note on the additions/corrections or a comment? I mean, fewer than 3% of people actually vote anything other than a 10. Seems like an unnecessary (and likely mostly unused) complication to have another option for that 3% of the time when there is already a fully functional way of giving people details (instead of a non-descriptive vote) about why you aren't voting on their page.


It was brought up that if this second option didn't reduce a score, more folks would use it, so it should become more than just 3%. It give the feedback that basically says, "I've looked at your page and it needs some work". Without it, a person might feel like nobody saw their page because it was submitted at night or whatever. Of course anyone could leave comments or an addition/corrections, but that rarely happens because it takes time beyond the average attention/care span. Let's face it, the easier and quicker the feedback mechanism, the more it will get used.

Montana Matt wrote:
Bob Burd wrote:Thirdly, things garner more votes when sitting in the "What's New" page, so it becomes important to some, *very* important to others. I think votes that come later after some one has stumbled upon a page or picture and taken the time to vote are more valuable as they're not just reacting to the newness factor. The voter's not just piling on when a page first gets submitted. I think there might be more weighting given to votes that come at a later time. Maybe cut it off at some point, like a year or two to keep the very oldest pages from having an advantage with this.

That's an interesting idea. The time of the vote is already saved in the database, as is the time of creation of a page. Something could be included in the weighting to include the time the vote was cast relative to the age of the page. So, for example, perhaps votes that occur between 0 and 30 days after creation get weighted by 0.75, votes between 30 and 500 days get weighted 1.0 and votes after 500 days get weighted 0.75? Though I prefer the idea of weighting based on page hits over this.


This wasn't suggested as a way to unweight popular pages. That was the next suggestion. This was a way to make all those initial votes garnered because they're on the What's New page a little less valuable than those that come later, usually with more thought and heart behind them. Something like 0.5 weight for first week, 0.75 up to 3 months, 1.0 beyond that, was closer to what I had in mind.

Montana Matt wrote:
Bob Burd wrote:Ok, lastly (for now), perhaps add an inverse weighting based on the number of summit logs (or perhaps page hits), with the intent to keep pages like Rainier and Everest from popping to the top based solely on popularity.

That's another interesting idea.

I can certainly play around with these things and see what I can come up with. My biggest question at this point is how to actually determine and display the score based on the number of "likes." So that problem is two fold:
1) How to weight the votes? Power will definitely play a role, as it always has, but should we consider other things as well, such as Bob is proposing? I really like the idea of using the number of page hits to weigh down the vote for pages with a lot of hits.
2) How to calculate the actual page score? Would more people prefer to see a simple cumulative number, like power, or would more people prefer a percentage. If it were a percentage, it would probably be something like the
Code: Select all
($sum_votes/$sum_max_votes)*100

Where $sum_votes is the weighted sum of the votes for the current page and $sum_max_votes is the sum of the votes for page with the highest sum.


I think having an open-ended score will make more sense to more people. Let people see that their vote adds something substantive, even if only a few points rather than a few hundredths of a percent.
User Avatar
Bob Burd
Forum Moderator
 
Posts: 4161
Joined: Sun Jul 22, 2001 10:42 pm
Location: San Jose, California, United States
Thanked: 487 times in 251 posts

Re: Changes to Voting System

Postby Josh Lewis » Mon Jan 14, 2013 1:52 am

Bob Burd wrote:That was the next suggestion. This was a way to make all those initial votes garnered because they're on the What's New page a little less valuable than those that come later, usually with more thought and heart behind them. Something like 0.5 weight for first week, 0.75 up to 3 months, 1.0 beyond that, was closer to what I had in mind.


I definitely agree with this statement. :)
My Websites: Personal Website · PeakBeta · AceMaps
User Avatar
Josh Lewis

 
Posts: 2776
Joined: Thu Jan 06, 2011 11:12 pm
Location: Lynnwood, Washington, The Cloudiest Place on Earth, United States
Thanked: 673 times in 434 posts

Re: Changes to Voting System

Postby mrchad9 » Mon Jan 14, 2013 3:31 am

Matt- please give me two days to get a algorithm to you to consider. I've been thinking about this for some time and think I could propose something on how the page scores would be calculated that would clearly allow folks to use more of the voting scale comfortably.

I question the reasoning behind making scores go down much because votes are old. Voting on a page a year ago doesn't make it any worse today. Also this penalizes most of the types of peaks that are submitted today. Big name peaks were submitted long ago, and continue to get votes. But nowadays a new submission might get 20 votes the first two weeks and one or two over the next two years. This is also why I think using a vote to hit ratio could become problematic. Both using a ratio and time make more sense if users were forced to vote to some degree, but that has the potential to hinder ease of browsing.

Please give me till Monday or Tuesday night. I'd really like to see the full scale work and have an idea for the algorithm I just need to get the specific formula.
User Avatar
mrchad9

 
Posts: 4237
Joined: Thu Jul 02, 2009 12:01 am
Location: San Ramon, California, United States
Thanked: 1222 times in 829 posts

Re: Changes to Voting System

Postby mrchad9 » Mon Jan 14, 2013 4:17 am

Montana Matt wrote:
Bob Burd wrote:Perhaps what people are looking for is an open-ended scoring system, rather than the current 0-100%. Score would then be something like: sum(weight1*vote1, weight2*vote2,etc)

I think you may be right Bob. And we already do that calculation on the way to calculating the score (that sum of weighted votes is used in the calculation), so it wouldn't be hard to go that route. Then the score of a page would be similar to the way a user's power works now, with no finite end.
Bob Burd wrote:The goal is still the same - trying to sort good from bad. Currently, the page with the most 10 votes pops up on top. The above scoring would pretty much do the same, I think.

Yes, at this point it would be pretty much identical. As you say, the page with the most 10 votes is on top, followed by the page with the next most 10 votes, etc. So maybe the "score" of the page could simply be the sum of the weighted votes...that would be easy enough to implement and wouldn't require as significant of a change to the database, code and HTML.
Bob Burd wrote:I'm not sure what the advantage is of keeping scores in the 0-100% range. Does it do a better job of sorting for the very best somehow?

No. I'm not sure why we decided on the % at the end, but we decided to map it to 0-100. The score calculation yields a number between 0 and 1. I guess we assumed that most people would rather see a whole number than a decimal, so we multiply the score by 100 to get something between 0 and 100.
Bob Burd wrote:btw, I think weighting should be a significant factor. It offers some sort of quality check and keeps the creation of fake avatars for the sole purpose of voting. Having a waiting period on new members weighting may only delay this. I think some sort of participation (it could even be in the forums) should be a factor, aside from just voting.

I agree as well. I intend to keep the weighting in place as it is now.

Note however, that while an open ended scoring system accomplishes some of the goals, it lacks the ability to cast negative votes or decrease a page score. While this is infrequently used it is probably still good to incorporate. I'll have a algorithm to propose soon.
User Avatar
mrchad9

 
Posts: 4237
Joined: Thu Jul 02, 2009 12:01 am
Location: San Ramon, California, United States
Thanked: 1222 times in 829 posts

PreviousNext

Return to Site Feedback

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 0 guests

© 2006-2013 SummitPost.org. All Rights Reserved.