Vote Weight

Suggestions and comments about SummitPost's features, policies, and procedures. Post bugs here.
User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Vote Weight

by mrchad9 » Mon Feb 25, 2013 8:09 pm

I see a few folks putting more thinking into their votes on object pages now that we have a new voting system, which is great and provides good feedback to others.

But there are still a lot of members out there that vote 10/10 on everything, no matter how sparse the mountain page. These folks tend to put a 10 on a page if it is by a certain person, or meets a minimum criteria, and then just skip over the other pages if they feel a 10 is not warranted. Last week another member suggested that we might put in place a mechanism that would decrease the weight of votes by members who always voted with 10s, that way it would bring it more in line with the other members.

Basically we would have a mechanism that would automatically evaluate all the votes placed on objects, and if a certain percentage were 10s it could decrease the vote weighting for that member. So then vote weight would be based both on power points and voting history.

Just as an example with respect to how voting 10s on everything is not beneficial... last week I put an 8 on the page for Grinnell Mountain. The author is a brand new member, and this is his first page. The page is pretty good, but not perfect (actually the text is pretty damn good and at this point might be close enough to perfect... it looks like he has added to it). But the owner PM'd me just to ask what he could do to improve the page. We had a fairly productive discussion and I noted that with a few more pics and integrating a few into the page to should the mountain and route that he would not only have a higher vote from me, but also many more votes (if someone doesn't have pics on a page most of you that vote 10/10 on everything just skip it). This appears to be what has happened with his page... to date several of us have given it 8s and 9s due to his excellent writeup, but the folks who vote 10 on everything have unfortunately just skipped it (just because of a few pictures). I should also note that the writeups are generally more work than the photos, and I think they are often more valuable.

I will also point out that getting an 8 is much better for a page than someone not voting on it at all because they only use 10s. So... for those of you that believe that a 7 or 8 might hurt someone's pride... don't feel that way. Not voting on their page is worse (the equivalent of voting a 5 or 6). But also... this is not trying to discourage voting 10/10 on pages that deserve it. I've seen several pages recently (especially a few route pages) that I think most would agree really do deserve a 10.

So what do others think?
Last edited by mrchad9 on Tue Mar 05, 2013 12:58 am, edited 1 time in total.

The following user would like to thank mrchad9 for this post
Josh Lewis, Scott

User Avatar
Josh Lewis

 
Posts: 3412
Joined: Thu Jan 06, 2011 11:12 pm
Thanked: 1108 times in 677 posts

Re: Discounting 10/10 Vote Weight

by Josh Lewis » Mon Feb 25, 2013 9:21 pm

mrchad9 wrote:Basically we would have a mechanism that would automatically evaluate all the votes placed on objects, and if a certain percentage were 10s it could decrease the vote weighting for that member.


You win my agreement. :) Voting was originally created for sorting content. But I will note that this should not apply to past votes considering almost everyone on SP's power would drop right away. So perhaps there might be a way to have this apply from now on. I haven't been giving out many 10's these days. I voted a 9 the other day. :o (I thought it was a great page) 10/10 is reserved for folks who have gone amazing and beyond. 8)

User Avatar
Bob Sihler
Forum Moderator
 
Posts: 8486
Joined: Mon Aug 23, 2004 3:20 pm
Thanked: 2763 times in 1527 posts

Re: Discounting 10/10 Vote Weight

by Bob Sihler » Mon Feb 25, 2013 9:40 pm

I think you run the risk of discouraging people from voting if they think they are having their voting power diminished because they are not voting according to someone else's idea of the system.

Give more time for people to learn about the new system and adjust, in my opinion. I'm having a hard time myself breaking from the 10 habit-- have given some 7-9 but do admit I still mostly do 10 or not at all. Honestly, I tend either to like a page or not like it because I have a certain set of criteria for a good page and am looking for those things when I see a quality page, not for what will amaze me.

I would have preferred a like/not like system that somehow took into account power points, but for now, I'm happy to see how the new system goes. It's still basically in its infancy.
"Alcohol is like love. The first kiss is magic, the second is intimate, the third is routine. After that you take the girl's clothes off."

--Terry Lennox, The Long Goodbye (Raymond Chandler)

User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Re: Discounting 10/10 Vote Weight

by mrchad9 » Mon Feb 25, 2013 9:53 pm

Sounds like you need some encouragement to break from old habits Bob. Would this not provide some incentive? And BTW this isn't my personal idea of the system... it is the intent of the site:

1 The Worst. Ever.
2 Horrible
3 Pretty Bad
4 Poor
5 Needs Additions
6 Fair
7 Good
8 Great Work
9 Wow! Amazing
10 The Best. Ever.

It could be implemented in a way to boost the weighting of people who use the full scale, rather than discounting others.

But isn't this what we want? Why should one member get more weight than another just because they have submitted more pages and photos? That is the current system (and why is a like/dislike system based on your number of submissions better?). Seems to me one member should have more weight if their vote has more critical thinking behind it. I just don't follow that the number of pages submitted is the primary measure of how meaningful your vote is. How else do you explain that the Grinnell page is missing votes from a lot of the typical voters, but has them from a group that very possibly may have spent more time looking on what was written rather than the lack of a couple of photos? Just a thought... in an attempt to encourage people to think about their vote rather than think about what tit for tat vote might come in return.

Agreed Josh... some start date or equivalent would have to be determined.

User Avatar
Josh Lewis

 
Posts: 3412
Joined: Thu Jan 06, 2011 11:12 pm
Thanked: 1108 times in 677 posts

Re: Discounting 10/10 Vote Weight

by Josh Lewis » Mon Feb 25, 2013 10:01 pm

A "like system" was the second best alternative. But I think Chad's was the best (the voting system that is currently in place). If it was a like system, popular pages would guaranteed be the number one pages of the site even if SP's culture was a picky one when it comes to voting vs quality.

User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Re: Discounting 10/10 Vote Weight

by mrchad9 » Mon Feb 25, 2013 10:06 pm

Just to refresh folks on the history...

The current voting system with the new algorithm was actually easier for Matt to implement than a like/dislike, since the interface stayed the same. Just a few lines of code to change. So it was less effort, and gave a result with more flexibility. I'm always in favor of things that give a more sophisticated result with less effort to begin with.

As noted the current system has already demonstrated benefits via the productive conversation I recently had with a new member. Those treating the system as if it were a like/dislike system simply ignored his page, and would not have resulted in the discussion.

User Avatar
Bob Sihler
Forum Moderator
 
Posts: 8486
Joined: Mon Aug 23, 2004 3:20 pm
Thanked: 2763 times in 1527 posts

Re: Discounting 10/10 Vote Weight

by Bob Sihler » Mon Feb 25, 2013 10:29 pm

The idea of boosting the voting power of those who use the system as intended instead of lowering the power of those who don't is much better, in my opinion. The end effect might be the same, but the different emphasis would work better, I think.

Might also consider vote power bonuses to those who vote more often and in the intended manner at the same time. That would encourage more voting on more pages, which would be good even if a few losers were doing it solely to up their power.
"Alcohol is like love. The first kiss is magic, the second is intimate, the third is routine. After that you take the girl's clothes off."

--Terry Lennox, The Long Goodbye (Raymond Chandler)

User Avatar
Josh Lewis

 
Posts: 3412
Joined: Thu Jan 06, 2011 11:12 pm
Thanked: 1108 times in 677 posts

Re: Discounting 10/10 Vote Weight

by Josh Lewis » Mon Feb 25, 2013 10:40 pm

Boosting is probably a better method. It would create the least amount of complaints and the least amount of folks losing faith in the system.

User Avatar
surgent

 
Posts: 545
Joined: Wed Aug 20, 2003 2:45 pm
Thanked: 143 times in 80 posts

Re: Discounting 10/10 Vote Weight

by surgent » Mon Feb 25, 2013 11:33 pm

Leave things alone.

There's no objective criteria that defines a "10" versus a "9" versus an "8" mountain/sunset/flower/route and so on. The distinctions are too ill-defined. If 50 years and thousands of climbers can't agree on the distinction between class-3 vs. class-4, for example, we're not going to have better success on the voting criteria. Thus, you run the risk of reducing the voting weight of certain people when they don't "use the system correctly", but the system itself is inexact. How exactly would you explain your reasoning to an affected voter? The whole system is hopelessly subjective, and frankly, that's fine. If you can completely and axiomatically* develop an objective voting system, then build a robot to vote on all submissions.

Everyone votes in a system of their own choosing. Some vote 10 or not at all. Some use a smattering of the numbers. The end result is a mish-mash, yet in the broadest terms, the better pages float to the top, the worse ones to the bottom. Thus, there is nothing statistically significant between a page with a score of 82.15% versus one with a score of 82.53%. We may as well judge pages by its percentiles, that is, its relative "quality" against similar pages.

There is also the fact that certain types of mountains draw more views and more votes. The more famous peaks, and those that are more "mountaineery" with glaciers and knife-edges, outdraw the views and the votes of other types of peaks, e.g. desert summits, or forested Appalachian mountains, or other peaks that normally don't get considered when we think of mountaineering.

For example, as of February 25, 2013, Mount Rainier has a 100% rating, 263 votes and over 437,000 hits. Ruby Dome (in Nevada) has a 94.24% rating, just 46 votes and only 22,186 hits. Both are outstanding mountains with challenges unique to each, but is Mount Rainier better than Ruby Dome? What defines "better"? Their respective pages look about equal in quality. Both are well done, but Rainier wins because it's just a more famous mountain than Ruby Dome. More people gravitate to Rainier than to Ruby Dome simply because more people have heard of Rainier.

Now, view the page for Wheeler Peak (Great Basin, Nevada): http://www.summitpost.org/wheeler-peak/150191. This page as a near 98% rating, but the page itself is very scant and hasn't been updated since 2005, when SP1 was in place. People aren't voting on the page, they're voting on the mountain. A crap page on some beautiful peak in the Alps will outdraw, outvote and outscore a well-constructed page on a desert peak in New Mexico or Morocco any day of the week. That's just reality. That's the unavoidable bias that we all have toward snowy, pointy mountains.

We've already toyed with scaling, weighting and applying logistical curves to voting, and it won't solve whatever problem that we believe to still persist. You're just moving laterally after a certain point. Some might say there really is no problem, that abuses of the system are far outweighed by "intended" uses of the system, or even balanced by other abuses...

What we have now works in its clunky, taped-together way. Thus, I vote to leave things alone. Bear in mind, my vote has a weight of 94.39%, and I am most proud of all the hard extra work I have done to get that extra 0.39. By June, I hope to be somewhere around 94.46.

(* No need to remind me about Godel's incompleteness theorem)

The following user would like to thank surgent for this post
Bob Sihler, Brian C, chugach mtn boy, lcarreau, rgg

User Avatar
Josh Lewis

 
Posts: 3412
Joined: Thu Jan 06, 2011 11:12 pm
Thanked: 1108 times in 677 posts

Re: Discounting 10/10 Vote Weight

by Josh Lewis » Tue Feb 26, 2013 12:06 am

surgent wrote:If 50 years and thousands of climbers can't agree on the distinction between class-3 vs. class-4, for example, we're not going to have better success on the voting criteria.


It's pretty obvious. A 10/10 means it's one of SP's best pages. When folks say "I wish I could vote a 11/10" they should really mean to vote a 10 but use it less often to reflect that level of awesomeness on that page. Every post on SP cannot be "the best ever"... right? In the climbing world there may be some confusion on what is class 3 to 4. But we sure know well what the difference between class 3 and 5 is! That's only 2 number difference. :wink:

User Avatar
Marmaduke

 
Posts: 1541
Joined: Tue Feb 02, 2010 1:08 am
Thanked: 729 times in 562 posts

Re: Discounting 10/10 Vote Weight

by Marmaduke » Tue Feb 26, 2013 12:42 am

This new system seems to be a very nice step to making a system that had major flaws to close to being perfect. Maybe there will be a few minor changes needed but the real change needs to come from the SP members to now realize that an 8 or 9 vote is still a positive vote. When before it was a 10 or nothing, there are options and the repetitive 10 needs to go away. In fact I might go to Chad's pages and give him a bunch of 8's or maybe even 7's :wink:

The following user would like to thank Marmaduke for this post
Josh Lewis, mrchad9

User Avatar
Josh Lewis

 
Posts: 3412
Joined: Thu Jan 06, 2011 11:12 pm
Thanked: 1108 times in 677 posts

Re: Discounting 10/10 Vote Weight

by Josh Lewis » Tue Feb 26, 2013 12:48 am

Marmaduke wrote:In fact I might go to Chad's pages and give him a bunch of 8's or maybe even 7's :wink:


Feel free to do that on my pages too. :)

User Avatar
Bob Sihler
Forum Moderator
 
Posts: 8486
Joined: Mon Aug 23, 2004 3:20 pm
Thanked: 2763 times in 1527 posts

Re: Discounting 10/10 Vote Weight

by Bob Sihler » Tue Feb 26, 2013 2:03 am

I still think the best thing is to give it time and let people learn and adjust.

Should there be a reconsideration of the system, consider a 5-point scale as there was way back. Honestly, I think the 10-point scale is too much. Something that I find difficult is deciding between 7, 8, or 9. Thus, I usually end up voting 10 or not at all so far. It will take time for me to get a feel for the new system and use it as intended. But on a 5-point scale, I would find it much easier to give the 7-9 pages a 4.

I think it's too hard to choose between 2 and 3, 4 and 5, 7 and 8, etc. It's much easier on a smaller scale. A 1 would be for an awful page-- missing critical information, plagiarized, etc. 2 for a weak page-- sparse detail, poor writing, little useful information. 3 for a so-so page-- gets the job done but is nothing special, like pages with very limited but accurate text and short on good and useful pictures and diagrams. 4 for a nice, solid job that comes short of wowing you. Chad's example of Grinnell would get a 4 from me because it has everything needed but lacks an effort to incorporate pictures with the text; as of now, though, I have a hard time voting on it because it is not a 10 for me but I don't know which of 7, 8, or 9 it is, so I may end up voting 10 simply to encourage a new member to keep contributing. 5 would go to pages with excellent detail, excellent writing, and good/useful photographs incorporated into the body of the page.

Surgent is right that it's very subjective. We all look for different things. Some people here love huge pictures and tables; I tend to find those things more visually impressive than actually useful, and such pages can also be quite slow to load. Who is right? Neither. It's subjective.

Please don't take this as a complaint against those who worked hard to change the voting system. I am happy that I can now vote 7-9 and have it be a positive instead of wrecking the page's score. But I'm just not used to it yet and haven't made up my mind yet on how I want to vote on pages that are good but not great. And the fact that there's not much difference between vote scores-- my 4 on a page with no votes yields a 69.64% while my 10 yields a 71.6-- makes it even harder. Knowing that improving my page to get someone to change from 7 to 10 will not make much difference in the score provides virtually no incentive to expend that extra effort.
"Alcohol is like love. The first kiss is magic, the second is intimate, the third is routine. After that you take the girl's clothes off."

--Terry Lennox, The Long Goodbye (Raymond Chandler)

User Avatar
chugach mtn boy

 
Posts: 942
Joined: Sat Jun 10, 2006 7:54 pm
Thanked: 224 times in 129 posts

Re: Discounting 10/10 Vote Weight

by chugach mtn boy » Tue Feb 26, 2013 2:16 am

This might just be a regional thing. I grew up in North Carolina, whereas Chad, Josh, and Marmaduke are from out on the left coast. And I can tell you that back in North Carolina, we're not quite sure they raise people right out there. :?

Let's say I go to Bob Sihler's for dinner. Bob throws some huge steaks on the grill that must have cost a fortune, but he leaves them on a little too long and they get a little dry. Meanwhile, the missus whips up some potato salad that's really pretty good. It would probably be between "Great Work" and "Amazing" on the scale Chad loves to quote. "Susie," I say, "This potato salad is out of this world." Now, I could say something to Bob about the steaks but I'll probably just talk about other stuff, unless he asks me directly. Bob goes for the Glenlivet. It's going to be a good night.

Let's add Josh, Chad, and the Duke to the party. "I agree with Chugach," Josh chimes in. "That potato salad is about 8.5 out of 10." "Yes indeedee," says the Duke, "it's a definitely better than most I've had at fast food places, but not quite as good as the one I had at Chad's the other night." "Thanks," says Chad, "mine was a little better, but I've still got to hand it to Susie--this potato salad really hit the spot. It's nice to have something moist when the steak is dry." Conversation kind of dies out and and pretty soon the three left coasters are heading down the front walk. "Huh," says Chad, "I just don't get these east coasters. Don't they even have dessert around here?"

Hey, barbarians (and I mean that in the kindest way), this just ain't happening. There is NO WAY you are going to get a lot of SPers very interested in dishing out "pretty good" grades to other people on their creations. Regular people are not comfortable interacting that way. The fix to the old, bizarre SP voting algorithm was great, as far as it went. But now you're talking social engineering, and what you're hoping for is beyond the power of SP to achieve, no matter how many of these threads we see (and we've seen quite a few ...).
Last edited by chugach mtn boy on Tue Feb 26, 2013 2:19 am, edited 1 time in total.

User Avatar
mrchad9

 
Posts: 4545
Joined: Thu Jul 02, 2009 12:01 am
Thanked: 1338 times in 911 posts

Re: Discounting 10/10 Vote Weight

by mrchad9 » Tue Feb 26, 2013 2:18 am

Bob it sounds like you are cognizant of the recent changes but are simply choosing to disregard them? If you are having trouble choosing between a 7 and 8 but then give a page a 10 instead then perhaps this suggestion in the OP will help you make up your mind more easily between the options.

Your explanations are leading me to believe this is a better idea than I had originally believed.

Also note that there is a big difference between an 8 and a 10. It takes fully twice as many votes if they are predominately 8s to reach the same score as a page that has 10s. Couple this with how pages power points are determined and there is a huge incentive to make higher quality pages (if you compare the old versus the new system the incentive is dramatically different). Previously the difference between a terrible page with one vote and a page in the top 10% was only about 1 power point, whereas now that is 3+ points. Just for the quality difference on a single page.

Next

Return to Site Feedback

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 0 guests

cron