Bob Sihler wrote:Matt, Dow's 8 is not much different from my 8, but compare my 1 to a new member's 1. It is a massive difference.

Right. That's intentional in order to stop people from creating fake accounts and spam voting, which was a serious problem before SPv2 I guess. In order for a new user's vote to count much, they have to submit at least a few things (10 points is the cutoff right now). That threshold could be lowered though or we could make other things count for more points (such as signing a summit log, commenting on a page, etc.), if people believe it to be too high currently.

Bob Sihler wrote:You're right that the academic scale that most of us are used to is a big part of the dissatisfaction with the way SP's voting system works. You either have to fix it to align with what people are accustomed to or throw it out and try something else. And since it seems you and most others here want to try something else, let's enjoy the ride!

Yeah, it's going to likely be a crazy one in terms of power point changes for users and score changes to the pages.

Almost immediately after I joined I saw that there was rarely a vote other than 10. Rather than trying to use the whole scale, I followed suit, because despite the flaws of the system, I saw the distorting effect that votes of 8 and 9 had on some good pages.

I think it would be an improvement to discard the scale in favor of Like/Dislike. People are familiar with that. I still expect very few dislikes, as most people will simply refrain from voting if they don't like something, but I think that the option of saying that a page is bad would be useful. I see no merit in having more than two options. Too many choices leads to the same problems as with the 1-10 scale; most people won't use them.

As for calculating page scores, I'm against an open scale. As pointed out earlier, that would simply lead to popular peaks and routes having a higher rating, but it wouldn't reflect the quality of the page itself. Even now, popular peaks typically get more votes and hence higher scores than unknown ones, but with a 0-100% scale, the effect is not dramatic.

I would suggest all low (1 thru 5) votes are converted to Dislike, votes 9 and 10 to Likes and the others discarded. After that, the current page score calculation, wouldn't even have to be changed significantly: just substitute a 10 again for every Like, and a 1 (or perhaps a 2 or 3) for every Dislike and keep everything else the same.
While I believe this will lower the score for almost everything, I don't see any problem with that. Scores were supposed to help the best stuff rise to the top, so they are relative anyway, right?

Regarding comments about an "abstain" option.
I figured not voting at all would be the same (score-wise, it would), but I can see that there is some additional feedback provided to the submitter - it shows that someone has looked on it with at least some criticality (page hits would give you an idea as well, but doesn't mean that someone actually spent any time looking at it). Maybe instead of "dislike", it's just a "needs work" vote which has zero score effect instead of negative. I don't think having dislike negative scores will change the over rankings much anyway - something with no likes is in the same boat as a page with a few dislikes.

So, maybe "like/needs work" as the two options.

Secondly, it might be good if the "needs work" votes sort of dissolved over time, maybe 6-12 months. That way, no one has to go begging for old votes to be removed/changed. It wouldn't change the rankings much since pages still need lots of "like" votes to score high. I imagine some folks might like a "please revisit" button that can be used to ping "needs work" voters to reconsider if work has been done to improve it. I could see this being abused, but it might be worth a try.

Thirdly, things garner more votes when sitting in the "What's New" page, so it becomes important to some, *very* important to others. I think votes that come later after some one has stumbled upon a page or picture and taken the time to vote are more valuable as they're not just reacting to the newness factor. The voter's not just piling on when a page first gets submitted. I think there might be more weighting given to votes that come at a later time. Maybe cut it off at some point, like a year or two to keep the very oldest pages from having an advantage with this.

Ok, lastly (for now), perhaps add an inverse weighting based on the number of summit logs (or perhaps page hits), with the intent to keep pages like Rainier and Everest from popping to the top based solely on popularity.

So ..... when's all this supposed to go into effect

rgg wrote:As for calculating page scores, I'm against an open scale. As pointed out earlier, that would simply lead to popular peaks and routes having a higher rating, but it wouldn't reflect the quality of the page itself. Even now, popular peaks typically get more votes and hence higher scores than unknown ones, but with a 0-100% scale, the effect is not dramatic.

rgg's assertion is correct in that popular peaks (most visited pages) will have a huge advantage over lesser-known peaks in terms of score calculations for the simple fact that they'll garner more votes. In order to mitigate that, we'd have to devise some sort of weighting system for the votes. Bob's ideas (weighting votes by the time they're cast, or weighting the vote inversely proportional to the number of entries in the summit log or hits) are good ones to help with that. I particularly like using the number of hits to the page to inversely weight votes. The more hits a page has, the less each vote counts.

rgg wrote:While I believe this will lower the score for almost everything, I don't see any problem with that. Scores were supposed to help the best stuff rise to the top, so they are relative anyway, right?

The scores and power could be (will almost certainly be) dramatically affected by this change. I won't really know how it will affect them until I make some changes on the test server. And I'll likely have to do some tweaking to variables that were defined to work with the current system's power/score calculations in order to get things working reasonably (giving reasonable results) with the new system.

Bob Burd wrote:So, maybe "like/needs work" as the two options. Secondly, it might be good if the "needs work" votes sort of dissolved over time, maybe 6-12 months. That way, no one has to go begging for old votes to be removed/changed.

If I understand you correctly, the "needs work" wouldn't actually affect the score, right? Is there really a need for this "needs work" button. Couldn't people just leave a note on the additions/corrections or a comment? I mean, fewer than 3% of people actually vote anything other than a 10. Seems like an unnecessary (and likely mostly unused) complication to have another option for that 3% of the time when there is already a fully functional way of giving people details (instead of a non-descriptive vote) about why you aren't voting on their page.

Bob Burd wrote:Thirdly, things garner more votes when sitting in the "What's New" page, so it becomes important to some, *very* important to others. I think votes that come later after some one has stumbled upon a page or picture and taken the time to vote are more valuable as they're not just reacting to the newness factor. The voter's not just piling on when a page first gets submitted. I think there might be more weighting given to votes that come at a later time. Maybe cut it off at some point, like a year or two to keep the very oldest pages from having an advantage with this.

That's an interesting idea. The time of the vote is already saved in the database, as is the time of creation of a page. Something could be included in the weighting to include the time the vote was cast relative to the age of the page. So, for example, perhaps votes that occur between 0 and 30 days after creation get weighted by 0.75, votes between 30 and 500 days get weighted 1.0 and votes after 500 days get weighted 0.75? Though I prefer the idea of weighting based on page hits over this.

Bob Burd wrote:Ok, lastly (for now), perhaps add an inverse weighting based on the number of summit logs (or perhaps page hits), with the intent to keep pages like Rainier and Everest from popping to the top based solely on popularity.

That's another interesting idea.

I can certainly play around with these things and see what I can come up with. My biggest question at this point is how to actually determine and display the score based on the number of "likes." So that problem is two fold:
1) How to weight the votes? Power will definitely play a role, as it always has, but should we consider other things as well, such as Bob is proposing? I really like the idea of using the number of page hits to weigh down the vote for pages with a lot of hits.
2) How to calculate the actual page score? Would more people prefer to see a simple cumulative number, like power, or would more people prefer a percentage. If it were a percentage, it would probably be something like the

Code: Select all

($sum_votes/$sum_max_votes)*100

Where $sum_votes is the weighted sum of the votes for the current page and $sum_max_votes is the sum of the votes for page with the highest sum.

Thanks everyone so much for all of their feedback and comments. You all have been extremely helpful providing excellent ideas and feedback. I think we're getting close to having details worked out for something that will work better for us than the current system.

The Chief wrote:Why am I not allowed to see/participate in this Voting Thread?

"You are not authorised to read this forum."

You are registered as a Dem and a Repub, so they banned you from voting. Cheater!

Bob Burd wrote:Regarding comments about an "abstain" option.
I figured not voting at all would be the same (score-wise, it would), but I can see that there is some additional feedback provided to the submitter - it shows that someone has looked on it with at least some criticality (page hits would give you an idea as well, but doesn't mean that someone actually spent any time looking at it). Maybe instead of "dislike", it's just a "needs work" vote which has zero score effect instead of negative. I don't think having dislike negative scores will change the over rankings much anyway - something with no likes is in the same boat as a page with a few dislikes.

So, maybe "like/needs work" as the two options.

I disagree. I think that there needs to be some mechanism for a negative vote to be registered. Pages that are just bad need to just get bad votes. If the 'initial construction period' passes and the page is still bad, it needs to get dinged.

Secondly, it might be good if the "needs work" votes sort of dissolved over time, maybe 6-12 months. That way, no one has to go begging for old votes to be removed/changed. It wouldn't change the rankings much since pages still need lots of "like" votes to score high. I imagine some folks might like a "please revisit" button that can be used to ping "needs work" voters to reconsider if work has been done to improve it. I could see this being abused, but it might be worth a try.

In my model with the 'abstain' choice, those that have pressed this button ought, once having done so, receive a reminder to revisit the page after a certain amount of time has elapsed. I think we have the something similar in mind here.

Thirdly, things garner more votes when sitting in the "What's New" page, so it becomes important to some, *very* important to others. I think votes that come later after some one has stumbled upon a page or picture and taken the time to vote are more valuable as they're not just reacting to the newness factor. The voter's not just piling on when a page first gets submitted. I think there might be more weighting given to votes that come at a later time. Maybe cut it off at some point, like a year or two to keep the very oldest pages from having an advantage with this.

I think this is a really good idea. Could this be installed retroactively, i.e for votes cast in 2011 on pages created in 2007?

Bubba Suess wrote:I disagree. I think that there needs to be some mechanism for a negative vote to be registered. Pages that are just bad need to just get bad votes. If the 'initial construction period' passes and the page is still bad, it needs to get dinged.

Not voting will result in the score remaining 0. Only getting one "like" won't do much for the score. So pages without many or any votes will remain at the bottom. Why do we need to have a negative vote? Why can't people just express why they're not voting by using the comment or corrections/suggestions interface?

Bubba Suess wrote:I think this is a really good idea. Could this be installed retroactively, i.e for votes cast in 2011 on pages created in 2007?

Yes. All votes that were cast after SPv2 came online have a date stamp associated with them. Votes on SPv1 do not, but they could just all be weighted the same.

Montana Matt wrote:

Bob Burd wrote:So, maybe "like/needs work" as the two options. Secondly, it might be good if the "needs work" votes sort of dissolved over time, maybe 6-12 months. That way, no one has to go begging for old votes to be removed/changed.

If I understand you correctly, the "needs work" wouldn't actually affect the score, right? Is there really a need for this "needs work" button. Couldn't people just leave a note on the additions/corrections or a comment? I mean, fewer than 3% of people actually vote anything other than a 10. Seems like an unnecessary (and likely mostly unused) complication to have another option for that 3% of the time when there is already a fully functional way of giving people details (instead of a non-descriptive vote) about why you aren't voting on their page.

It was brought up that if this second option didn't reduce a score, more folks would use it, so it should become more than just 3%. It give the feedback that basically says, "I've looked at your page and it needs some work". Without it, a person might feel like nobody saw their page because it was submitted at night or whatever. Of course anyone could leave comments or an addition/corrections, but that rarely happens because it takes time beyond the average attention/care span. Let's face it, the easier and quicker the feedback mechanism, the more it will get used.

Montana Matt wrote:

Bob Burd wrote:Thirdly, things garner more votes when sitting in the "What's New" page, so it becomes important to some, *very* important to others. I think votes that come later after some one has stumbled upon a page or picture and taken the time to vote are more valuable as they're not just reacting to the newness factor. The voter's not just piling on when a page first gets submitted. I think there might be more weighting given to votes that come at a later time. Maybe cut it off at some point, like a year or two to keep the very oldest pages from having an advantage with this.

That's an interesting idea. The time of the vote is already saved in the database, as is the time of creation of a page. Something could be included in the weighting to include the time the vote was cast relative to the age of the page. So, for example, perhaps votes that occur between 0 and 30 days after creation get weighted by 0.75, votes between 30 and 500 days get weighted 1.0 and votes after 500 days get weighted 0.75? Though I prefer the idea of weighting based on page hits over this.

This wasn't suggested as a way to unweight popular pages. That was the next suggestion. This was a way to make all those initial votes garnered because they're on the What's New page a little less valuable than those that come later, usually with more thought and heart behind them. Something like 0.5 weight for first week, 0.75 up to 3 months, 1.0 beyond that, was closer to what I had in mind.

Montana Matt wrote:

Bob Burd wrote:Ok, lastly (for now), perhaps add an inverse weighting based on the number of summit logs (or perhaps page hits), with the intent to keep pages like Rainier and Everest from popping to the top based solely on popularity.

That's another interesting idea.

I can certainly play around with these things and see what I can come up with. My biggest question at this point is how to actually determine and display the score based on the number of "likes." So that problem is two fold:
1) How to weight the votes? Power will definitely play a role, as it always has, but should we consider other things as well, such as Bob is proposing? I really like the idea of using the number of page hits to weigh down the vote for pages with a lot of hits.
2) How to calculate the actual page score? Would more people prefer to see a simple cumulative number, like power, or would more people prefer a percentage. If it were a percentage, it would probably be something like the

Code: Select all

($sum_votes/$sum_max_votes)*100

Where $sum_votes is the weighted sum of the votes for the current page and $sum_max_votes is the sum of the votes for page with the highest sum.

I think having an open-ended score will make more sense to more people. Let people see that their vote adds something substantive, even if only a few points rather than a few hundredths of a percent.

Bob Burd wrote:It was brought up that if this second option didn't reduce a score, more folks would use it, so it should become more than just 3%. It give the feedback that basically says, "I've looked at your page and it needs some work". Without it, a person might feel like nobody saw their page because it was submitted at night or whatever. Of course anyone could leave comments or an addition/corrections, but that rarely happens because it takes time beyond the average attention/care span. Let's face it, the easier and quicker the feedback mechanism, the more it will get used.

Good point. I was just thinking about a potential way of making it even easier and quicker for people. Basically if someone visits a page and doesn't "like" it, that would count as a "I've looked at your page and it needs some work" vote, in a sense. As you suggested, hits could be factored into the weighting of the votes. Taking a ratio of votes to hits and using that ratio in some way to weight the vote total could work well to prevent popular mountain (or route, area, etc.) pages from always ending up on top. Currently we don't distinguish between a logged in user's hit and an anonymous visitor to a page. So if we used "hits" as it's currently saved in the database, it would hurt pages that get a lot of search engine traffic. But I could add a way to track logged in user's hits and use that in the calculation. Obviously this new addition would require a period of building up of "logged in hits" but I think it could work well in the long term and for all new additions to the site.

Bob Burd wrote:This wasn't suggested as a way to unweight popular pages. That was the next suggestion. This was a way to make all those initial votes garnered because they're on the What's New page a little less valuable than those that come later, usually with more thought and heart behind them. Something like 0.5 weight for first week, 0.75 up to 3 months, 1.0 beyond that, was closer to what I had in mind.

Got it! That makes sense and I could definitely implement that.

Bob Burd wrote:I think having an open-ended score will make more sense to more people. Let people see that their vote adds something substantive, even if only a few points rather than a few hundredths of a percent.

I agree. That seems to be what most people want to see.

Matt- please give me two days to get a algorithm to you to consider. I've been thinking about this for some time and think I could propose something on how the page scores would be calculated that would clearly allow folks to use more of the voting scale comfortably.

I question the reasoning behind making scores go down much because votes are old. Voting on a page a year ago doesn't make it any worse today. Also this penalizes most of the types of peaks that are submitted today. Big name peaks were submitted long ago, and continue to get votes. But nowadays a new submission might get 20 votes the first two weeks and one or two over the next two years. This is also why I think using a vote to hit ratio could become problematic. Both using a ratio and time make more sense if users were forced to vote to some degree, but that has the potential to hinder ease of browsing.

Please give me till Monday or Tuesday night. I'd really like to see the full scale work and have an idea for the algorithm I just need to get the specific formula.

mrchad9 wrote:I question the reasoning behind making scores go down much because votes are old. Voting on a page a year ago doesn't make it any worse today. Also this penalizes most of the types of peaks that are submitted today. Big name peaks were submitted long ago, and continue to get votes. But nowadays a new submission might get 20 votes the first two weeks and one or two over the next two years.

Yeah, the more I consider it, the more I'm not sure if time elapsed since the vote occurred should have any influence in the score. A vote is a vote, in some sense, and people only get to vote once. Why should people's votes be penalized for being cast early? If the page is especially spectacular, I certainly wouldn't want to wait to "like" it or have to remember to come back after 60 days to "like" it then for my vote to fully count. Also, people could "like" a page soon after creation and then "unlike" it later only to "like" it again to get around the "early voting" penalty. That is, unless some checks were put in place to watch for that sort of thing, which sounds like a lot of unnecessary bookkeeping.

mrchad9 wrote:This is also why I think using a vote to hit ratio could become problematic. Both using a ratio and time make more sense if users were forced to vote to some degree, but that has the potential to hinder ease of browsing.

That's true. But I think, in the long term, it will work out. People are already motivated to vote (10s only!) on the things they think are good and not vote on the things that they don't care about or don't think are very good. Once enough "likes" enter the system and enough pages are viewed without "likes", I think it will work well. Of course, there's no way to know that for sure. I'd have bet my left gonad that the current voting system would have worked exceptionally well. We simply didn't foresee everyone voting 10 or nothing.

In terms of calculating a page for a score, right now I'm thinking more along these lines:
If a user is logged in and visits a page, they automatically cast a vote of 0. Assuming they didn't land on the page by accident, they'll likely read it (or in the case of a photo, look at the picture) and then decide if they "like" the page or not. At which point they can decide to press the "like" button. This will change the vote from a 0 to a 1. Score would be calculated based on users' power weight plus the pages' "popularity" weight, which would be the number of "1" votes divided by the total number of logged in users to visit the page.

Of course members will need to be made aware of this fact and change their usage and voting (now "liking") habits accordingly.

mrchad9 wrote:Please give me till Monday or Tuesday night. I'd really like to see the full scale work and have an idea for the algorithm I just need to get the specific formula.

Sure. I'd be happy to see what you come up with. I might go ahead and implement the current idea I'm most fond of on the new server and see what happens to all of the scores and powers and everything else that will be affected. But I can always pull over data from the current SP database again, revert to it, and implement any other ideas to see how they work out before we make a final decision about what route to go.

Montana Matt wrote:

Bob Burd wrote:Perhaps what people are looking for is an open-ended scoring system, rather than the current 0-100%. Score would then be something like: sum(weight1*vote1, weight2*vote2,etc)

I think you may be right Bob. And we already do that calculation on the way to calculating the score (that sum of weighted votes is used in the calculation), so it wouldn't be hard to go that route. Then the score of a page would be similar to the way a user's power works now, with no finite end.

Bob Burd wrote:The goal is still the same - trying to sort good from bad. Currently, the page with the most 10 votes pops up on top. The above scoring would pretty much do the same, I think.

Yes, at this point it would be pretty much identical. As you say, the page with the most 10 votes is on top, followed by the page with the next most 10 votes, etc. So maybe the "score" of the page could simply be the sum of the weighted votes...that would be easy enough to implement and wouldn't require as significant of a change to the database, code and HTML.

Bob Burd wrote:I'm not sure what the advantage is of keeping scores in the 0-100% range. Does it do a better job of sorting for the very best somehow?

No. I'm not sure why we decided on the % at the end, but we decided to map it to 0-100. The score calculation yields a number between 0 and 1. I guess we assumed that most people would rather see a whole number than a decimal, so we multiply the score by 100 to get something between 0 and 100.

Bob Burd wrote:btw, I think weighting should be a significant factor. It offers some sort of quality check and keeps the creation of fake avatars for the sole purpose of voting. Having a waiting period on new members weighting may only delay this. I think some sort of participation (it could even be in the forums) should be a factor, aside from just voting.

I agree as well. I intend to keep the weighting in place as it is now.

Note however, that while an open ended scoring system accomplishes some of the goals, it lacks the ability to cast negative votes or decrease a page score. While this is infrequently used it is probably still good to incorporate. I'll have a algorithm to propose soon.