SummitPost.org

Chad and Matt,

This is just a suggestion and only my opinion, but in a way, rather than have the page value vs. page score (at least for mountain pages/routes) be more linear, or perhaps even using an exponent less than one, so value would level off at some point.

Having it exponential at the beginning can be good because poor pages still receive positive votes, but usually not as many.

The only reason I propose having the value level off at the end rather than to be exponential is because it creates a bias for high traffic pages and against new members.

Just about any mountain page (there probably is the rare exception) that is going to be high traffic has already been added to SP, probably years ago. As time passes, more and more lesser known mountains will be added (since the available well known mountains to add will shrink as time goes on).

Personally, from building my own pages I think that it is much harder to create a really good page on a lesser known mountain than it is with a well-known one.

I recently inherited a few pages for a well know mountain from a member whom deleted his/her content. Rebuilding it was easy because I could rely on not only not only on information gathered from my own climb, but check many online sources/guidebooks, etc. for more information, to refresh memory, etc. The mountain had several well-known routes as well and it was easy to list them.


On the other hand, when you create a page on an obscure mountain, it’s a lot more work. You have only your own personal experience to rely on. If there is any other information out there, you will really have to dig for it. You have to make sure your information is especially accurate because there are no other sources to check your memory against.

Random examples of two equally difficult and spectacular mountains in their own way:

http://www.summitpost.org/capitol-peak/150528

http://www.summitpost.org/morne-trois-pitons/770386

The first page was easy to write for the reasons listed above. The second page, although it appears to have less information was actually harder and more work to create.

On the front page, good pages will typically get more votes than bad pages, but once off the front page, the number of votes is fairly non-relevant and high traffic pages on well-known mountains will always receive the most votes (for obvious reasons). A page with 10 votes doesn’t always mean that it’s higher quality than a page with 50 votes; it just means that more people were searching for that mountain.

Of course this isn’t a complaint, only a suggestion and not a pressing issue.

Montana Matt wrote:Good suggestion Scott.

Scott wrote:On the front page, good pages will typically get more votes than bad pages, but once off the front page, the number of votes is fairly non-relevant and high traffic pages on well-known mountains will always receive the most votes (for obvious reasons). A page with 10 votes doesn’t always mean that it’s higher quality than a page with 50 votes; it just means that more people were searching for that mountain.

Then maybe a good solution would be to feature more obscure pages that get less traffic but are very well done? While at the same time, trying to avoid featuring pages that are already getting a lot of traffic/votes?

That is exactly one of the things I am going to be doing.

Just out of curiosity, when the new voting regime is implemented, how will all the previous votes count? Will they still count as 10/10, and all the pages just have really high scores?

Sarah Simon wrote:

Bubba Suess wrote:This is totally nit-picking, but ought canyons at least be equivalent to Mountains and Areas? Keep the physical features at the same level. Pages like this, which has lots of climbing beta should be worth as much as a mountain.

Bubba, while I agree that the canyon pages add value to this site, I also believe that canyons are ancillary to the purpose of this site - SummitPost. If we keep physical features at the same contribution level, does that mean we start adding Rivers and thus develop a PaddlePost element to the site? It's important that we maintain focus.

Focus on what? Most of our members are hikers rather than climbers, and canyons are just as important to many of us as mountains are. We should bear in mind that we have various things that are called "mountains" that really aren't. Two that readily come to mind are http://www.summitpost.org/blackrock-big-meadows/230710 and http://www.summitpost.org/buck-ridge/405273. Then there's the question of "routes;" we have various kinds (alpine, rock climbing, and hiking). Perhaps we should first define what SummitPost is about - maybe it should be the first item in FAQ or be stated on the "About" page.

Scott wrote:Chad and Matt,

This is just a suggestion and only my opinion, but in a way, rather than have the page value vs. page score (at least for mountain pages/routes) be more linear, or perhaps even using an exponent less than one, so value would level off at some point.

Good suggestion... but this is the case Scott!

OK... well not page value vs page score... but page value vs number of votes. Using an exponent less than one happens to make page value vs page score pretty linear. With the changes to the page values, I was trying to reward highly scoring pages, which I think most agree deserve more credit. Still all good pages will receive a consequential number of points. While the system may not be perfection, I think this design will drive the correct behaviors (encourage quality).

Note that page scores level off. True that page value vs score does not level off (I have at least capped them and removed the exponential component of them, which based on your post you would have a tremendous issue with the system we have been living with).

But since page scores level off (and page values are driven off this) as pages score more votes the impact on points decreases continuously.

Note the graph below. As you can see as pages reach 30-40 votes (which is what seperates really good pages with really popular mountains) the number of points decreases significantly as more votes are aquired. By the time you have 49 votes you can only get 1 more point (14% more credit) even if your page got 1 million more votes after that. After getting 30 votes you only have the potential to get 25% more credit with 1 million more votes.

Bubba Suess wrote:Just out of curiosity, when the new voting regime is implemented, how will all the previous votes count? Will they still count as 10/10, and all the pages just have really high scores?

I would think they remain 10s. Note that scores will decrease a lot though, as they would be part of the new process that scores pages in the 70-90% range for the most part. But they would have some advantage over newer pages, since new pages will get votes in the 6-10 range.

I think the cost is small. Good behaviors will be rewarded in the future. The resolution to this, however, would be for Matt to discount the weighting of votes prior to the transition relative to any new votes a page might get. Easy to do with the future mechanism if it is decided to do so.

mrchad9 wrote:I would think they remain 10s. Note that scores will decrease a lot though, as they would be part of the new process that scores pages in the 70-90% range for the most part. But they would have some advantage over newer pages, since new pages will get votes in the 6-10 range.

That is fine. I was just curious how that was going to transition.

I would point out that sometimes a simple topographic map (which is a picture), or lowly picture with a route drawn on, is more valuable than a long mountain "page."

http://www.summitpost.org/thimble-peak/344399

vs

http://www.summitpost.org/direct-way-up ... 2/c-344396

MoapaPk wrote:or lowly picture with a route drawn on, is 10x more valuable than a long bullshit mountain "page."

You mean like this?

http://www.summitpost.org/c-closer-view ... ute/271543

Then maybe a good solution would be to feature more obscure pages that get less traffic but are very well done? While at the same time, trying to avoid featuring pages that are already getting a lot of traffic/votes?

Yes.

but this is the case Scott!

Using an exponent less than one happens to make page value vs page score pretty linear.

But since page scores level off (and page values are driven off this) as pages score more votes the impact on points decreases continuously.

By the time you have 49 votes you can only get 1 more point (14% more credit) even if your page got 1 million more votes after that.

Thanks for the graph/explanation. Personally, after the page has been on the front page; I don't think the number of positive votes has that much relevance (though I guess many might disagree). Maybe for the first dozen votes or so, but not much after that. This can even be more the case for things like adopted pages, obscure mountains vs. popular, etc.

As a random example, what would you say the actually quality difference between the two pages below would be?

http://www.summitpost.org/tabeguache-peak/150374

http://www.summitpost.org/frisco-peak/287560

Of course that is an extreme example, and perhaps I am somewhat biased because I typically find the pages on obscure mountains/routes most interesting and useful (especially if SP is the only source for the information). (On a different note, the number of recent negative votes usually is very relevant because it may point out abandoned pages, uncorrected errors, etc.)

I have at least capped them and removed the exponential component of them, which based on your post you would have a tremendous issue with the system we have been living with

Yes, yours is better.

In the end, this isn't much of an issue.

Scott wrote:As a random example, what would you say the actually quality difference between the two pages below would be?

http://www.summitpost.org/tabeguache-peak/150374

http://www.summitpost.org/frisco-peak/287560

We need a good mechanism to help identify and highlight pages that have been fixed up and improved. Will have to think on that.

Thanks a lot Matt, Chard, Bob, Scott and everybody trying to fix the current voting system.

I like the new proposed algorithms, especially the change in the weighting of the different objects (less importance given to the pictures).

Thinking about the psychology of the average voter (myself included), I don't think the proposed calculation with every vote above 5 increasing the page score would help establishing a better hierarchy in page quality. To the opposite, popular pages with high number of votes will invariably be on top, as almost nobody will give a vote lower than 5.

I'm ready to bet that we will continue having >96% of votes being 10/10, so the new algorithm might not change anything… And even if a few more 6/10, 7/10, 8/10 or 9/10 are given, this won't affect the "ranking" of existing pages, which will continue to be mainly based on the existing 10/10 votes.

Actually, most of the few current votes in the 6/10-9/10 range were meant to reduce the page score and politely invite the owner to update/revise his/her page. The new algorithm might simply remove the effect of these votes, pushing the ranking of such pages that need to be fixed even higher.

It is actually very difficult to introduce a new voting system that is not an indicator of popularity (instead of quality), especially if we are keeping the millions of existing 10/10 votes. Whatever new algorithm you introduce, it won't affect much the existing pages. This was the reason why I was favouring a dual system ("Good / To be improved") hoping that more members would dare to use the "To be improved" option. But maybe the system can't be fixed, or doesn't even need to be fixed, as you don't really look for mountains and routes based on page score...

Anyway, thanks again for the great work and proposals.

@ Bruno : When the new voting system is in, lets go to town voting honestly. Then when our inbox's get flooded with "Why did you vote a 8/10?" we can explain about the new voting system. I think when SP V3 is out, it will be obvious that things changed and there will probably be an announcement of some sort.

Thanks for the kind words on the weighting Bruno. You always have thoughtful opinions.

The method of scoring a page is a difficult problem to solve, and I don't think the proposed algorithm is a perfect solution, but I do think it is the best solution possible. There probably isn't a perfect solution, and it is at least a nice step forward. People will still get to use votes to encourage better pages... votes can be 1-5 too, and I think many folks will strive to do quality work so they get more 9-10s if they see they are getting 6-7s. At least I hope so.

I bet you are correct that tons of 10s still get doled out... but I bet most of the thoughtless 10s start going to photos, which frankly I don't really care about. Voting on pics IS a photo popularity contest, and there is really no point to telling someone with an 8 that a photo is ok but not so perfect. But I bet we do start seeing a lot more than 4% of votes being 1-9s on mountain pages... and having that feedback mechanism is really the point. Time will tell.

I appreciate your comments... Good to hear from you!

Bruno wrote:It is actually very difficult to introduce a new voting system that is not an indicator of popularity (instead of quality), especially if we are keeping the millions of existing 10/10 votes. Whatever new algorithm you introduce, it won't affect much the existing pages. This was the reason why I was favouring a dual system ("Good / To be improved") hoping that more members would dare to use the "To be improved" option. But maybe the system can't be fixed, or doesn't even need to be fixed, as you don't really look for mountains and routes based on page score...

One of the suggestions was to use the page hit count to inverse weight the score. In a balanced world, this would mean a page with lots of hits and lots of votes would rank similarly to a page with fewer hits and fewer votes, all else being equal. It would be interesting to see how this might work on the existing database. It might have the interesting effect of drawing obscure pages to attention where they might get more hits (and hopefully more votes) and thus become less obscure - leaving room for more obscure pages to rise to the top.

Matt - if page count were to be used, could it be page count based on unique IP addresses? It would be very easy to write a script to make many hits on a page to drop its score. Sadly, this would be inevitable if there was no mechanism in place to counter it.