Page 6 of 24

Re: Changes to Voting System

Posted: Tue Jan 15, 2013 1:36 pm
Nice work, Chad, I'm confident this new approach will bring us the resolution we need.

Matt, thanks for giving Chad's proposal a shot. It seems to offer a good effort/outcome ratio.

Re: Changes to Voting System

Posted: Tue Jan 15, 2013 10:36 pm
Ok... I do not know if this is a good addition to the formula or not, or if it will work well to code, but this is what I came up with to allow low scores to have an outweighted impact. This was actually much more difficult to dream up than creating the original formula, and isn't a pretty, but nevertheless I think accomplishes the goal. Most of the ideas I had that seemed good to start with ended up falling apart when trying to implement and testing for reasonableness over a wide range of voting scenarios.

If z<=1 then no change:
Page score = 70 + 30 * [1-1/e^((x-5.5)*y/112.5)]

If z>1 then the following:
Page score = 70 + 30 * [1-1/e^((x-5.5)*y/112.5)] - 40*(z-1)

Where z is a number than can range from 0 to 4 and is used to quantify the impact of 'trash votes' to a page. When above 1 the formula kicks in to start greatly reducing the page score.

z = ( a/.25 + b/.35 + c/.5 ) / y

e = 2.718281828459

x = simple weighted average page score between 1 and 10 based on voter weighting (if all voters give a 10 then this average is a 10)
y = total number of votes for a page

a = weighted number of 1 votes on a page (to include weight, take voting power times each one vote / total power)
b = weighted number of 2 votes on a page
c = wieghted number of 3 votes on a page

The logic here is that until a page gets 25% 1 votes, the 1s are treated normally and hurt the score only as much as a ten helps it. Once a threshold of 25% 1s are recieved, the formula kicks in with a stepchange to the rate of change (but not a stepchange to the score itself). 2s are included with a 35% weight and it takes twice as many 3s as 1s. Combinations of 1-3 votes are accounted for. My reasoning here is that 1-3 votes are really meant to damange a pages, whereas 4 and 5 are more middle of the road type votes.

I don't think this is very pretty, but the best I have been able to create so far. On the plus side I think/hope the formula doesn't come into play too often (and therefore hope it isn't too computationally intensive, but need Matt to advise on that). Also need Matt to advise if this is even doable (I assume it is, even if not pretty on the coding as well).

Here is a graph that shows the impact on page score if a page already has a set of 20 votes of 8s (the difference is small regardless of whether the page already had 10s, 8s, 6s, etc...). The lines show what happens when the 21st vote is a 1 (blue), 2 (green), or 3 (red). At vote 21 the score starts to slowly decrease... effectively the same as removing previous votes. Once the 27th vote is cast the graph shows two blue lines. The dashed is the original formula and the lower line is what happens with the new addition. By the time half the votes are 1s, the score of this (and any other page) will be around 20%. 2s and 3s are treated similar but to a lesser degree. The formula goes negative, but Matt advised he can treat that as a zero.

We could easily adjust how quickly this kicks in, but I felt 25% being 1s as a reasonable start. The constant 40 can also be changed to affect how dramatic the impact is once the formula is initiated.

NOTE- I also played around with adjusting the original formula so that page scores started at 50 or 60% instead of 70%. In the end I think 70% is better. The benefit of going to a lower starting point is marginal... it doesn't have much impact on pages that get at least 10+ votes. I feel it could hurt the voting process if people interpret 60% as a 'bad' page score and are more inclined to vote 10s initially to help the page score out. If a page isn't good and deserves to be under 70%, then folks should really start giving it low votes to bring the score down into the 50s and 60s. My thinking here is being driven by what I think the behaviors and voting would be like on the vast majority of pages, and trying not to negatively affect that process as a side effect of making a better design for an unusual occurence.

ALSO- note that if a page is very poor and the intial, or only, votes are in the 1-3 range then the formula takes effect right away and the page score will be abysmal. I think this is the goal of what Bob and Chris initially suggested and in that respect I think it has been met. Also I would anticipate this is the scenario where it most often comes into play.

So if anyone has feedback on this new addition, let us know.

negativevoteschart.jpg (115.6 KiB) Viewed 469 times

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 4:52 am
Good deal Matt. Yes I think the new formula will be a huge improvement. We are probably 90% of ideal. Any changes here on are getting quite marginal. As I noted the 'if' part of the formula has most effect when a new page that is horrid gets only a couple of 1 votes and nothing else. This keeps it from being in the 60% range and I am thinking it is a wise suggestion by the others the more I contemplate it.

I still think we may need to adjust the formula used to calculate an objects power points as I mentioned in the other thread. Especially since it is proably an easy adjustment as this is implemented. Can you confirm what a 75% page gives versus a 90% and 95% page using the current formula? My feeling is that the disparity is too great. Just want folks to feel comfortable with their submissions. Just a thought.

I will try to post a mockup before the weekend. To make it easier I will probably cut up a printout and make some notations on it. It won't be as sexy as Josh's mockup but I think will be clear and an ok starting point for discussion. Feedback from others will be great. I'm not married to the specifics I will show... the main thing is to make some changes and try to highlight new content as much as possible.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 5:44 am
The image points seem far too high at the high score end. Seems like the intention back on SPv2 was to have about a 100:1 ratio between a mountain/route/area pages to image pages.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 7:02 am
Here was my previous post. I think a good starting point but I need to check each over a range of scores.

Mountains and Rocks/Areas and Ranges = (Page Score)^3 * 12
Routes = (Page Score)^3 * 10 [this increases their relative value from the existing standard]
Articles = (Page Score)^3 * 15
Albums = (Page Score)^5 * 2
Images = (Page Score)^20 * 5 [keeps most images very low on the score, but the most popular ones worth about 1-2 points]
Other objects = (Page Score)^3 * 8

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 2:29 pm
The image points seem far too high at the high score end. Seems like the intention back on SPv2 was to have about a 100:1 ratio between a mountain/route/area pages to image pages.

Just my opinion, but make images votable, but not worth any points or whatever. It may help solve the debates about dumping images/etc. that seem to plague SP with predictable frequency. It really doesn't take any real effort to add a photo anyway.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 2:51 pm
Scott wrote:
The image points seem far too high at the high score end. Seems like the intention back on SPv2 was to have about a 100:1 ratio between a mountain/route/area pages to image pages.

Just my opinion, but make images votable, but not worth any points or whatever. It may help solve the debates about dumping images/etc. that seem to plague SP with predictable frequency. It really doesn't take any real effort to add a photo anyway.

I think that's a good idea. Honestly, I'd rather see no voting on pictures at all, which would put a quick end to the dumping and the manipulation games. Imagine if people focused their attention on the actual pages the pictures enhance! But I know very few will agree with me on that.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 2:55 pm
This is totally nit-picking, but ought canyons at least be equivalent to Mountains and Areas? Keep the physical features at the same level. Pages like this, which has lots of climbing beta should be worth as much as a mountain.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 3:16 pm
I'll do that Bubba. I'll put up a graph later today showing what objects could be worth over the typical range of scores, and folks can advise if it should be adjusted up or down.

What to others think about images. I think they should be worth something at least. Right now even comments are worth 1/20th of a point. I think it is good that folks add photos and of those that dump them, I don't think they do it for points (folks that add massive amounts of images don't get votes and aren't the people who care about profile status).

No voting on pictures, as Bob suggests, is a very interesting idea though!

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 3:21 pm
As the heart of this site is mountains and routes, I really think they should count more than other objects. But this is just an opinion, not another wrench to try throwing into the process.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 3:53 pm
Bob Sihler wrote:
Scott wrote:
The image points seem far too high at the high score end. Seems like the intention back on SPv2 was to have about a 100:1 ratio between a mountain/route/area pages to image pages.

Just my opinion, but make images votable, but not worth any points or whatever. It may help solve the debates about dumping images/etc. that seem to plague SP with predictable frequency. It really doesn't take any real effort to add a photo anyway.

I think that's a good idea. Honestly, I'd rather see no voting on pictures at all, which would put a quick end to the dumping and the manipulation games. Imagine if people focused their attention on the actual pages the pictures enhance! But I know very few will agree with me on that.

I like the voting on pictures. I don't need to pump up my ego by comparing my photos with those of others, but it is nice to see how others compare them with each other. I'd get rid of the power points for them though; that just encourages people to post as many photos as they can. The valuable photos will get posted regardless of power points, and some of the most valuable photos get low scores anyway because they often aren't the most artistic photos. I'd also get rid of the power points for albums.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 3:58 pm
Bob Sihler wrote:As the heart of this site is mountains and routes, I really think they should count more than other objects. But this is just an opinion, not another wrench to try throwing into the process.

I can't agree that routes should be worth more than canyons. But, again, just an opinion.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 4:10 pm
Bubba Suess wrote:This is totally nit-picking, but ought canyons at least be equivalent to Mountains and Areas? Keep the physical features at the same level. Pages like this, which has lots of climbing beta should be worth as much as a mountain.

Bubba, while I agree that the canyon pages add value to this site, I also believe that canyons are ancillary to the purpose of this site - SummitPost. If we keep physical features at the same contribution level, does that mean we start adding Rivers and thus develop a PaddlePost element to the site? It's important that we maintain focus.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 4:14 pm
As for points for photos and albums... I'm ok with granting some points for photos and albums, but the points awarded should be miniscule compared to more substantive contributions (Areas/Ranges, Mountains/Rocks, Routes, etc.)

As for Routes...if anything's under-appreciated on this site, it's our routes. They are the utility workhorses of this community.

Re: Changes to Voting System

Posted: Wed Jan 16, 2013 4:24 pm
Sarah Simon wrote:As for Routes...if anything's under-appreciated on this site, it's our routes. They are the utility workhorses of this community.

This is mainly because the layout of this site is difficult for routes...especially technical ones. As far as route descriptions MountainProject has a better layout simply because most route descriptions are only a few sentences. Here on SP a three sentence route description may be 100% accurate but many users downvote it thinking it's not good enough.

Part of the climbing adventure is having a sense of mystery to the route you are climbing so a detailed step by step description typically isn't published. This may be partially the reason many climbers have left the site. Hiking and scrambling routes fit better on the current layout for route pages.

The best way IMO to make a better route page if there is no more text to be written is to incorporate photos in the page itself. When I judge route pages I look at the accuracy of the text regardless to whether it's one sentence or a whole book then see if there are relevant photos. Photos with lines speak volumes but I know many climbers hesitate to publish these...take the Mox Peaks for example.

I guess what I'm trying to say is that if you are a climber and you are disappointed with how your accurate but brief route pages were treated, try adding some photos if you didn't already.
If you are more of a hiker/scrambler, please don't ridicule the many brief technical route descriptions that are posted by climbers. Most of the time they are still accurate. Simply suggest where photos could be added.