SummitPost Scores. V2 vs V3 vs the Eiger

SummitPost Scores. V2 vs V3 vs the Eiger

Page Type Page Type: Article
Activities Activities: Mountaineering

Confusion

One morning in January 2013 I noticed that the score of one of my pages had gone from 90% to 95%. What took them so long to see the true value of my writings, I said to myself. Life was good. For a while anyway. Until I saw that some of my scores had gone up and some had gone down. I had an uncomfortable feeling of confusion. The sticks didn’t fit together. Something had to be done to find out what had happened.

Mountains

Mountains and mountaineering are our business. Why do we climb mountains? Many have tried to answer that question. Sir Christian Bonington expressed it like this in the very early days of E-mail communication: ”For the sheer joy and fun of it! The physical process. The thrill of playing a risk game. The joy of exploration. The beauty of the mountains. Friendship. A bit of competition - kept in balance” I say: Given a high and a flat spot I choose the high one ten times out of ten. And ask me to pick a mountain and I say the Eiger ten times out of ten.
Jungfrauregion The Jungfrauregion with Eiger, Mönch and Jungfrau will always be a 10 for me.
Votes are just the issue here. 52 votes given to a Mountain page gave the score 90.29 in Version 2 which has increased to 95.32 in the new Version 3.
Mountain votes Mountains.
The diagram shows the number of votes (21, 28 and 52) and the corresponding scores received in the old SummitPost Version 2 and the new Version 3, respectively, for my three mountain pages. The lines connecting the points indicate an approximate score for an arbitrary number of votes in the range from 21 to 52. The blue dotted line refers to Version 2 data while the red solid line refers to Version 3 data. Similar diagrams are drawn for the other type of pages (Huts & Campgrounds, Albums, etc.). In the last diagram data for all my pages are plotted.

Huts & Campgrounds

The old Mittellegi hut The old Mittellegi hut on the Eiger.
The Mittellegi ridge on the Eiger is rightly called "the knife edge". The hut is located on a place on the ridge where it is possible to move a little more freely without being lost in the abyss. We were welcomed by a sign saying, "The snow is our drinking water, use the toilet!" Toilet was perhaps not the right word, "A hole with free-fall", would be a better term. The ridge has two faces, one to the north where you far below can see green meadows and mountain villages, and the south where the view is a dramatic mountain and glacier world.
But that is not the issue here. Instead 18 votes given to a Hut page gave the score 88.34 in V2 which has decreased to 84.25 in V3.
Huts & Campgrounds. Huts & Campgrounds.

Albums

My first album on SP was about death and my second was about books.  
Anderl Heckmair (1906-2005) Friend Anderl Heckmair
It rained all day in the German mountain village Oberstdorf. Instead of mountain climbing I choose cemetery walking. Alley up and alley down but nowhere was he to be found. He was the first up the Eiger north face in 1938 and I met him in person on the day fifty years later. I gave up and walked by the church on my way out. And there he was in a prominent grave. It said: “Anderl Heckmair, Bergfuhrer (Mountain guide) Wetltburger (World citizen) Freund (Friend).” I said: “See you friend” and walked away in the rain.  
Eiger from the armchair route... “The White Spider” and other Eiger books.
A man wished to acquire a collection of Alpine books. He began with Leslie Stephen’s “Playground of Europe” and compared each succeeding volume by that book’s standard. His library remained a collection of one book. Having read it I honestly agree that it is a good book, but not that good. I myself bought a paperback in a hurry a day in late spring 1975. I read all night and could not stop until it was finished just as the sun rise. “The White Spider” by Heinrich Harrer, it was. It got me into the mountains so it must be a good book. The score of the picture with the Eiger books changed dramatically from about 12 in V2 to 80.82 in V3. The reason for the low score was a number of low point votes from the days of Version 1. We conclude that all old low score votes are affected by the new algorithm in Version 3. Votes are just the issue here. Typically 26 votes given to an album gave the score 88.96 in V2 which has decreased to 88.18 in V3.
Album Votes Albums.

Articles

 
Early lonely morning on the Mittellegigrat Balancing on the Eiger Mittellegi ridge.
The Wetterhorn, the Jungfrau and the Mönch are three famous mountains on the north side of the Bernese Oberland in Switzerland. All three can be admired from green meadows at close distance. They may have many things in common. One less known is that I have written articles on SP about them. But one mountain is missing: the Eiger, being the most famous of the Oberland mountains. I have an idea for an SP article about the Eiger. Some research work will first be needed though. On the other hand there are several new outstanding articles to be found on SP. Take Eric Viola’s recent articles on Mont Blanc where he tells of famous events in mountaineering history. The 1961 drama of the Central Pillar of Freney and L'Affaire Freney only to mention two. But that is not the issue here. Instead 41 votes given to an article gave the score 89.99 in V2 which has increased to 93.10 in V3.
Article Votes Articles.

Trip Reports

Eiger seen from the summit of Wetterhorn The Mittellegi ridge on the Eiger.
I often think about it as one of the best things I have ever done. It would have been sad to have to say: “Why? Why didn’t I do it when I could?” But fortunately I did it, I climbed the Eiger. I have noticed that very few people know of it when I mention the Eiger. At first it was telling about it that was important but not anymore. Just thinking of that I have step by step, hand by hand, worked my way up there make my day to this day. The trip report tells my story.
But that is not the issue here. Instead 78 votes given to a Trip Report gave the score 91.28 in V2 which has increased to 98.17 in V3.
Trip Report Votes Trip Reports.

Do the sticks fit together?

 
Looking back on the Mittellegigrat Looking back on the Mittellegigrat.
Putting the data from all my SP pages into one diagram indicates that the same algorithm is used to calculate the score for all type of pages. The small kinks on the curves that can be seen are hard to explain. Vote weight may come into the picture? Another effect not taken into account is the slow decrease of the score vs time in Version 2. Just as an example, a page having 26 unchanged votes decreased from 89.13 to 88.96 during 570 days. This corresponds to a score decrease of 0.11 units per year. The reason for this feature is not very clear. Is it still in work in V3 you may ask? It is important to remember that the votes here are 10s with very few exceptions. If the situation will remain the same in the future no one knows. It is reasonable to assume that the same number of people will vote in V3 as voted in V2. The difference will be that some 10s now will appear as 6s to 9s. The V3-curve in the diagram below looks familiar. In fact the curve looks very much as the profile of the final part of the Mittellegi ridge just before the summit of Eiger. I would say this shape is reason enough to approve of the SP V3 implementation (pun intended).
All type of pages. All type of pages.
This diagram can be used to translate the old known scores from V2 to the new scores in V3. We see that 29 votes give the same score in V2 and V3, for more votes V3 wins, and for fewer V3 looses. We see that the good old V2 score of 90 required about 48 votes. The same number of votes gives a score of 94.5 today. In addition, the next diagram shows how the vote values 10 to 6 influence of the page score. A mixture of 10s and 9s gives a curve in between 10/10 and 9/10, etc.
Score vs Vote for Different Vote Weight. Score vs Vote for Different Vote Weight.


Comments

Post a Comment
Viewing: 1-12 of 12
Chaberton

Chaberton - Feb 5, 2013 1:15 pm - Voted 10/10

Your Conclusions?

My best compliments.
You did an amazing job of statistics.
But in the end I would like to ask you please, if you feel satisfied from the changes or not.
All the best.
Roberto.

hansw

hansw - Feb 5, 2013 1:39 pm - Hasn't voted

Re: Your Conclusions?

Well, the answer is simple: I don’t know.
I first did this for myself to see how the new scores were related to the old ones. I could easily do this since I two times per year store the votes and scores given to my pages. As it happened I did this only two days before the change to V3 was made. This means that I have results for V2 and V3 based on identical votes. And as I did the investigation I thought maybe someone else would like to know.
/Hans

mrchad9

mrchad9 - Feb 5, 2013 1:34 pm - Voted 9/10

Yes?

I agree with Roberto... so what is your conclusion?

I should note that page scores are independent of the object type. All other things being equal, a series of votes on a TR, article, or mountain page will give the same page score in both systems. What is different is the number of power points the score provides for an object. Also, as you demonstrate well here, the new system results in a larger range of scores (objects start lower... at 70%... but can often go higher into the upper 90s). Objects with few votes have lower scores (but more points than they would otherwise have with the same page score in v2).

Also the shape of the curve, and apparently the Mittellegi Ridge on the Eiger, is an asymptote.

And just as you theorized the kinks in the curves would be due to things like vote weights and the time impact in v2 (time has no factor in v3). The actual algorithms have no kinks.

hansw

hansw - Feb 5, 2013 2:09 pm - Hasn't voted

Re: Yes?

Well, as I said: I don’t know – yet.
Thanks for the information. I see the asymptotic behavior. The chosen parameters do not seem to be optimized for images. The images on the first two pages have a score higher than 99.5 which shows them all as 100. I guess there is a simple fix for this problem.
/Hans

mrchad9

mrchad9 - Feb 5, 2013 2:15 pm - Voted 9/10

Re: Yes?

True... it is not optimized for the top images. Other objects were the main consideration here.

In the end it was decided having 60-70 images rounding up to essentially 100% (the actual calculation is more precise, but SP doesn't store that) was ok. There are over 600,000 images on SP and we didn't think it worth modifying things for 0.01% of images in case it had even the slightest negative impact anywhere eles. They seem to still be ranked somehow, though I am not sure how. If it becomes an issue, it can still be addressed.

EDIT... well it looks like creation date is the tiebreaker. Newest wins.

hansw

hansw - Feb 5, 2013 2:28 pm - Hasn't voted

Re: Yes?

I buy that. Back to business:
Since this page didn’t exist a few weeks ago I can not ask what your vote had been then. But I ask anyway.
/Hans

mrchad9

mrchad9 - Feb 5, 2013 6:10 pm - Voted 9/10

Re: Yes?

SPv2 this is obviously a 10/10. But those are much harder to come by now! Note in the vote dropdown a 9/10 is 'Wow! Amazing'. It is a fine article and I don't mean to do anything to disparage it.

To get a 10/10 it probably has to be more mountain related for me (like the article on glacier worms).

pvnisher

pvnisher - Feb 10, 2013 9:52 pm - Hasn't voted

3D Jungfrau region

Where did you get that 3D Jungfrau region map? I had planned to do the trilogy last summer but got weathered off and only managed the Moench. I'd love one of those maps, though, to help keep me motivated to get back there.

hansw

hansw - Feb 11, 2013 2:33 am - Hasn't voted

Re: 3D Jungfrau region

I found the 3D model almost 25 years ago in the book store in Grindelwald. It was my first visit there also including my first visit to the summit of the Mönch. My guess is that it is hard to find one today.
Thanks for visiting,
/Hans

TimB

TimB - Mar 20, 2013 8:54 pm - Voted 10/10

Re: 3D Jungfrau region

That was going to be my question, also-where'd ya get the model?
:D
My favorite mountain in the entire world is the Eiger.

visentin

visentin - Mar 29, 2013 4:44 pm - Hasn't voted

how

how did you realize these curves ?
don't tell us you monitored every day the amount of votes and the score of each of your pages... this is sick !

hansw

hansw - Mar 30, 2013 5:52 am - Hasn't voted

Re: how

Please read my first comment.

Viewing: 1-12 of 12

Recent Forum Posts
Current Time: 11:32 am
Thread Time