Welcome to SP!  -
Summitpost Bulk Uploader
Article
 
Children 

Summitpost Bulk Uploader

 
Summitpost Bulk Uploader

Page Type: Article

Object Title: Summitpost Bulk Uploader

 

Page By: mvs

Created/Edited: Feb 10, 2009 / Jan 15, 2011

Object ID: 488357

Hits: 8230 

Page Score: 95.83%  - 55 Votes 

Vote: Log in to vote

 

Introduction

ANNOUNCING a Windows-based Summitpost Bulk Uploader for your images. This article contains all the information you need to use it. Download the zip file from here.



Please don't make use of it to store your entire collection of pictures of butterflies up here of course! I just want to save you time for uploading the usual collection of high-quality mountain-themed images! :-)


A Summitpost Forum discussion about the tool exists as well.

Why did I create this tool?



I love Summitpost. But individually uploading 10-15 images for a trip report was getting old. Sites like Flickr offer several nice ways to quickly upload your images, and even extract title and description from embedded metadata! So I started to use that. At the same time I felt bad because then my images aren't on Summitpost for mix-n-match inclusion in Mountain, Area or Route pages by other authors.

So I hunkered down and came up with this tool. My hope is that it will encourage people who, like me, started yawning and sneaking away to watch TV when they contemplated the manual work involved in uploading each picture. That is the drudge-work of content creation! Of course you want to do some actual climbing sometimes too!

I'm not affiliated with Summitpost other than being a member. So it's not an official tool. In fact the folks who run the site are a little worried about this thing. They are afraid that if people have this tool they'll get lazy and upload crap "willy-nilly" rather than carefully chosing what pictures are worth sharing with the rest of the world.

So please, continue to exercise care in the number and quality of images you upload. With that good citizenship on display, we will prove the naysayers (who expect that us ordinary users aren't mature enough to handle this tool) wrong!

UPDATE: March, 2010



This article has been around a while, you could say the Bulk Uploader was in "beta test." I want to thank the Elves for their vote of confidence in publishing this article on the front page, especially Bob and Gangolf. No "enemies of progress" they! I really appreciate it.

Lately there were a lot of requests for a Mac version. This recent move from the Elves inspires me to pick up the trowel again and see what I can do.

How to get it


Download the zip file from here. Current Release: 1.0.0.12 - see bugs/features below for new work!


This version should run on Windows XP, Windows Server 2003, Windows Vista. I haven't tried it on Windows 98 or Windows ME, please let me know if you try running it there but have trouble. It should work on the new Windows 7 Beta too (again, haven't tried it).

Here are the files extracted to a directory
Here are the application files, extracted from SPBulkW.ZIP into a directory on my local hard drive.

  • Extract the files in the zip file to a directory on your disk. The directory should look like the image above.
  • Double click on vcredist_x86.exe. This installs a Microsoft-provided library file needed for the application. You only need to do this once on your system.
  • Double click on SPBulkW.EXE to start the application.
  • That is all you have to do!


Now you are running the application!

Instructions for use

The opening screen
The opening screen.


You can drag files from the Windows Explorer into the large white box on the left for upload, or you can press the "Add Files" button to gather a set of files using the standard Windows Open File dialog box.

NEW: The Bulk Uploader now reads from the JPEG file when you add it to the file list. It populates the title and description edit boxes with what it finds in the file. You can edit these to customize the text further. Just as before, if you exit the program when you restart it all of your title and description customizations will remain as you last left them. As a technical note the IPTC keys I look for are:


  • Iptc.Application2.ObjectName - this is read as the title

  • Iptc.Application2.Caption - this is read as the description



Programs such as
Adobe Lightroom can populate these fields in the file as part of your post-processing workflow.

Once you have the files you want to upload, and if you don't have IPTC metadata in your file, you'll need to assign a Title and Description to each one. Select a file, then click on the Title edit box to type in a title. When you navigate to another file, the title will be saved.

Control-N and Control-P are handy shortcuts to quickly assign title and description to many files. Note that Summitpost requires you to fill these fields in, so if you have a lot of files these keyboard shortcuts become very handy. After pressing Control-N or Control-P the Title edit box is highlighted, so you can just go ahead and type the title. You can then hit the key to advance to the Description edit box. This one is a multiline edit control, so you can hit the key for a newline.

Main window with a file selected.
Filling in Title and Description fields.


Latitude and Longitude


The program currently supports only setting the same latitude and longitude for the complete set of pictures being submitted. Ideally, you would take all the pictures from the same mountain and upload those, then if you want to change the latitude and longitude for some other pictures you will upload those in a different set.

You can click on the underlined text latitude or longitude to cause your browser to open up a summitpost page which allows you to discover the appropriate values for these fields. On this page you can navigate using the usual Google Maps interface to the location you are interested in. Note down the latitude and longitude values, and type them into the fields in the Summitpost Bulk Uploader.

Username and password


If the username and password fields aren't filled out properly with your own Summitpost username and password, then the upload will fail with a message "Unable to login". For example, see the image below where I tried to log in as user "I AM NOT A VALID USER":

The error received with a bad username
The error message when login fails.


Where are images attached?


If you don't enter an object id in the "Attach to..." edit control, then the images will be attached to your user account. If you'd like to attach them elsewhere, then type in the object id of the Mountain page (or any other kind of page) at Summitpost. If you click on the "Attach to..." text, a web browser will open directed to the Summitpost web page. You could then use the "Search" function at the top of that page to discover the object you want to attach your images too. You can find the object ID in the URL of the object, or (easier) it is listed in the header section of the object page.

Note that you have to attach all the images in one upload to the same object. If you have several objects you want to upload and attach images to, do those in separate batches.

What about the image IDs?


By popular request, after an upload, you now get a notepad window with the object IDs of the images you uploaded, along with the titles of the images. They are written in a format suitable for insertion in a Mountain, Area, Route or Trip Report page.

Be patient for a few seconds after the upload completes, and the notepad window will appear. The image below shows an example:

Object IDs are returned!
Here are your image ids!


Automatic local saves


The Summitpost Bulk Uploader remembers settings between runs, such as:
  • the last directory you retrieved files from
  • the username and password
  • the latitude and longitude for the picture set
  • the files currently in your list
  • the title and description you set for each of those files

This is convenient in case you need to close the application before you have filled in all of your title and description fields. It will be remembered for when you come back.

During an upload
Have fun, now you are uploading!


MBPost



Now, in the File menu there is a checkbox that can be toggled to allow you to upload to MBPost.com. All of the same features are available.

Bulk Detaching


On the Delete/Detach panel, enter the search query you want to use to find images you'd like to detach. Multi-select those images and click the "Detach" button. Don't worry, you'll only be able to detach images if you have the appropriate permissions.

Special authorization topics


In order to prevent the tool from being mis-used, I've added a few authorization features. When you log in, a page on the Summitpost server is checked to verify that your name is not on a "baddie" list. If it is, your login will be denied. Note, that it may be that you can still log on to Summitpost manually, it's just that you are being denied the bulk upload tool. This list can be edited by me personally and the Elves. A letter to the Elves is the right recourse if you find yourself on the list unwittingly. Just upload pictures for your Mountains, Trip Reports, etc., and you'll stay on the right side of the "law!"

Another feature is to protect against bugs in the tool itself: a mandatory upgrade checker. A page on the Summitpost server is checked to verify that your version of the software is at least the minimum supported version. If it is not, then login/upload will fail with a message asking you to come to this page and download the latest version of the tool.

Finally, the tool makes sure to log in again every 5 minutes and check those features above. This prevents a situation where your tool lives as a process on your machine for weeks, and you keep using that old version at harm to yourself or others.

Bulk Delete Feature


By popular demand of our site protectors, The Elves, I have added a "Bulk Delete" feature. At this time, it is exclusively for their use. To view the Bulk Delete screen you need to know a secret password. So for now, unless you are an Elf, skip over this section as this aspect of the tool isn't useful for you.

To use the Bulk Deleter, click on the Bulk Deleter tab just below the menu in the tool. The screen below shows how you'll be prompted for a special password to view the Bulk Deleter:

Trying to access the Bulk Deleter
Trying to access the Bulk Deleter


By default the Search edit box is populated with the URL for the most recent new images. Press the Search button, and the list control will populate with information on the most recent photos on the site in a tabular format.

As an elf, you can select the pictures in this list that you want to delete, then press the Delete button and it will be done right away. As a normal user, the only deletes that will succeed will be those on images that you yourself own.

The Status column in the list box will be updated to indicate if the object was deleted or not. Just to repeat, the tool doesn't give you any special capabilities at all: you can only delete what you own unless you are an Elf. The password protection for entering this screen is merely an extra precaution. I think that password should be removed eventually, but for now the feature is in testing phase for the Elves in particular.

Below is a more interesting example. I've searched for the least popular images with the text "Mont Blanc" on the web site. I then take the URL from the first results page, and paste it into the Search text box, like so:

Using the Bulk Deleter
Using the Bulk Deleter


I select some of these images with the mouse (control and shift-clicking are used to make multiple selections). I pressed Delete, and got the following results:

Delete results
Delete results


Obviously, because I am not an Elf I was unable to delete these images. I hope the Elves will not get carried away with this feature! ;-)

Important note: At this time (version 1.0.0.4), you can only paste in URLs from image searches on the site. The list format for searches is not parse-able yet by the tool. For now that is okay, because what the Elves usually need to delete in bulk are images.

Pledge of Authenticity and Security


 
User  mvs
I give my word.
I, Michael Stanton, user mvs on Summitpost.org, pledge that I am not collecting any data from your use of this tool. Everything that the tool does is 100% public and documented here on this page. My own private web server is used to host the binary distribution of the tool, but that is all.

Known issues

Issue Number Description Status
Issue #1: If you downloaded before 5:00 PM PST on February 11, 2009, then you should download the zip file again. I needed to include the file vcredist_x86.exe because most machines don't have the necessary libraries to run the application. Install steps were altered accordingly on this page at 5:03 PM PST on the same day. Fixed in version 1.0.0.0
Issue #2: Any kind of file can be added to the file list, but only JPEG files should be available. Fixed in version 1.0.0.1
Issue #3: Vertically oriented images display incorrectly in the preview window.
Issue #4: The title and description fields should be grayed out when nothing is selected, otherwise you might think you set a title for an image but you didn't. Fixed in version 1.0.0.1
Issue #5: Duplicate pictures can be inserted in the file list. Fixed in version 1.0.0.1
Issue #6: The static text "images to upload" should be left-justified. Fixed in version 1.0.0.1
Issue #7: To make the "remove selected files" button more useful, you ought to be able to select multiple images.
Issue #8: You shouldn't be able to press the Submit button without at least one image in the list. It should be grayed out. Fixed in version 1.0.0.1
Issue #9: If a field is not valid, please tell me which field is invalid, like username, or title. Don't just say "a field is invalid." And don't use the word "aborting," sounds too serious.
Issue #10: The hotkeys don't work properly. There are duplicate hotkeys for the two remove items buttons. Also -A doesn't work for the add files button unless it's a capital letter A.
Issue #11: The latitude and longitude hyperlinks are confusing if you are not logged in to a browser session on summitpost. You will be taken to a summitpost login page. Maybe link to a public web site with a map and latitude and longitude? That doesn't require a login?
Issue #12: If you are uploading one image, then cancel button obviously doesn't work. The image will be uploaded. (The cancel button is only checked between image uploads)
Issue #13: If you change the username or password it will be ignored on future submits. In fact the system is trying to be efficient and not log in again. But what if you now need to log in as a different user. Fixed in version 1.0.0.1
Issue #14: In communication with the server, SPBulkW should identify itself as a particular kind of UserAgent (rather than mimicking an agent like IE or Firefox), that gives the site owner flexibility in authorizing user A to bulk upload but not user B based on as-yet-to-be determined criteria should the desire or need arise. It also makes it easy to determine from logs exactly what percentage of photo uploads are done with the tool. Fixed in version 1.0.0.2
Issue #15: Enlarging the application should allow enlarging the description zone only and not all text fields on 1 line
Issue #16: From Bob Sihler: I can't type the character "2"! That is weird! Fixed in version 1.0.0.5
Issue #17: From Selinunte01: Special characters like ü, ö, etc. can't be processed. Fixed in version 1.0.0.8
Issue #18: The uploader reports that "some files could not be uploaded" even though they were. This is annoying, can you fix it? Fixed in version 1.0.0.10
Issue #19: Gangolf reports unexpected crashes.
Issue #20: Josh Lewis reports that files could not be uploaded. Actually this was a general problem that should probably prevent everyone from uploading. Fixed in version 1.0.0.12

Version history

This records the bugs fixed in each release. In the future I'll provide links to old versions when it's appropriate. If you go to the "Help" menu and choose the "About" menu item you can see the version number of your installed copy.

To upgrade to a newer version just copy the new files over the old ones. There is no need to re-run the vcredist_x86.exe setup program.

Version Number Date Description
Version 1.0.0.0 Wednesday, Feb 11, 2009 First release!
Version 1.0.0.1 Thursday, Feb 12, 2009. (at 12:07 AM German time) 6 bug fixes, 1 feature
Version 1.0.0.2 Friday, Feb 13, 2009. (at 2:20 AM German time) 1 bug fix
Version 1.0.0.3 Saturday, Feb 14, 2009. 2 new features
Version 1.0.0.4 Wednesday, Feb 18, 2009. 4 new features
Version 1.0.0.5 Friday, Feb 20, 2009. 1 bug fix
Version 1.0.0.6 Sunday, Nov 15, 2009. 1 feature added, development restarted
Version 1.0.0.7 Sunday, Nov 15, 2009. 1 feature added
Version 1.0.0.8 Tuesday, Nov 17, 2009. UNICODE support added (characters like ü, ö now work)
Version 1.0.0.9 Thursday, Nov 19, 2009. IPTC metadata is read from the JPEG file to auto fill title and description
Version 1.0.0.10 Wednesday, Dec 3, 2009. Bug fix for the annoying Issue #18
Version 1.0.0.11 Monday, October 4, 2010.. Bug fix for a server format change that broke upload ability.
Version 1.0.0.12 Saturday, January 15, 2011.. Bug fix for a server format change that broke upload ability. (Thanks to Josh Lewis for pointing it out!)

Feature suggestions

Here I'll keep track of feature ideas. They might come from my febrile (I mean fertile!) brain, from PMs or email to me, from the comments in this page, or from a forum post. The list is not in a prioritized order.

Feature Number Description Status
Feature 1 How about extracting title/description metadata from the JPEG file? Applications like Picasa, LightRoom and others allow you to embed this data. Summitpost Bulk Uploader could just read that. Added in version 1.0.0.9
Feature 2 How about getting the files from a Flickr Set instead of your local hard drive? Not Implemented
Feature 3 How about allowing the user to choose what page they want to attach the pictures to? Added in version 1.0.0.1
Feature 4 From sjarelkwint: How about automatic image resizing before upload to make the images smaller? It takes longer than I'd like to upload the originals. Not Implemented.
Feature 5 From sjarelkwint: How about being able to re-order the images in the list? Not Implemented.
Feature 6 From MoapaPk: After the upload return a list of the picture IDs to make it easier to add them to a Mountain page. Added in version 1.0.0.3
Feature 7 From nartreb: Make the description box bigger. Added in version 1.0.0.1
Feature 8 From nartreb: Don't automatically store the user's password between sessions. Not Implemented.
Feature 9 From my wife: Remove images that were successfully uploaded from the file list. Added in version 1.0.0.4
Feature 10 Be able to upload to MBPost (mountain bike post) as well...it is essentially the same site. Added in version 1.0.0.6
Feature 11 To aid in avoiding uploads of duplicate photos, keep track of the last N photos that were uploaded (say N=1000?). If the user adds those photos again to upload, warn them that it looks like they are uploading a duplicate. Not Implemented.
Feature 12 From FortMental: You should get the rest of the fields on the upload page, like "Image type." Not Implemented.
Feature 13 From The Elves: In case this is misused, give us a bulk deleter! Added in version 1.0.0.4
Feature 14 Can "bad users" be denied? Added in version 1.0.0.4
Feature 15 What if a version of the software has problems, can we force an upgrade? Added in version 1.0.0.4.
Feature 16 From Gabriele: give me a bulk detacher to quickly remove off-topic photos from my pages! Added in version 1.0.0.7.
Feature 17 From Gangolf: In case of a crash I lose my caption information. Can you save captions regularly, for example every time I click on a different picture?


Technical details

Language and compiler: The application is written in C++, compiled using Microsoft Visual C++ 2008 Express Edition.

Libraries used: The GUI is programmed using the wxWidgets cross-platform UI framework. LibCURL provides HTTP functionality. The C++ STL provides strings, collection classes, etc. The C++ CRT library is installed via the vcredist_x86.exe setup program provided by Microsoft. I would prefer to use a static CRT library to avoid this pesky step. Since version 1.0.0.8, the build is Unicode. The Exiv2 library is used to read IPTC/EXIF metadata from the JPEG file.

Porting: Thanks to wxWidgets, there is a minimum of Windows-specific code, and a Mac port is very feasible. Please contact me if you want to help with this. Otherwise it may have to wait a few months.

Other remarks: The application started out as a Python script running on OSX. It worked for me, but I had a very hard time packaging the application for others to use without all the requisite Python libraries (like wxWidgets for Python). I tried using PY2EXE and made a seemingly-decent Windows version with a setup program. But it was very flaky, and I had little control over how I could fix it (Gangolf tried it and it was a real waste of his time!). So I bit the bullet and coded in C++. Despite the many extra lines of code to write, the application feels much more robust.

Open source? I think it would be great to make this an open source project. Let's see if anyone comes out of the woodwork expressing interest in adding features. We could host it on SourceForge.

Feature polls

To discover what people think about things I'm adding a few polls here:

Maximum Image Dimensions




Maximum Image File Size






Images

User "mvs"During an uploadThe error received with a bad usernameDelete resultsThe opening screenMain window with a file selected.Here are the files extracted to a directory
Object IDs are returned!Trying to access the Bulk DeleterUsing the Bulk Deleter

Comments


[ Post a Comment ]
Viewing: 21-40 of 45 « PREV 1 2 3 NEXT » 

Vid PogachnikProblem with detacher

Vid Pogachnik

Voted 10/10

Hi Michael,

tried your bulk detacher, but gives me an error (when clicking Search). Problem with software...do you want to notice Microsoft...

Cheers!
Vid
Posted Nov 16, 2009 2:10 pm

mvsRe: Problem with detacher

mvs

Hasn't voted

Hi Vid, can you tell me the exact steps? Did you first paste another query into the search text box, or just accept the default? Did you type in your username and password on the bulk uploader panel? thx!
Posted Nov 16, 2009 3:26 pm

Vid PogachnikRe: Problem with detacher

Vid Pogachnik

Voted 10/10

Hm? I first downloaded your zip file and unzipped it on my computer. Then I started the exe file from there, so I don't know how your program could recognize my user name and password. Aha! It looks I forgot to login from your program as well! May be that's it. Well, after starting the program, I just copy-pasted the address of one page and clicked Search button. I will try again - at home, now I'm in the office :(
Thanks! Cheers!
Posted Nov 17, 2009 4:06 am

mvsRe: Problem with detacher

mvs

Hasn't voted

Ah okay, I think that is the problem: "I just copy-pasted the address of one page and clicked Search button".

The Detach feature is kind of "advanced" (another word for cryptic). You have to paste in the URL of a summitpost image search query. For example, here is an image search on Mont Blanc: http://www.summitpost.org/object_list.php?object_type=3&object_name_3=mont+blanc&page=1

And yes indeed, it will complain of an error if you don't enter the login/password info on the main screen. Good luck with 2nd try!
Posted Nov 17, 2009 5:06 am

Schputnika mac version.

Schputnik

Hasn't voted

Hello.
Is it possible to get a mac version.

Cheers, Marko
Posted Jan 20, 2010 4:20 am

mvsRe: a mac version.

mvs

Hasn't voted

Hi Marko,
Well I just got some new incentive to do that thx to the front page bump. Not yet, but it is on the list :-).
Posted Mar 2, 2010 4:32 pm

SchputnikRe: a mac version.

Schputnik

Hasn't voted

Super...
Thanx
Posted Mar 3, 2010 12:32 pm

nattfoddFinally

nattfodd

Voted 10/10

'twas about time to see this on the frontpage! I've been using your application for my trip reports and it has worked flawlessly. Thanks!
Posted Mar 2, 2010 6:00 pm

mvsRe: Finally

mvs

Hasn't voted

Awesome! Yeah it's a nice vindication. I'm really glad to help you post your astonishingly good photos! :-)
Posted Mar 2, 2010 11:06 pm

Arthur Digbeebulk downloader?

Arthur Digbee

Voted 10/10

I'd like to be able to back up my SP contributions but right now I'd have to do that page-by-page, image-by-image. Would it be feasible to design a tool that works in both directions?
Posted Mar 2, 2010 9:34 pm

mvsRe: bulk downloader?

mvs

Hasn't voted

Yes, though I think you could do pretty well with a web crawler application. Our very own user "OZNID," veteran of many climbs with me has an application that does this (http://www.snatcherandson.com/).

You would just point that at your profile page and let it crawl 2 links deep. To get all the photos you might let it point at each profile gallery page. Once you get the files you'll have to sort/organize yourself a bit.
Posted Mar 2, 2010 11:04 pm

PellucidWombatTime to write some more trip reports

PellucidWombat

Voted 10/10

now that it isn't so tedious to properly post them! Thanks a bunch!
Posted Mar 3, 2010 11:07 pm

silversummitSeems like a great app

silversummit

Voted 10/10

especially for the instances cited in comments above but lately it seems I have seen it used extensively by "new" members to post 30 - 50+ poorly shot, incompletely (if at all) labeled, redundant IMHO photos that may never be associated with even an album.

And actually I'm guessing that they are using the bulk uploader; they may not be but looking at posting times and the fact that they have so many in continuous succession I have to think so!

You can tell I am a regular New Images browser! I so wish there was some kind of minimum requirement for posting individual photos before using the bulk uploader.

I know! I know! The bad stuff is supposed to sift down to the bottom but should we have to wade through two pages of muck to get to the next decent pic?

(sound of me stepping down from my soapbox)
Posted Mar 11, 2010 10:51 am

mvsRe: Seems like a great app

mvs

Hasn't voted

I guess it's a no-brainer that the average picture quality will decline as the ease of uploading a picture increases. If you are just looking at pictures on this site then I agree and commiserate with you that maybe this is happening.

My vision though, is more one of making it easier for people to make mountain and trip report pages. And in those you'll end up selecting good photos to accompany the text. I'd argue that these kinds of pages are the "finished product" of summitpost, and easier upload ability should result in those pages being better.

But sure enough, if you are just looking at the raw picture feed passing by it might have lessened the experience!
Posted Mar 11, 2010 3:32 pm

nattfoddUnknown login error

nattfodd

Voted 10/10

I tried to use the tool again today and it failed when I submitted. It tried to login for a while, then said "Unknown login error" and "Upload complete, but there were some errors". It then opens a text file with the list of uploaded files, which is empty.

I triple checked my password and I am positive it was typed correctly. I am using the program on Win 7 64, in case that makes any difference.
Posted Sep 16, 2010 4:11 am

mvsRe: Unknown login error

mvs

Hasn't voted

Okay, it has been fixed. A server side change required the client to adapt. Go download the new version and it should be a-ok. Thanks for the note! BTW, Windows 7 64 bit should be fine (I tested it there).
Posted Oct 4, 2010 6:52 pm

PellucidWombatProblems with the newer SP?

PellucidWombat

Voted 10/10

I tried using the uploader today and kept receiving errors. Since the uploader used to work, I'm assuming that the recent changes to SP may have affected it? The problem seems to be the same as that described by nattfodd. :-/
Posted Oct 3, 2010 8:25 pm

mvsRe: Problems with the newer SP?

mvs

Hasn't voted

Okay, it has been fixed. A server side change required the client to adapt. Go download the new version and it should be a-ok. Thanks for the note!
Posted Oct 4, 2010 6:52 pm

PellucidWombatRe: Problems with the newer SP?

PellucidWombat

Voted 10/10

My pleasure. Thanks for keeping up with maintaining this little gem.
Posted Oct 4, 2010 9:39 pm

Josh LewisWhat would I do without this tool?

Josh Lewis

Voted 10/10

Thanks for fixing the latest error, now I am able to upload a lot more photos without having to wait every time for a single image. Part of the reason I have so few posts is the photo uploading (pages look bad without photos). Great tool!

Cheers Josh Lewis.
Posted Jan 15, 2011 3:37 pm

Viewing: 21-40 of 45 « PREV 1 2 3 NEXT »