"GatherUsers"

On summitpost, 2skinners has a page that lists all users with their own personal web pages. For a fun Java project, I made a program that gathers this information in an automated way, and spits it out in a formatted html way. This should make maintaining that page much easier. People have some interesting home pages!

I didn't release the program yet, as it's still in testing.

One thing kind of interesting. It seems clear that our Elves are constantly waging a war against spammers. When running GatherUsers there will invariably be a set of users with web pages at the end of the list that are clearly spam. For example I saw all these guys:

35995, JutiJora, http://best-cars-online.info/bmw-2006/2006-bmw-750i.html
35997, EsbraLymim,
http://best-cars-online.info/car-tunes/car-tune-up-cost.html
http://best-cars-online.info/dodge-caravan420/dodge-caravan-mini-van.html
36003, KolyLoipus,
http://best-cars-online.info/car-rent851/car-insurance-rental.html
36007, InilioMimloi,
http://best-cars-online.info/ford-trucks/ford-truck-seattle.html
36009, LyqweThera,
http://best-cars-online.info/york-nissan/nissan-vann-york.html
36010, InityLiini, http://best-cars-directory.org/honda-parts/
36011, PisiniLiopi,
http://best-cars-directory.org/car-part496/car-cheap-part.html
36012, KuiniMimbra,
http://best-cars-directory.org/car-wax/new-car-wax.html
36014, InigoGoopi,
http://best-cars-directory.org/volvo-850/850-cargo-cover-volvo.html
36015, TitiHethe,
http://best-cars-directory.org/car-images/racing-car-image.html
36016, InipusEsini,
http://best-cars-directory.org/ford-contour/ford-contour-clutch.html
36017, InikuTipis,
http://best-cars-directory.org/car-repair811/car-quote-repair.html
36022, LioloiPisjo,
http://best-cars-directory.org/dallas-hyundai/hyundai-dealer-dallas.html

How depressing!

Operating the program

To run the program, go to a console window and type:

  GatherUsers.cmd database.txt

This will begin gathering user information and storing it in database.txt. If the program stops for any reason (system shutdown, loss of network connectivity), you can run it again and it will pick up roughly where it left off, by consulting the current state of database.txt. This file is in the format

  (user id), (user name), (user web page, if any)

When the program quietly exits, then you've gathered everything. The software is pulling users from a list sorted by the date the users joined, so if you want to efficiently get new home page data, just run it every week or so against your existing database.txt file. The program will start gathering the new users right away. If you decide that your information became stale, throw away your database.txt file and start afresh. Caution - it can take many (10+ hours) of running to gather data for all users. Though an incremental update should be just a matter of minutes.

Formatting

You could format the data in database.txt yourself, or make use of a special mode of the software which does rudimentary formatting for you. If you run:

  GatherUsers.cmd database.txt format > output.txt

Then the program will

Sort the users alphabetically
Remove duplicate lines (these will be present if any restarts of the process were made)
Remove users with no web pages
Print users and their web pages, one per line

Have fun!

The current data (all of the summitpost users as of March 2, 2007)

-

-Koen-: http://www.tigrayrockclimbing.tk

0

01aCRViper: http://www.grbphotography.net/
030262: http://www.ohactive.com

1

1000Pks: http://webs.lanset.com/pyamagata

...
The data has been moved to this page.