Pre-requisites for gathering election night results:
- A reliable fast internet connection (NOT dial up)
- LOTS and LOTS and LOTS of disk space
- Someone to babysit the process and start it every so often. (I like starting every 15 minutes)
There are options - I exclude any graphics, limit the capture to 5 simultaneous connections, do not cache the files, and set a maximum external link depth of 1. This prevents the program from finding additional links and trying to copy them too.
Usually a couple of dry runs are needed to make sure that you're getting the files you want without too much extra 'stuff'. During a night of result gathering, I also update my source file with new links as they become available. If available, using a second computer to gather and update your source file is worthwhile.
Do a bit of experimenting with the program first to familiarize yourself with it.
Here are some of the "Compendium" link files created by John Howard for the 2008 primaries:
Indiana: http://www.bbvforums.org/docs/tools/IndianaCountiesV4.htm
Oregon: http://www.bbvdocs.org/OR/state/Oregon-74022.htm
South Dakota: http://www.bbvforums.org/docs/tools/SouthDakotaCounties.html
2) WEB SNAPSHOT
A tool prepared by extraordinary citizen Donovan Levinson. In the course of just a few days, Levinson developed four versions, each with additional features. Interwoven into the developing versions are tips and instructions, so I'm going to provide versions 1.0, 2.0, 2.1, and 2.2 in order with instructions and tips that came with each.
Here they are, in order, with notes as posted at the time:
quote:1.0: first release of the WebSnapshot tool. Version 1.0: http://www.bbvforums.org/docs/tools/WebSnapshot-1.0.jar
(18 KB)
2.0: Added CNN support http://www.bbvforums.org/docs/tools/WebSnapshot-2.0.jar
(27 KB)
2.1: Added CNN exit poll support http://www.bbvforums.org/docs/tools/WebSnapshot-2.1.zip
(49 KB)
2.2: Added data collection for South Dakota counties http://www.bbvforums.org/docs/tools/WebSnapshot-2.2.zip
(52 KB)
2.2.1: Add a time out when downloading URLs
since the CERS URLs sometimes hang
http://www.bbvforums.org/docs/tools/WebSnapshot-2.2.1.zip(53 KB)
VERSION 1.0 NOTES, GENERAL USE
http://www.bbvforums.org/docs/tools/WebSnapshot-1.0.jar
Usage: java -jar WebSnapshot.jar [ ]
Where is the path to a file containing URLs, one on each line. The and arguments are optional. The is the number of snapshots to create, and the is the number of seconds to wait between each snapshot. Here's an example URLs file:
http://www.cnn.com/
http://www.yahoo.com/
MV "BARACK OBAMA" "HILLARY CLINTON" http://www.lakecountyin.org/cms-host/voters/microvote/2008PrimaryElectionResults /LakePrecSumm561.htm
The last line in the file is a special notation for downloading all the precinct summaries for an entire county. The notation for this is:
MV "CANDIDATE 1" "CANDIDATE 2" "..."
Where is the path to the last precinct in the county. The example above downloads all 561 precinct summaries and produces a csv file with results per precinct.
Example usage:
java -jar WebSnapshot.jar urls.txt 60 5
This would create a snapshot folder based on current date/time and then capture each of the URLs contained in urls.txt. It would then wait for 60 seconds and repeat the process, taking a total of 5 snapshots.
You will need Java 5.0+ installed. You need to run the tool from the command line (Start -> Run... "cmd" on Windows XP). Next I plan to add support for generating spreadsheets (with results per county) from CNN pages.
WEB SNAPSHOT 2.0 w/ CNN support
http://www.bbvforums.org/docs/tools/WebSnapshot-2.0.jar
Here is the updated WebSnapshot.jar and an example input file, adds a special notation for capturing CNN results per county in a csv file. Take a look at the example input file to see how this is done.
Installation instructions
---------------------------------------
1) Create a Directory C:\WebSnapshot
2) Download WebSnapshot.jar and urls.txt to this directory
http://www.bbvforums.org/docs/tools/urls-74129.txt
3) Open up a command prompt (Start -> Run... 'cmd')
4) Switch to directory (cd C:\WebSnapshot)
5) Run the java application from the command line (java -jar WebSnapshot.jar urls.txt)
Capture periodic snapshots by using the optional command line arguments (read the first post).
Press Ctrl+C to terminate the application early.
Java is completely free. Here is the instructions for downloading and installing Java 5.0.
1) http://java.sun.com/javase/downloads/index_jdk5.jsp
2) Select Java Runtime Environment (JRE) 5.0 Update 15
3) Choose your platform (e.g. Windows)
4) Download the Windows Offline Installation
5) Run the Installer (jre-1_5_0_15-windows-i586-p.exe)
