Webdownload lik "Webgrabber"

[Alexander Holst]

On Sat, 2004-12-04 at 01:18,
yellowdog-general-request at lists.terrasoftsolutions.com wrote:


> Message: 10
> Date: Fri, 3 Dec 2004 19:18:06 -0500
> From: Eric Dunbar <eric.dunbar at gmail.com>
> Subject: Re: Webdownload lik "Webgrabber"
> To: Yellow Dog Linux General Discussion List
>         <yellowdog-general at lists.terrasoftsolutions.com>
> Message-ID: <77520bee04120316181f94a004 at mail.gmail.com>
> Content-Type: text/plain; charset=US-ASCII
> 
> To follow up on that request (and top-post ;-):
> 
> Is there an app that you can programme to cycle through and download a
> bunch of slight variations on an URL? I have about 5 years of monthly
> weather data that I'm going to have to d/l for five different weather
> stations and don't really cherish the thought of modifying 12x5x5 =
> 300 URLs.

How about:

#!/bin/bash

Locations="Location1 Location2 Location3 Location4 Location5"
Months="01 02 03 04 05 06 07 08 09 10 11 12"
#Months="Jan Feb Mar ..."
Years="1950 1951 1952 1953 1954"

for theYear in ${Years}
do
  for theMonth in ${Months}
  do
    for theLocation in ${Locations}
    do
      wget [opts] "http://...${theYear}${theMonth}${theLocation}.html"
    done
  done
done


> Is wget the app for this (doesn't have to be a Linux only solution...
> OS X would work for me too)?

wget itself can't do that - but with a little wrapper it will ;)

Greetings,
Alex

-- 
Alexander Holst
Hergulesse Design
ph: +49 [0]711 887 5195
fx: +49 [0]711 887 5196
<alex.holst [at] hergulesse [dot] de>