Download file with changing url

1

Hi,

I have a data source that I have to download from a site but the url is changing every year due tot the filename that changes.

Is there a way to list the files on a webpage and filter by type of file or keywords so that we can download the file.

Thanks.

2

Hi Nikolaas,

You can try to use “PowerShell” action to do than. Here is a tutorial on how to parse HTML pages with PowerShell: http://woshub.com/parsing-html-webpages-with-powershell/#h2_2

3

Thanks for the suggestion !

4

Additional question. Grabbing the urls and filtering works fine !

But then, when I have that one url filtered, I would like to use that single value column in the file download action with an expression but the file download action says that previous actions are not calculated. If this would be possible, I would not have to use an iteration to pass this as a parameter to another project…
What could be the reason that I cannot access the column of the url in the expression builder ?

image
image.png881×604 15.7 KB

5

Hi Nikolaas,

“Columns” option in expression editor in the “Download file” action should not be displayed. It seems to be a bug from our side.

Columns can be specified in expressions only in actions like “Calculate new column”, “Modify column”, etc.

The only way to assign URL option from a column value is to use subproject.

6

Hi Andrew

Ok thanks.

I tested the solution with the powershell. It worked fine on desktop but it does not work on our EeasyMorph server. We will try that again when we update our server to the latest version.

I have a workaround in the mean time: downloading the webpage as .txt and then reading in the txt and filtering out the url that I need.

Kind regards