Manual: How to Retrieve Export Files

This guide to retrieve export files is designed to assist our valued partners in downloading our files, particularly those of significant size. If you typically download files through your browser, we’ve implemented an e-tag to facilitate the automatic initiation of partial downloads. In this scenario, no additional steps are required. However, if you are a user who employs terminal or command prompt tools, please adhere to the following instructions.

Generating the Link for Retrieval

https://<USERNAME>:<PASSWORD>@data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz

https://data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz	Link of the file.
https://	Secure protocol
freexml	Select your folder depending on Icecat subscription: – freexml for Open, – level4 for Full.
<USERNAME>:<PASSWORD>@	an option to include username and password into URL. Doesn’t work on PowerShell

Navigation Within Terminal

mkdir NewFolder	create new folder
cd NewFolder	enter folder
rmdir NewFolder	delete folder (must be empty)
rm NewFile.xml	delete a file
rm *	delete everything in a folder
cd ..	go to parent folder

Windows PowerShell

1. Open the Search on Windows, and search for PowerShell. For sophisticated scripts it is recommended to use Visual Studio Code. Save your script as <PowerShellScript>.ps1.

2. Enable Basic Authentication:

$username = '<USERNAME>’
$password = ‘<PASSWORD>’
$securePassword = ConvertTo-SecureString $password -AsPlainText -Force
$credentials = New-Object System.Management.Automation.PSCredential($username, $securePassword)

3. Now your credentials are stored as a token in $credentials variable.

4. Parameters:

-Method	HTTP method, for downloading it is GET
-Credential	way to send your login and password
-OutFile	save the response into the file
-Headers	specify the custom headers. Useful for Content Tokens
-Resume	if -OutFile is specified, tries to resume downloading on connection drop.

Examples:

# Example with Basic Auth
$username = "<USERNAME>"
$password = "<PASWORD>"
$url = 'https://data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz'
$outputFile = "FeaturesList.xml.gz"
$securePassword = ConvertTo-SecureString $password -AsPlainText -Force
$credentials = New-Object System.Management.Automation.PSCredential($username, $securePassword)

Invoke-WebRequest -Uri $url -Method GET -Credential $credentials -OutFile $outputFile -Resume  

# Example with headers
$url = 'https://data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz'
$outputFile = "FeaturesList.xml.gz"
$headers = @{
    "Api-token" = "<API_TOKEN>"
}

Invoke-WebRequest $url -Method GET -OutFile $outputFile -UseBasicParsing -Headers $headers -Resume

Source: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-7.4

GitBash

curl is pre-installed on GitBash. Navigate to curl section of the manual.

Beware that compressed XML file might return weird-looking output in your GItBash. Consider adding -o <FILE_NAME> to hide the log mess.

Ubuntu

curl

1. curl is pre-installed on Linux OS and GitBash, but you may check its presence:

sudo apt-get install curl
curl --version

2. Include your username :

https://<USERNAME>:<PASSWORD>@data.icecat.biz/File.xml.gz
https://data.icecat.biz/File.xml.gz -u <USERNAME>:<PASSWORD>

3. Other parameters:

-C <BYTES_TO_SKIP>	continue on connection drop, use -C – to continue from any place.
-u <USERNAME:PASSWORD>	Basic Auth.
-o <FILE_NAME>	save file with a different name. Highly recommended for GitBash.
-H <NAME: VALUE>	add custom header to request. Useful for Content Tokens.
-v/-vv/-vvv	enable verbose (detailed information). The more v after dash, the more info.
-s	silent, do not show errors and progress bar.
-X GET	use explicit method.
–connect-timeout 1000	timeout on connection, 0 by default.

Examples:

# Simple silent request with basic auth
curl "https://data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz" -s -u <USERNAME>:<PASSWORD> -C -

# Request with Api-token
curl "https://data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz" -C - -H "Api-token: <API-TOKEN>"

# Download to Features.xml.gz
curl "https://data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz" -u <USERNAME>:<PASSWORD> -C - -O "Features.xml.gz"

Source: https://quickref.me/curl

wGet

1. wGet is pre-installed on Linux OS, but you may check its presence:

sudo apt-get install wget
wget --version

2. Include the username and password to the link with one of two methods:

https://<USERNAME>:<PASSWORD>@data.icecat.biz/File.xml.gz
https://data.icecat.biz/File.xml.gz -username <USERNAME> -password <PASSWORD>

3. Other parameters:

–username	your user name for Basic Auth.
–password	your password for Basic Auth.
–header=<NAME:VALUE>	set the custom header, might be useful for Content Tokens.
-c/–continue	continue downloading on connection drop.
-O/–output-file=<FILE_NAME>	save file with a different name.
-t/–tries=<RETRIES>	set number of retries.
-T/–timeout=<MILISECONDS>	set all timeout values.
-r/–recursive	specify recursive download (get the page and everything linked to it).
-d/–debug	print lots of debugging information.
-q/–quiet	quiet (no output).
-v/ –verbose	verbose (detailed info), enabled by default.

Examples:

# Short syntax
wget -d -t 30 -O "FeaturesList.xml" https://valera-shop:mv3CmPjGEXVTHS3@data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz

# Long syntax
wget --continue --user="<USERNAME>" --password="<PASSWORD>" --output-file="FeaturesList.xml" --recursive --quiet --timeout=1000000 https://data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz

# Example with api-token
wget -q --header="Api-token: <API-TOKEN>" https://data.icecat.biz/export/freexml/refs/FeaturesList.xml.gz

Source: https://gist.github.com/antonydevanchi/b968f61a62e3b6ed711a6d7889aae084

If you still have any questions, feel free to Contact Icecat.