Tutorial 03: Automating FTP sessions: backup and upstreaming

November 2nd, 2006

In this third tutorial we learn how we can leverage Fileaze’s file filtering, scheduling and network operations to monitor one or more folders on our hard disk and upload only files that change to an FTP server. This is sometimes known as “upstreaming”.

Note: this tutorial presumes that you followed the previous ones and are able to perform the operations explained there.

Suppose you have one folder that contains files you want to be always updated on a remote server. This could be the case of critical files you want to preserve from hardware failure or equipment theft. Or it could be a local folder containing web content (HTML and image files) that you want to edit locally using your favorite applications and be automatically updated online. In any case, the sequence of operations you’d manually perform are: Every time I change a file I must remember to open my FTP application, open the connection, upload the file and close the client. Optionally I could want to encrypt the file before uploading. You could do this periodically, every day or week but you should be sure that you remember about all the files you changed. Or you could use a backup application but they usually don’t support FTP so Web upstreaming is not possible. Wouldn’t it be nice if this could happen in the background, every minute or so, without any intervention and even without remembering to run the backup and upload applications?

Automating this kind of task in Fileaze is really simple. Let’s simplify and suppose we have a C:\online folder where we put files we want to publish to a Web server. At least a simple Hello World HTML file named index.html.

This is the content of the file

and here’s how it appears inside our Web browser

We want Fileaze to monitor the entire C:\online folder for changes so let’s create a new job and add a Folder source. We’ll probably want to include subfolders and keep the folders’ structure so our FTP server will reflect the content of our local folder:

2006-12-19_152036.jpg

Now we need to select and upload only files that have changed since last time. Here Windows’ filesystem comes to the rescue with the Archive attribute: every time you create or modify a file, Windows sets the Archive bit on that file to flag it for archival (this is used by all backup applications). What we need to do is just open the Folder and file filter and set the attributes correctly:

This tells Fileaze to include only files that have the Archive bit set (indicated by the checkmark) and whatever the state of the other three attributes (this is the meaning of the little squares in those checkboxes).

Let’s close the filter window and save the source.

Now we’ll add the FTP action. Move to the Actions tab and select New..FTP. In the window that opens we need to tell Fileaze what FTP account to use:

The first time we create this kind of action we won’t have any FTP account setup so let’s click the dotted button at the right of the FTP account combo and create an account clicking the New button:

Saving will get back to the FTP accounts window. Let’s select the newly created account and click the Select button on the right

This will bring us back to the action’s window and select the account in the combo. We can also add a path to a remote FTP folder if we want. This depends on the specific server and how we want to use it. For example:

Let’s save the action and the job and try to run it. What happens is that Fileaze finds the archive attribute on the newly created file and uploads it

If the FTP server is mapped to a Web server, you’ll find your file online:

We’re not done yet. After the files have been uploaded, we need to clear their archive attribute otherwise they’ll keep on being uploaded at every job run, even if they have not been changed. To achieve this let’s add a Set attribute action and set it up this way:

This tells Fileaze to clear the archive attribute after upload and leave the others to what they are. Now every job run will set files to “don’t upload” and they will be uploaded again only if they are modified by another application.

Now on to the automation! We want the upload to work without our intervention. Let’s edit the job and select the Schedule tab. This is where we can setup recurrent job runs that happen in the background, even without Fileaze running. For an upstreamer we could want the job to run every minute but you can fine-tune this to your preference:

Let’s save the job again. Fileaze will ask for the system password of the current login account. This is needed for the job to run in the background:

With these settings, the job will run every minute without needing FIleaze to be running and even after a system restart. This means that the folder will be under constant monitoring and every time a new file will be created or one will be modified, it will be uploaded to the FTP server. If we edit the index.html file and change it to this:

within one minute, refreshing our browser this is what we’ll see:

The file has been updated online without us doing anything and just because we saved the file locally!

A couple of notes:

  • You can easily monitor more folders or single files or any other source that’s supported by Fileaze. Just add them to the sources folder and everything will be monitored and upstreamed.
  • If you want to start with an existing folder, probably some of the files could not have the archive attribute set so you’ll need to edit that folder’s properties using Windows’ interface and flag the folder for archival (this is done through the Advanced property dialog) and accepting to recursively set the permissions when asked.

Adding optional encryption

When data you’re upstreaming is not public and your FTP server is not mapped to the Web, you could want to add encryption to your files. This is easily done by adding an Encrypt action to the job.

Consider though that the Encrypt action encrypts files that are found in the sources and this is not always what you’d want to end up with, especially in this case since we only want to encrypt files for upload. For this reason we need add a Decrypt action after the FTP upload that will restore our files to what they were before the job run.

First create the encrypt and decrypt actions:

Be sure to enter the same password in both actions. The options to automatically add and remove the .flz extension to the enctypted files won’t have much effect locally since the files are encrypted and then decrypted but it’s better to keep them enabled so that the files on the FTP server will carry the .flz extension and will be easier to recognize.

Use the Move up button to move the encrypt action after creating it since by default they will be both created at the end of the action list. At the end of all this, the actions tab will look like this:

With these changes, every minute the job will run, it will encrypt files that have changed since last run, add the .flz extension, upload them and decrypt them locally. Finally they’ll have their archive attribute cleared. If we surf to the index.html.flz file this is what we’ll see:

The file is online, protected and safe!