Using 4D Version 6.5 to Browse and Download Files via FTP

By Steve Hussey, CEO Alto Stratus LLC

Technical Note 99-30

Technical Notes for Technical Notes for 99-07-July 1999

The File Transfer Protocol (FTP) is a series of protocols that originated in 1971 as a means for different computer systems to exchange files. It is the primary means of transferring documents and applications from one computer to another on the Internet. An FTP site can be hosted on Unix, Mac OS or Windows. Client applications can also run on a variety of platforms, and can log onto any FTP server in order to upload or download text or binary files irrespective of whether the FTP server and client are based on the same operating system.

4D v6.0.x with PDM's Internet Toolkit or 4D v6.5 with the Internet Commands plug-in installed. The 4D v6.5 installer installs the Internet Commands in the new universal 4DX folder.

An Internet connection: the connection can be a dial-up connection via modem or a full-time connection. If you are using a dial-up (modem) connection on Mac OS, you can force a connection to open. If you are using a dial-up (modem) connection on Windows, you will need to have an open connection before connecting to an FTP site.

The 4D installer installs the Internet Commands plug-in in the new universal plug-in folder:

On Mac OS, this is in the System:Preferences:ACI:Mac4DX folder.

On Windows, this is in the C:\Windows\ACI\Win4DX folder.

Plug-ins installed in the new universal folder are automatically accessible to all copies of 4D v6.5/4D Server v6.5 installed on that computer. They do not appear in the 'normal' Mac4DX/Win4DX folders.

When specifying pathnames in FTP commands, you must treat file locations on the FTP site as Unix directories, even if you know the FTP host to be Mac OS or Windows running FTP server software. Whatever the platform, the FTP server software will internally convert the Unix pathname to the format it requires to serve its documents.

There are plenty of FTP client applications available for Mac OS, Windows, and Unix, most of which are either freeware or shareware. So why go to the trouble of implementing a FTP client in 4D? Consider the following scenarios:

You store the names and paths of documents or files that you need to download at some point in time. Using 4D, you could set up an automated process that would connect at a suitable time (for example, late at night when network traffic is hopefully lower) and cycle through a list of files to download. You could connect to multiple sites simultaneously.

You want to download files from a FTP site and store them in a database immediately.

You want to monitor na FTP site for new files. After connecting for the first time, you could traverse the site's directories and build a map of the site -- in effect a directory of all the existing files. On the next connection you could check for new files and download them automatically.

You have multiple files that need to be uploaded to different FTP sites. You could create a system that would allow you to add files (to be uploaded later) during the course of the day and batch upload them at night or at some other suitable time.

You wish to build an FTP client with a custom interface. Using 4D, you can build programmable FTP clients with custom interfaces.

The first time you use the application, you will be prompted for a anonymous logon name and your email address. These are used to logon to FTP sites that support anonymous logon.

Select the Add Site... from the FTP Sites menu. This creates a new FTP site record:

You enter the FTP site name here. The logon name and password take their default values from the preference record.

Connect to a site by clicking the Connect button. The following dialog is displayed:

This dialog displays the following information:

The name of the window is the FTP site name.

The home directory file/directory list for the site.

The size of each file. Directories may have their sizes listed as 0, depending on how the site is configured.

The type of each object in the listing -- either a file or a directory.

The date last modified for each item listed.

The Close button closes the connection to the FTP site and closes the dialog.

The Get button has one of two actions:

If a file is selected when you click the button, it will be downloaded to your computer.

If a directory is selected when you click the button, it will make the selected directory the home path and lists its contents. You can then select a subdirectory or file and click Get again, as required.

In the dialog box:

Clicking on a directory or file displays the full name in the display area at the top of the dialog.

The History drop down list maintains a list of the paths/directories previously visited.

Selecting a directory from the drop-down menu will position you back at that directory. This effectively allows you to navigate up and down the directories.

The FTP client application has a simple structure:

A Preference table that stores the default anonymous logon name and password.

A FTP_Site table that stores the names of FTP sites to which you want to connect.

FTP sites fall into two categories:

Private FTP sites --These sites are not open to the public. You need a logon name and password to connect. A good example is the partner FTP site at ACI US.

Public FTP sites -- These are often provided as a way of distributing software updates. You can logon to them by using the generic logon name 'anonymous' and the password 'your_email_Address@your_host.com'.

The first time you launch the FTP client application, it prompts for a logon name to connect to anonymous FTP sites. The default is 'anonymous' (as discussed). The system then prompts for your email address, which will be used as the default password for anonymous logon.

If you need to change these values, select the Preference item from the File menu.

The Connect button has the following Object Method:

      `<< OM: btnLogon
      `<< Created: 7/01/99 for ACI US by Steve Hussey,
      `<< Alto Stratus LLC shush@harborside.com
      `<< Initialize the arrays we will use to display the ftp directories

These arrays will be used to store and display (as scrollable lists) the directory lists from the FTP site:

   ARRAY DATE(arydateModDates; 0)
   ARRAY INTEGER(aryintKinds; 0)
   ARRAY LONGINT(arylongSizes; 0)
   ARRAY TEXT(arytextNames; 0)

The following array is used to store the history of your browsing for this site -- a list of the directories that you visited. It will always have at least one value -- the home directory of the site.

   ARRAY TEXT(arytextPaths; 1)

The connection ID for the FTP session is probably the most important value.

   C_LONGINT(FTP_ID)  `<< This is our ftp session ID
   C_TEXT(textFTP_Path)

It can take more than a few seconds to connect to the FTP site, so it might be a good idea to change the cursor display so the user has some feedback that something is happening. This code changes the cursor to the "OS busy" cursor -- the user then knows that the system is doing something. You could also display a progress meter or some other indication that the system is busy.

   SET CURSOR(4)  `<< Change the cursor so that the user knows something is happening

For Mac OS, you need to ensure that the TCP drivers are loaded. This also has the unintended side effect of forcing a dial-up (modem) connection if one is not already open.

   $Err := FTP_Init   `<< Load Mac TCP drivers - required for Mac OS only

This part of the code connects to the FTP site. You pass the site name, logon name, and password.

   $Err := FTP_Login ([FTP_Site]FTP_Site; Logon_Name; Password; 
      FTP_ID; WelcomeText)  `<< Log on to the ftp site

The function returns three values: an error code (with zero if all went well), the FTP ID and any welcome text message. The FTP ID is a long integer and is the reference required for most commands that you will use later. It is needed since you could be making multiple simultaneous connections -- even to the same site.

Many sites display a welcome message. This message may warn of access restrictions, time limits, or announce general usage policies. This application does nothing with this text. You could test for the presence of a message and display the text in another window if required.

If there was no error, you can proceed:

   If ($Err=0)

Record the root directory as the first directory you have visited.

   arytextPaths{1} := "/"  `<< We are starting at the root directory

Obtain a directory listing for this site. You need to pass the FTP reference ID.

   $Err   :=   FTP_Dir_Get (FTP_ID; "")  `<< Obtain a directory for this site

Clear the display variable:

   textFTP_Path  :=  ""  `<< Clear the on form variable

Center a window on the screen:

   WND_Cntr (560; 360; 5; FTP_Site; "Wnd_Cls")  `<< Center a dialog in the application window

Set the cursor back to the regular cursor.

   SET CURSOR(1)  `<< Set the cursor back again

Display a dialog that presents the user interface to the FTP site.

   DIALOG([FTP_Site]; "FTP_dlg")  `<< Display the dialog which shows the ftp listing

A display variable at the top of the form displays the full file or directory name of the item selected from the list.

The drop down list "arytextPaths" displays the history of the directories that we have visited.

The four grouped scrollable arrays list the directory information for the current directory. You could also use a plug-in such as AreaList Pro to display this data.

Four scrollable areas were created:

arytextNames

arylongSizes

aryintKindsText

arydateModDates

Each of these contains code in the form:

   textFTP_path  :=  arytextNames{arytextNames}

This code means that if you click on a row in the scrollable area, the value of textFTP_path will be set to the text value contained in the row clicked for that scrollable area. You need to have similar code in each scrollable area; for example, the code in the "arylongSizes" area is:

   textFTP_path := arylongSizes{arylongSizes}

Each scrollable area has the same code, since the user could click in any of the four scrollable areas for any row.

The form events must be set to recognize On Clicked.

The rightmost scrollable area should be brought to the front of the form, so that when you group the four scrollable areas together it will display the scroll bar. Only one scroll bar is displayed -- it is the one for the frontmost scrollable area.

After the user clicks the Close button, the window needs to be closed.

      CLOSE WINDOW
   Else 

If there was an error, display it so that it can be looked up.

      ALERT("Error: " + String($Err))  `<< If there was an error, display it
   End if 

The code calls the following methods:

      `<<<<< Declarations >>>>>
   $0 := 0
   PLATFORM PROPERTIES($Platform)
   Case of 
    : ($Platform<=2)  `<< Mac OS
      $0 := IT_MacTCPInit 
    : ($Platform>2)
         `<< TCP connection needs to be open on Windows
      $0 := 0
   End case 

   ARRAY TEXT(arytextNames; 0)
   ARRAY TEXT(aryintKindsText; 0)
   ARRAY LONGINT(arylongSizes; 0)
   ARRAY INTEGER(aryintKinds; 0)
   ARRAY DATE(arydateModDates; 0)
      `<<<<<    Constants    >>>>>

Our passed parameters:

   $FTP_ID := $1
   $Path := $2

This command is passed the FTP reference ID and the path to the directory to be listed:

   $0 := FTP_GetDirList ($FTP_ID; $Path; arytextNames; arylongSizes; aryintKinds; 
            arydateModDates)

It returns four arrays:

arytextNames – an array of the file or directory names.

arylongSizes – an array of the file or directory sizes.

aryintKinds – an array of the file or directory kinds.

arydateModDates – an array of the file or directory modification dates.

The aryintKinds array is not in a usable form; however, the array contains integer values where 0 indicates that the item is a file, and 1 that it is a directory. Therefore, this should be converted to more recognizable values in a loop.

First, an array of the correct size is created:

   ARRAY TEXT(aryintKindsText; Size of array(aryintKinds))

Loop through the aryintKinds array and test the values. Assign text values instead.

   For ($Row; 1; Size of array(aryintKinds))
      Case of 
      : (aryintKinds{$Row}=0)
        aryintKindsText{$Row} := "File"
      : (aryintKinds{$Row}=1)
        aryintKindsText{$Row} := "Dir"
      End case 
   End for 

This routine positions a window in the center of your screen, irrespective of the screen resolution:

      `<< PM: WND_Cntr
      `<< Created : 6/06/99 Steve Hussey, Alto Stratus LLC
      `<<<<< Parameters   >>>>>
      `$1 = window width
      `$2= window height
      `$3 = window type
      `$4 = window title
      `$5 = close box
      `<<<<< Declarations >>>>>
   C_REAL($1; $2; $3; $sw; $sh; $ww; $wh)
   C_STRING(255; $4)
   C_STRING(15; $5)
      `<<<<<    Constants    >>>>>
   $sw := <>ScreenWidth/2
   $sh := <>ScreenHeight/2 + 10
   $ww := Int($1/2)
   $wh := Int($2/2)
      `<<<<<    Start    >>>>>
   Case of 
   : (Count parameters=5)
      Open window($sw - $ww; $sh - $wh; $sw + $ww; $sh + $wh; $3; $4; $5)
   : (Count parameters=4)
      Open window($sw - $ww; $sh - $wh; $sw + $ww; $sh + $wh; $3; $4; "WND_Cls")
   End case 

While browsing an FTP site, you can click on any row -- each row represents a file or directory at that level. If you click the Get button, one of two actions will occur:

If a file is selected, it will be downloaded, if it has not been downloaded already

If a directory is selected, it will become the home directory and its contents will be listed.

The code to implement this is contained in the Get button's object method:

   Case of 
   : (Form event = On Load)
   : (Form event = On Clicked)

Set the cursor to the "OS busy" cursor as this may take a few seconds.

      SET CURSOR(4)
      textFTP_Path := ""

Test to see whether a path or directory has been selected.

       Case of 
       : (aryintKindsText{arytextNames}="Dir")  ` << This is a directory

Remember that the FTP protocol uses Unix file conventions and delimiters.

         $FileDelimiter := "/"

Build the path for the directory command, this comprises the selected directory name plus the delimiter:

         $Path := arytextNames{arytextNames} + $FileDelimiter

Add an element to the array that keeps track of the directories you have visited.

         INSERT ELEMENT(arytextPaths; (Size of array(arytextPaths) + 1))

Add the latest directory chosen to the array:

         arytextPaths{Size of array(arytextPaths)} := arytextPaths{(Size of array(arytextPaths) - 1)} +
               "/" + arytextNames{arytextNames}

Get the new directory listing:

         FTP_Get ("Dir"; $Path)

Otherwise, it must be a file rather than a directory:

         : (aryintKindsText{arytextNames}="File")  ` << This is a file

So there is no file delimiter:

         $FileDelimiter := ""
         $Path := arytextNames{arytextNames} + $FileDelimiter

Get the file:

FTP_Get ("File"; $Path)
End case 

Revert the cursor to the regular one:

      SET CURSOR(1)
   End case 

The FTP_get code is as follows:

      `<< PM:  FTP_Get
      `<< Created  : 7/01/99 for ACI US by Steve Hussey,
      `<<<<< Parameters   >>>>>
      `$1 = "File" or "Dir"
      `$2 = Path
      `<<<<< Declarations >>>>>
   C_TEXT($Path)
      `<<<<<    Constants    >>>>>
   $Type := $1
   $Path := $2
      `<<<<<       Start       >>>>>

The first parameter defines whether the selected item is a file or directory.

   Case of 
    : ($Type="File")  `<< File
      $Err := FTP_Prgrs (aryTextNames{aryTextNames})  `<< Set up the progress thermometer
      $Err := FTP_Receive (FTP_ID; $Path; $Path; 1)  `<< Receive the selected file

Error code - 48 indicates that this file has already been downloaded.

      If ($Err= - 48)
         ALERT("This file already exists (it has been downloaded already)")
      End if 

If the selected item is not a file but a directory, then obtain a directory listing for it:

    : ($Type="Dir")  `<< Directory
      $Err  :=  FTP_Dir_Get (FTP_ID;  $Path)
      End if 
   End case 

      `$1 = File name to download
      `<<<<< Declarations >>>>>
   C_TEXT($1;  $Text)
      `<<<<<    Constants    >>>>>
   $Filename  :=  $1
      `<<<<<      States       >>>>>
      `<<<<<       Start       >>>>>

The - 1, - 1 parameters force the window to display in the center of the screen. The window will have a title, which is the site name (contained in [FTP_Site]FTP_Site), and the progress file name will be the value contained in $Filename.

   $0 := FTP_Progress ( - 1;   - 1 ;  [FTP_Site]FTP_Site;  $Filename;  "Cancel")

The Internet Commands documentation provides a list of all error codes that may be generated.

An FTP server will disconnect accounts that do not show activity in a period of time determined by its administrator. Each command that interacts with the FTP server forces a reset of the inactivity timer for your session. The FTP_VerifyID command resets the inactivity time for the specified FTP session without altering the current state or directory. This allows a user to keep a session active.

When executed, the FTP_VerifyID command verifies that the connection has not already been closed.

If the session is still open, the command tells the FTP server to reset the timeout counter for the session back to zero.

If the connection has already closed, FTP_VerifyID returns the appropriate error code, free memory used by the FTP session, and return a zero value back to FTP_ID.

You would probably want to create a process that executes this command at a regular predefined intervals if you need to ensure that a connection is kept open.

Using the Internet Commands plug-in, it is a straightforward task to create an FTP client in 4D. A future technical note will document a method for uploading files via FTP.