Downloading Email with 4D Internet Commands

Downloading Email with 4D Internet Commands

By Steve Hussey, CEO Alto Stratus LLC

Technical Note 99-46

Technical Notes for Technical Notes for 99-10-October 1999

Introduction

A wide variety of free email clients are available today: Claris Emailer Lite, Outlook Express and Eudora, to name but a few. Since these are widely available, and free, why bother going to the effort of writing your own email client?

You or your users need a custom interface.

Integration of email with a database sales automation system.

Receiving and processing of orders from web sites.

Ability to program schedules that will download email at preset times or when predetermined conditions are met.

Requirements

4D v6.5 plus 4D Internet Commands.

A connection to the internet.

An email account, the name of the POP server (or IP address), and the password to the account.

This solution is cross-platform and requires no platform-specific code.

The Process

Log on to the POP server and process the email using the IT_Review commands. You will need to be online until the last message is processed to do this. This method can only handle messages that are smaller than 32 KB.

Log on and download the messages to the local drive (then you can drop the connection if required) and then process the messages offline. Messages greater than 32 KB in length can be handled this way. This technical note uses this second method.

Log On

The first stage in retrieving your email is to log on to the POP server that processes your email account. To do this you will require:

An email account

A valid account name

A password

The sample database provided will connect you to a POP server after you have set this data in the preferences table.

Find The Messages

When you first log into the POP server, nothing will happen. You then need to send messages to the POP server to tell it what to do. The sample database sends a command to the server that queries how many messages are in the 'in-box.'

Download The Messages

The number of messages in the in-box is used to create a loop that will process the contents of the in-box. In the sample database the messages will be downloaded to the local hard disk and then processed. (They could be processed directly on the POP server instead.)

Look for Attachments

Messages that are being sent to you may have multiple file attachments. You will need to detect if any particular message has any attachments and extract the attachments as necessary.

Delete The Message

After the message has been processed successfully you will probably want to delete it from the in box – otherwise the next time you log on it will still be there, and will be downloaded again.

However you may want to check the size of the message, if it is too large you may choose to skip downloading it and leave it for later. This could be useful if you are traveling and are using a slow modem connection.

Log Out

After all desired messages are downloaded you will log out of the mail server, if you do not most mail servers will deny you further access (thinking that you are still logged in) until they time-out.

The MailBox Application

Launch the MailBox application

Select File: Preferences

Enter your ISP's POP3 server host name or IP address, your account name and password. The name of this depends on your ISP, it is often mail.ispname.com, or pop.ispname.com. Check with your ISP if in doubt.

Save the Preferences record.

Select Download from the Mail menu.

Your email and attachments will be downloaded from the specified mail server. The emails are temporarily stored in the 'Mail Files' folder and deleted once processed. Any attachments will be downloaded to the 'Attachments' folder inside the application folder. Once finished the application will log you out of the mail server.

Select List from the Mail menu to list and review your email.

The Code

MAIL_Download

The following arrays must be declared before they are used:

ARRAY TEXT ( ary_documents; 0 )  `<< Must be explicitly declared
ARRAY TEXT( ary_enclosureList ;0 )  `<< Must be explicitly declared

In the sample database, the user is given the opportunity to specify a folder into which the email will be downloaded.In a finished application you would probably make this part of a preference record, and store the values in the database. Alternatively you might create these folders in a fixed location such as in the ACI folder.

$DwnldPath := Select folder ( "Select folder to download email to." )

To maintain this value, it is duplicated to another variable (the reason for this will be explained in more detail later).

$DecodePath := $DwnldPath

When the email files are downloaded, you must specify both a complete path and the file name that they will be temporarily stored under. The Select folder function allows the user to choose the folder in which the files will be stored; the filename 'Temp' is added to this path. The first email file to be downloaded will be named 'Temp' and will be stored in the folder selected. The next file will be named 'Temp1' and so on. The name chosen is irrelevant since it is not related to the contents of the file.It is only a temporary measure.

$DwnldPath := $DwnldPath + "Temp"

The user is given the opportunity to select a folder for downloaded attachments.

$AttachPath := Select folder ( "Select folder to download attachments to." )

A connection is now made to the POP server:

MAIL_Connect

MAIL_Connect

Load the preference record:

ALL RECORDS ( [Preference] )

Log on to the POP server:

$Err := POP3_Login ( [Preference]POP_Host; [Preference]Account_Name; [Preference]Password; 0; pop3_ID )

POP_Host = Name or IP address of the POP server e.g. 'mail.earthmail.com'

Account_Name = your email account name such as 'John_Doe'

Password = your log on password

0 = aPOP or cleartext login (0 = Cleartext Login, 1 = APOP Login)

pop3_ID = a session ID number that is returned to you, you will need to use this number in future calls during this session

The MAIL_Connect method has now established the connection to the POP Server, the program flow returns to the MAIL_Download method to find out how many messages are waiting for you.

$Err := POP3_BoxInfo (pop3_ID; $MsgCount; $MsgSize )  `<< Number of messages & total size

$MsgCount = the number of messages in the mailbox

$MsgSize = the total size of the messages

You can use the $MsgSize value to decide whether to download the messages. The messages can now be downloaded:

 
 `<< Download the email files
For ( $Msg; 1; $MsgCount )
  $Err := POP3_Download (pop3_ID; $Msg;0 ;$DwnLdPath )
End for

pop3_ID = session ID

$Msg = message number from the loop counter

0 = header only or entire message (0 = Entire message, 1 = Only header)

$DwnLdPath = the path to download the file to.

At the conclusion of this loop you will have all your downloaded email files in your specified folder.

In normal usage you will probably want to delete the messages after they have been downloaded, you can use the following syntax:

4Err := POP3_Delete ( pop3_ID; startMsg; endMsg )

startMsg = starting message number (e.g. 1)

endMsg = end message number (usually the same value as $MsgCount)

This code has not been implemented in this sample database for safety reasons.

Now you need to process these downloaded files. You start by building a list (in the form a of an array) from the contents of the folder:

DOCUMENT LIST ( $DecodePath; ary_documents )

This command will build an array, the size of which equals the number of files in the downloads folder, where each array element is the name of one file. You can now loop through this array.

For ( $File; 1; Size of array ( ary_documents ) )

For each iteration of the loop you build the file name from the path to the folder (which includes the platform specific delimiter as the last character) and the file name which is contained in the array element:

 $FileName := $DecodePath + ary_documents {$File}

Now the header info can be extracted from the message. Who is the message from (the return address):

$Err := MSG_FindHeader ( $FileName; "From:"; $From )

Who was it addressed to? (That should be the user whose account you logged into.)

$Err := MSG_FindHeader ( $FileName; "To; "$To )

What is the subject?

$Err := MSG_FindHeader ( $FileName; "Subject:" ;$Subject )

What is the date and time when the message was sent?

$Err := MSG_FindHeader ( $FileName; "Date:"; $Date )

Notice the format for the date string that is returned – 'Sat, 18 Sep 1999 17:16:07 -0700'. The time is the local time that was set on the sender's computer and the offset from Greenwich Mean Time (GMT) that was set on their computer when the message was sent. (On MacOS this is set using the Date and Time control panel). Using the GMT offset you can adjust the time if necessary to your local time.

The next task is to extract the body of the message:

$Err:= MSG_GetBody ( $FileName; 0; 32000; $Body )

0 = the starting character offset

32000 = the maximum number of characters (from the starting point) to extract.

4D text fields can only store a maximum of 32,000 characters per field (not 32,767 which is 32 KB). If the message body was longer than that you could repeat the process and store the remainder of the text in additional fields or records.

The next command will determine whether the message has any attachments, and if it does how many there are:

$Err := MSG_HasAttach ( $FileName; $AtchmntCnt )

If $AtchmntCnt is greater than zero you can extract the attachments as required:


If ( $AtchmntCnt > 0 )
    $Err := MSG_Extract ( $FileName; 1; $AttachPath; ary_enclosureList )
End if

If there are any attachments, you extract them into the attachment folder. The names of the extracted files are placed in the array ary_enclosureList (which must be predeclared, as mentioned earlier). The extract command extracts however many attachments there are.

$FileName = the name of the temporary downloaded email file

1 = decode the attachments if possible (un-binhex for example)

$AttachPath = the location of the folder to extract the attachments to

ary_enclosureList = the array of attachments names

Now you can create the 4D record holding the email:

CREATE RECORD ( [Mail_Rcvd] )
  [Mail_Rcvd]From := $From
  [Mail_Rcvd]To := $To
  [Mail_Rcvd]Subject := $Subject

(You would probably want to parse the date and time into separate fields.)

  [Mail_Rcvd]Date := $Date
  [Mail_Rcvd]Body := $Body

You may need to record the names of the attachments that were downloaded:

If ( Size of array ( ary_enclosureList ) > 0 )
    [Mail_Rcvd]Attachments := ary_enclosureList {1} + "; "
    For ( $File; 2; Size of array ( ary_enclosureList ) )
      [Mail_Rcvd]Attachments := [Mail_Rcvd]Attachments + ary_enclosureList { $File } + "; "
    End for 
 End if

The above code concatenates the names of the attachments into one field.

Save the 4D record:

SAVE RECORD ( [Mail_Rcvd] )

Delete the temporarily downloaded file:

DELETE DOCUMENT( $FileName )

End the loop.

End for

Disconnect from the POP mail box:

MAIL_Disconnect

MAIL_Disconnect

$Err := POP3_Logout ( pop3_ID )

You can now list your email messages and review them.

Preferences

Internet tools allows you to set some preferences:

$Err:=POP3_SetPrefs (stripLineFeed; msgFolder; attachFolder)

stripLineFeed allows you to strip line feeds of PC generated messages

msgFolder allows you to specify the messages folder procedurally

attachFolder allows you to specify the attachments folder procedurally

Summary

______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________4D v6/v6.5 with 4D Internet Commands can be used to build sophisticated email applications quickly and easily. One major advantage of this approach is the ability to integrate email with a 4D database. An email client built with 4D is fast, can use a custom interface, and create a keyword indexed database of your email.

You could build sophisticated email responders to process and answer your email, and to build and manage address lists.