E-mail Analyzer
By Pascal Pradier, Connectivity Program Manager
Technical Note 00-21
Technical Notes for Technical Notes for 00-05 May 2000
Introduction
The purpose of this technical note is to demonstrate how to analyze e-mail messages retrieved through the 4D Internet Commands plug-in. It also illustrates how to look for a specific string and how to extract data from it when it is found. An example database is included.
Principle
When e-mail messages are sent, it is not uncommon that the recipient's address is incorrect or that the receiving server is not available. This usually generates an error.
In some cases, it can be useful to analyze the error data, so that you can programmatically manage a typical reaction to an error type.
For example, if an address is returned as being incorrect, it is a good idea to extract the recipient's address and remove it from your address book. This keeps your address book free of any invalid addresses.
Design Choices
The main dialog of the database provided with this technical note allows you to define search criterias such as:
The contents to search for.
The range in the document where the search will be performed.
The direction of the search.
The status of the search (enabled or disabled).
Rules are stored in a preferences document that you can edit with a standard text editor.
It is possible to apply several rules to the same e-mail message, and the order in which they will be applied can be set.
The result of the analysis for each e-mail message can be used somewhere else in the database.
The following illustration illustrates the interface of the Rule Management dialog:
Rule Structure
Rules are stored in a preferences file located at the same level as the database's structure file.
The syntax of a rule is as follows:
[RULE]
[Criteria]String to search for[End_Criteria]
[Start]beginning line[End_Start]
[Stop]ending line[End_Stop]
[Offset]0[End_Offset]
[Active]Is the Rule enabled/disabled[End_Active]
[Dir] Direction of the search[End_Dir]
[End_RULE]
The beginning and end of each rule is marked by [Rule] ... [End_Rule] and a character with the ASCII value of 1 is inserted after each rule.
The string to search for is determined by [Criteria] ... [End_Criteria].
To limit the search range to the useful part of the e-mail message, so the analysis is not slowed down, the beginning and ending lines are defined. The search will only take place between those two lines. This data is delimited by [Start]beginning line[End_Start]...[Stop]ending line[End_Stop].
The Offset tag is not used.
The purpose of the [Active]...[End_Active] tags is to let you enable or disable a rule.
The [Dir]...[End_Dir] tags allow you set the direction of the search in the e-mail message, relative to the position of the criterion.
Internal Operation
Dialog management:
This flow chart describes the management and selection of the Rule preference file.
The path to the document is stored in the STR#1700 resource.
The method that manages the preferences file is named oRul_File. The code for this method is included in the Appendix of this technical note. The only unusual aspect of that method is that it calls itself if the file is not found.
Once the document is created or found, the oRul_Show method creates the arrays used to store the rules and assign their values. This method receives the preferences file path as parameter, opens the file, loops through the rules, and assigns the values to the arrays. The code for this method is included in the Appendix.
At this point, the dialog is displayed on the screen. It displays the existing rules and allows you to edit them or enter new ones.
Applying the rules
The actual processing of e-mail messages is performed in a separate process, so that it does not interfere with other possible processing. This the purpose of the method oML_Process_Errors. The code for this method is included in the Appendix.
The technique used in that method is simple: the code loops through a selection of e-mail messages stored in a selection of 4D records. In this loop, the oML_Parse method is called. The actual text of the e-mail message is passed to that method.
It then returns an error code that specifies if the e-mail message is valid or not. If the e-mail message is not valid, the tRes array will contain the requested data. For now, only the e-mail address is returned, but it would be easy to also return the error type, the e-mail text, and so on.
The following flow chart describes how rules are applied:
Analysis of an e-mail message
Method oML_Parse:
This method receives pointers to the rule and text arrays. It returns an error code based on the error it detected and also fills an array with the recipient's e-mail address. You could also use this array to return other data from the e-mail message.
The search is based on the principle that e-mail addresses are delimited by the characters "<" and ">". To retrieve the e-mail address, you just need to detect "<" and concatenate the following data, until you reach ">".
Summary
Using the system described in this technical note, you can manage e-mail errors without having to hard-code each case you are looking for. If additional error types appear, you just need to add the corresponding rule and your code will address that new error.
Appendix
oRul_Show project method
C_TEXT ($Rule; $PathRules)
$PathRules := oRul_File
If ($PathRules # "")
ARRAY TEXT (tCriteria; 0)
ARRAY INTEGER (tStartLig; 0)
ARRAY INTEGER (tStopLig; 0)
ARRAY INTEGER (tOffset; 0)
ARRAY STRING (5; tBefore; 0)
ARRAY BOOLEAN (tActive; 0)
` Retrieving the rules
$RefDoc := Open document ($PathRules)
If (ok = 1)
While (ok = 1)
RECEIVE PACKET ($RefDoc; $Rule; Char (1))
If ($Rule # "")
$From := Position ("[Criteria]"; $Rule) + 10
$Long := Position ("[End_Criteria]"; $Rule) - $From
$Crit := Substring ($Rule; $From; $Long)
$From := Position ("[Start]"; $Rule) + 7
$Long := Position ("[End_Start]"; $Rule) - $From
$Start := Substring ($Rule; $From; $Long)
$From := Position ("[Stop]"; $Rule) + 5
$Long := Position ("[End_Stop]"; $Rule) - $From
$Stop := Substring ($Rule; $From; $Long)
$From := Position ("[Offset]"; $Rule) + 8
$Long := Position ("[End_Offset]"; $Rule) - $From
$Offset := Substring ($Rule; $From; $Long)
$From := Position ("[Active]"; $Rule) + 7
$Long := Position ("[End_Active]"; $Rule) - $From
If (Substring ($Rule; $From; $Long) = "Yes")
$Active := True
Else
$Active := False
End if
$From := Position ("[Dir]"; $Rule) + 5
$Long := Position ("[End_Dir]"; $Rule) - $From
$Dir := Substring ($Rule; $From; $Long)
INSERT ELEMENT (tCriteria; Size of array (tCriteria) + 1)
tCriteria{Size of array (tCriteria)} := $Crit
INSERT ELEMENT (tStartLig; Size of array (tStartLig) + 1)
tStartLig{Size of array (tStartLig)} := Num ($Start)
INSERT ELEMENT (tStopLig; Size of array (tStopLig) + 1)
tStopLig{Size of array (tStopLig)} := Num ($Stop)
INSERT ELEMENT (tOffset; Size of array (tOffset) + 1)
tOffset{Size of array (tOffset)} := Num ($Offset)
INSERT ELEMENT (tBefore; Size of array (tBefore) + 1)
tBefore{Size of array (tBefore)} := $Dir
INSERT ELEMENT (tActive; Size of array (tActive) + 1)
tActive{Size of array (tActive)} := $Active
End if
End while
CLOSE DOCUMENT ($RefDoc)
End if
vCriteria := ""
vStartLig := 0
vStopLig := 0
vOffset := 0
End if
oRul_File project method
` Function that manages the presence and creation of the document that contains
` the filtering rules
` By Pascal Pradier
` 12/99
C_STRING (255; $PathRules; $0)
C_BOOLEAN ($OkOpenDoc)
C_TIME ($RefDoc)
$PathRules := Get string resource (17000)
If (ok = 0) ` Resource was not found
MAP FILE TYPES ("PREF"; "PRF"; "Name of the document to create")
$RefDoc := Open document (""; "PREF")
If (ok = 1)
CLOSE DOCUMENT ($RefDoc)
SET STRING RESOURCE (17000; Document)
$PathRules := Document
$OkOpenDoc := True
Else ` The document was not opened
CONFIRM ("The file containing the rules was not found, do you want to create it?")
If (ok = 1)
$RefDoc := Create document (""; "prf")
If (ok = 1)
CLOSE DOCUMENT ($RefDoc)
SET STRING RESOURCE (17000; Document)
$PathRules := Document
$OkOpenDoc := True
Else
ALERT ("It will not be possible to process e-mail errors...")
$OkOpenDoc := False
End if
Else
$OkOpenDoc := False
End if
End if
Else
If (Test path name ($PathRules) < 0) ` The document is not where it should be
$OkOpenDoc := False
DELETE RESOURCE ("STR "; 17000)
oRul_File
Else
$OkOpenDoc := True
End if
End if
$0 := $PathRules
oML_Process_Errors project method
If (Count parameters = 0)
<>ProcTraiteErr := New process ("oML_Process_Errors"; 32000;
"Error Management"; "Dummy"; *)
Else
oRul_Show
ARRAY TEXT (tRes; 3)
QUERY ([Temp_Message]; [Temp_Message]Processed = False)
FIRST RECORD ([Temp_Message])
$Nb := Records in selection ([Temp_Message])
For ($i; 1; $Nb)
vToParse := BLOB to text ([Temp_Message]Msg; Text without length )
$err := oML_Parse (vToParse; ->tCriteria; ->tStartLig; ->tStopLig; ->tOffset;
->tBefore; ->tActive; ->tRes)
[Temp_Message]Processed := True
[Temp_Message]Mailto := tRes{1}
SAVE RECORD ([Temp_Message])
NEXT RECORD ([Temp_Message])
End for
ALERT ("E-mail processing completed!")
ALL RECORDS ([Temp_Message])
DISPLAY SELECTION ([Temp_Message])
End if
oML_Parse project method
C_TEXT ($1) ` String to analyze
C_TEXT ($ToParse; $ReducedText; $ToSearch; $Return)
C_STRING (80; $Adr)
C_POINTER ($2; $3; $4; $5; $6; $7) ` Arrays that contain the rules
C_POINTER ($8) ` Returned array
C_LONGINT ($0; $Nb; $i; $j; $NbLig; $Pos; $Direction; $k)
C_BOOLEAN ($Flag; $Flag2)
` Filtering out the Line Feeds
$ToParse := Replace string ($1; Char (13) + Char (10); Char (13))
$ToParse := Replace string ($ToParse; Char (10) + Char (13); Char (13))
` Retrieving the e-mail date
$Pos := Position ("Sent"; $1)
If ($Pos # 0)
$ReducedText := Delete string ($1; 1; $Pos + 7)
$8->{3} := Substring ($ReducedText; 1; Position (":"; $ReducedText) - 4)
End if
$Nb := Size of array ($2->)
For ($i; 1; $Nb) ` For each rule
If ($7->{$i}) ` If the rule is active...
$Dir := $6->{$i}
$ToSearch := $ToParse
$NbLig := 0
$ReducedText := ""
` ... decreasing the number of lines to analyze based on the transmitted criterias.
Repeat
$Pos := Position (Char (13); $ToSearch)
$NbLig := $NbLig + 1
If ($NbLig >= $3->{$i}) & ($NbLig <= $4->{$i})
$ReducedText := $ReducedText + Substring ($ToSearch; 1;
Position (Char (13); $ToSearch))
End if
$ToSearch := Delete string ($ToSearch; 1; $Pos)
Until ($NbLig > $4->{$i})
` Searching the String
$Pos := Position ($2->{$i}; $ReducedText)
If ($Pos # 0) ` If found
$Flag := True
$j := 0
While ($Flag)
$j := $j + 1
` managing the direction for the search of the e-mail address (before or after)
If ($6->{$i} = "Before")
$Direction := $Pos - $j
Else
$Direction := $Pos + $j
End if
If ($ReducedText[[$Direction]] = "<")
$Flag := False
$Adr := ""
$Car := ""
$k := 0
$Flag2 := True
While ($Flag2) ` From here the address is retrieved character by character
$k := $k + 1
$Car := $ReducedText[[$Direction + $k]]
If ($Car = ">")
$Flag2 := False ` The address is built so we go out of the loop
Else
$Adr := $Adr + $Car ` Building the address
End if
End while
$Return := $Adr ` Returning the address
$0 := 0
$i := $nb
Else
$Return := "The address was not found!"
` The character "<" is not found, hence no address
$0 := -1
End if
End while
Else
$Return := "The criterion was not found!" ` The string is not found
$0 := -2
End if
End if ` End of active rule
End for
$8->{1} := $Return
-