4D Web Serving Revisited

4D Web Serving Revisited
By Guyren Howe, ACI Technical support
	Technical Note 98-12

Technical Notes for 98-04-April 1998

4D has uniquely easy-to-use web serving capabilities built in. Nevertheless, the flexibility of these facilities means that you will need to understand a few issues to use them effectively.

This Technote supplements the Web Services chapter of the V6 Language Reference Manual. It is not a complete explanation of these capabilities, but rather an alternative explanation of some of the main concepts involved, to aid in learning to use this sophisticated feature. You should still read the Web Services chapter of the Language Reference Manual before attempting to serve material to the web from 4D. You should also familiarise yourself with the basics of web servers and HTML. This information is plentiful on the internet, for example at:

<http://www.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/Information_and_Documentation/Beginner_s_Guides/Beginner_s_HTML/>.

This Technote will describe the main differences between 4D client and a web browser, along with how 4D approximates the behaviour of 4D client within the limits of the web browser. There are also some code samples along with a framework for implementing sophisticated web access from your 4D database.

It is important to understand that 4D client and a web browser communicate with 4D in very different ways.

4D client is highly programmable and maintains a continuous connection with the server, through which it can be aware of what is happening in the database at all times. Forms and objects on forms can all have methods attached to them; the client program can have multiple windows and processes running; and all of these things can communicate with each other and with the user.

None of these things are usually true of a web browser. 'Usually' because it is possible to get a web browser to do some of these things, but you need to send a special-purpose program to the web browser to do them. These programs can be implemented through Java, Javascript, special browser plugins, or ActiveX controls. While you can achieve sophisticated effects with such techniques, they are well beyond the scope of this Technote. We will restrict ourselves here to what can be achieved with 4D, the 4D programming language, HTML, and a standard web browser.

When a web browser connects to 4D, it expects 4D to act just like any web server. The browser sends 4D a message which is intended to mean 'give me page x', or just 'give me your default page'. 4D then sends back a page, which is displayed in the browser window. The user usually reads the page for a while, possibly enters or modifies information in it, then clicks a button which sends information back to 4D. But the user may not do this at all; they may read information on the page 4D sent and then, perhaps by using their 'Bookmark' menu, might start using pages from another web site entirely. The user might also use the 'Go' menu or the 'Back' button to retrieve a page that was previously displayed by 4D, usually loading the copy they received last time from their local hard disk, without reloading that page from 4D, and without contacting 4D in any way. There is no ongoing connection between the browser and 4D in between these communications.

Here we find one potential problem: when the user employs the back button or the Go menu in their browser to view a page that displays out-of-date information, some other user may have modified or deleted information on that page, but 4D has no way of updating the information on it, because there was no connection back to 4D before the page was displayed.

Another difference between a web browser and 4D client is that a web page can have most of the same sorts of objects you can have on a 4D form, but they can't be placed as accurately. Rather than being placed as if they are in a draw-type program, anywhere on the page, a web browser is more like a word processor, which treats pictures, fields and so on like big characters in amongst lines of normal text. These 'big characters', just like normal characters, are placed left-to-right and top-to-bottom. 4D translates the objects placed on a 4D form to a web form in this same way. So while any form can be displayed more-or-less on a web browser, the results may come out looking very different from the original. 4D's web-aware option in the form wizard will take this into account, generating a form that will display very similarly on both 4D client and on a web browser. So while you do not have to use the web-aware option in a form you are going to display on the web, it will avoid unexpected display problems on the browser if you do.

The other important difference between a web form and one displayed in 4D Client is that the objects on the web form aren't programmable. Fields can be set to have a certain maximum length, and buttons can be used to send the information entered on the page back to 4D. Pictures and text can also be buttons, and pictures can tell 4D where they were clicked on. But that's all the form can do.

It is very important to keep these limits in mind when you are designing web forms in 4D, and when designing the methods associated with them.

The only way a web browser can send any type of information to 4D is through a URL (Uniform Resource Locator, or what is commonly called a web address; for instance, the URL for ACI US' main web server is http://www.acius.com/). A URL has up to three parts. Here is an example of a URL:

A URL begins with a protocol, which names a way for a browser and another computer to format the messages they send each other. The protocol will always be http:// when communicating with 4D.

The protocol is always followed by the address of a web server (remember, as far as the browser is concerned, 4D is acting just like any other web server) and a slash ('/') to mark the end of the address of the server. The address will be fixed for any web server. For example, ACI France maintains a 4D-based web server, and it is always at the address http://bugs.aci.fr/. A protocol and machine address like this is the minimal necessary information for a URL.

This necessary part of a URL can be followed by as much extra text as required, in what is referred to as the Path. 4D employs the path to represent what is going on during the session with 4D.

Buttons, pictures and text links on a page, which usually represent a link to another page, contain an encoded URL. In fact, this is the only way for a web page to represent a link to a URL. This means that someone using a browser will only get information back from 4D after clicking on a button, a picture or a text link.

There are four types of connections from a browser to 4D:

A 4D Method Call from a Simple HTML link: Here, the path begins with /4DMETHOD/ and is followed by the method name and a slash ('/'). If there is any text in the path after that, it is passed to the method in $1 (the text will include the slash). This means you can set up a link as a simple method call. If you are going to compile a database expecting these types of connections, any methods being called in this way will need to contain a suitable text or string declaration for $1;

A 4D Method Call from an HTML Form: You should read the section 'Binding HTML Objects with 4D variables — Part 1'. A suitably set up form will call a 4D method that you nominate, and will cause the values entered into the form to be passed into 4D process variables — this is discussed in more detail later. Note that a form can have as many POST buttons as you wish. Each button can call the same or different 4D methods, but they cannot encode extra information into the path. Any information in the path after the method name will simply be ignored;

4D Forms in use: When the user is interacting with 4D forms rather than external HTML files, 4D appends extra information to the path to indicate which process is standing in for the user, how many forms have been sent to the user of this process so far, and which button on a page was clicked. 4D uses this information to provide the user with an error message if he goes back to an earlier page in their browser, when he tries to use a button or other link from that page which communicates with 4D. All of this is handled automatically. When you are presenting 4D forms to the user, you can just write a form as you normally would, except that you must use only a subset of the normal forms programming techniques. This is discussed in more detail below;

A Simple Connection to the Server Without a Method Call: If a connection is made to 4D, and that connection does not include '/4DMETHOD/' at the beginning of the path, then the connection activates the On Web Connection method. This method, too, can receive extra information in the path through $1 and $2 ($2 contains information sent by the browser). Note that any such connection will always activate a new web process and will count against the total number of allowed web connections on the machine, so once a user has initiated a connection with 4D, the web pages you send them should always connect back to the server through /4DMETHOD/ URLs, so you don't start up unnecessary web connection processes.

Note that any of these types of connections except the '4D forms in use' type, can be used to initiate the first connection from a browser.

In order to make all of this look like a normal client to 4D's database engine, 4D starts up a process to mediate between the on-and-off nature of the web browser connection and the ongoing connection that a 4D client must represent. That process is effectively the client as far as the database engine is concerned.

Your 4D method can respond to a connection from a web browser by executing a normal command to display a 4D form (such as MODIFY RECORD, MODIFY SELECTION, or DIALOG). You can use a subset of the normal 4D form design features in a form displayed in a web browser.

One option is to display normal 4D input and output forms with automatic buttons and without other methods. Doing this does not require any special steps. 4D will automatically display an approximation of the 4D input and output forms and the information in the database when the user clicks on a next, previous, save, cancel, or whatever button. If the user uses the Back button or the Go menu from their browser to display a page that is no longer accurate, 4D will display an error message for the user after they use any of the buttons on that page.

You can also present a form to the user in which there are buttons with methods attached to them (they might also be automatic buttons).

You can write these methods using a subset of the features available in 4D client. You should read the section Web Services, HTML Support in the Language Reference for full details of this. The most important limits are:

The user's web browser will only wait a certain amount of time before assuming that the connection to the server has been lost and producing an error message (see the section 'Web Connection Timeout' in the Language Reference Manual). Similarly, 4D is usually set up to only wait a certain amount of time for a reply from the browser before it will assume that the user is no longer interested in using 4D. In this case, 4D will immediately close the web connection process. Because of this, you should ensure that your method sends something back to the browser within a reasonable period of time, and you should not assume that any interprocess variables or semaphores being managed by a web connection are still being managed by that connection after a page has been sent to the user.

There is no menu bar and only a single window in the web browser; you must ensure that every control needed is on the current form, and that you manage the whole session within one window.

Page zero of multipage forms is not supported, so any objects you want on form pages must be placed directly on those pages.

Form events On Load, On Unload and On Clicked are supported but all other form events are not.

On Clicked is only supported for buttons and button grids. Invisible buttons must have a picture behind them or they will not function or be displayed.

Automatic actions are supported wherever they make sense.

Subtables are not supported.

Invisible, 3D and grid buttons and button grids placed over pictures are translated into image maps. 4D transparently calculates which of the buttons or which part of a button grid was clicked on, depending on which part of the image was clicked on in the browser. Invisible, 3D and grid buttons which aren't in front of images are ignored.

Grouped scrollable areas become separate scrollable lists.

The Userset is not supported.

All of the commands execute on the web server machine (see the section Web Services, HTML Support in the Language Reference Manual).

Horizontal lines are translated into HR tags, and other graphical objects (rectangles, lines other than horizontal, ovals) are ignored.

Background pictures are not supported.

While in this mode, 4D makes the web client look just like any other type of client as far as other connections to the database are concerned (with the single exception that the web users are counted against a separate licenses count). If the user modifies a record, that record will be locked until their web connection process closes it or until their web connection is closed. If the user tries to modify a locked record from their browser, they will get a message to that effect and will only be able to view the data. And so on. The message here is that while you might need to be careful about how you write the process that handles a web connection, you don't need to write normal 4D client processes any differently than you would if there were no web connections in the database.

It is possible to embed HTML, including Javascript and the like, in 4D forms destined for a web browser.

There are two ways to include HTML within 4D forms.

A 4D static text object containing the name of an HTML file in curly braces like this:

{Headbanner.html}

will make 4D include the contents of the file referred to in the HTML version of the form at that point. The name you put in the curly braces must be a path relative to the location of the structure file, or relative to the root chosen with the SET HTML ROOT command if that is used. Note that 4D does not currently support references to Java or Javascript files in this way.

A 4D text variable can also contain HTML text which is sent directly to the browser at that point in the form (so you could send Javascript this way, for example). You indicate that the text object contains HTML by making the text object's first character ASCII 1. The easiest way to do this is usually to store the text in some convenient place (an external file, a field in the database, a resource, or directly entered into the code), and use a method to set the value of the variable with ASCII 1 inserted in front (for instance, HtmlObj:=Char(1)+'…HTML Code…').

The alternative to executing a 4D command to display a 4D form to the browser is to send an external HTML file to the browser. This is always done with the command SEND HTML FILE.

It is important to remember that SEND HTML FILE, just like the normal ADD RECORD, DISPLAY RECORD and such commands, acts like a method call. That is, the method you ran it from is left on the stack until you tell 4D to return to it and it ends. In a form, a return is set up through a CANCEL or similar command; in web pages, a return is set up through a link to a method that does a SEND HTML FILE (""). An example is given in the Language Reference Manual in which an external file is sent which contains normal URL links to other files, with one of those files eventually linking to a method which returns to the method which sent the first page.

You can achieve a variety of sophisticated interactions between those pages and 4D. The following example demonstrates a likely basis for such a technique.

It will be common to employ $1 and a loop to work out which file to send next, using a method like this:

   WHILE (ContinueHTMLLoop)
      Case of
         :(PageParam="/1")
            <do some stuff here>
            SEND HTML FILE ("Page1.html")
         :(PageParam="/2")
            SEND HTML FILE ("Page2.html")



      End Case
   End While

and another method called say ReturnMethod which looks like this:

   If ($1="/END")
      ContinueHTMLLoop:=False
   Else
      PageParam:=$1
   End if
   SEND HTML FILE ("")

Then you call the first method from On Web Connection with a suitable PageParam set up, and you can choose which page you want to send next by returning a suitable parameter in your call to ReturnMethod. For example:

http://www.myserver.com/4DMETHOD/ReturnMethod/1

would result in Page1.html being sent, while

http://www.myserver.com/4DMETHOD/ReturnMethod/END

would make the first method end. You would then be back either to the line following the SEND HTML FILE command that first sent a page that linked to that method, or otherwise to your application's default menu bar.

Note that your web pages can just link to each other directly, using a URL, so you would not do something like the example unless you needed to do some processing in addition to just linking to the other page. But you can see the example as a kind of framework and a demonstration of the flexibility that 4D's web features can give you. A practical example is likely to encode more complex information into the $1 parameter. Note that external HTML files can have several links to the same method on a page, any or all of which can pass different parameters to $1. These links can also connect to different methods. There is a large range of possible interactions between the external files and 4D. Although they are fairly easy to use, it is easy to set up interactions which do not ever return from some of the SEND HTML FILE calls, so be careful that you do not generate a memory leak with this feature.

When 4D sends an external HTML file to the browser, it acts mostly like a normal web server, except that it will substitute the value of a 4D variable into the HTML text being sent, wherever that variable's name appears in square brackets. For instance:

<H1>Welcome to the company database, [FirstName].</H1>

Note that the text of the variable FirstName is substituted immediately and without modification into the HTML being sent, so that text can include HTML, Javascript or whatever you like this way. It could even be substituting values from the database into the middle of some Java or Javascript code.

Some developers using 4D's web serving capability will generate an HTML file on the hard disk from 4D (using Send Packet), before sending that file. This can be a useful technique where a lot of the content of the HTML being sent needs to be chosen or modified often.

Apart from this value substitution on the file being sent out, 4D otherwise acts as a normal web server. If the file being sent includes HTML links to other files on the same computer, they will be sent normally as well.

It is worth noting that you can achieve more sophisticated modifications of outgoing HTML using the AP SET WEB FILTER command in the most recent ACI Pack. This command establishes filter methods which are passed HTML code coming into or going out of 4D.

The one major issue remaining is how to use values in 4D that were entered into forms in external HTML files. These forms might be sent to the browser from 4D via Send HTML or by a link from another page previously sent by 4D. The form might even be on a page sent from some other server, but with a button that links to 4D.

Another form of this question is 'How can 4D be used as a CGI'?

Any link to 4D from a normal HTML file must connect to a 4D method from a Post button or from a link on a form, if it is going to send forms information to 4D. In this case, the objects on the form must have had names and default values set up in the HTML for the form. When the form is submitted, the values entered into the objects are sent to the 4D method that the button called. The method receives these values as process variables with the same names as the objects on the HTML form. Many methods to handle such forms will do little or nothing more than copy these variables into fields and save a record.

Check boxes and radio buttons are a special case. These objects can be given a default text value in the HTML of the form. If the HTML check box or radio button is checked or activated, the process variable in which 4D receives the value for that object is set to that default value. If the object was not checked or activated, the value of that variable will not be changed. So to handle these objects, the process variables associated with them must have their value set to something other than the default they are given in the HTML form, then the form should be sent, and the method that receives the information from the form must check whether those variables have changed their value.

An important consequence of all of this is that if you are going to compile your 4D web-serving database, you must declare the process variables which will receive a value from an external HTML form as text or string variables.

Note that a form can have several POST buttons, which can POST to the same or different 4D methods. Unlike other connections to 4D, though, any extra information after the method name in the path will not be sent to the method in $1. This information will be ignored.

A final comment: it is possible to want to use a large number of similar forms or to generate HTML from within 4D, resulting in forms being sent to the browser with so many variables that it is impractical to code references to all of them in 4D code.

In this case, you can use the GET POINTER command to access even very large numbers of variables by name. Note that you will not be able to compile your database unless you explicitly declare every variable your HTML form might contain.

This Technote supplements the Web Serving information in the 4D V6 Language Reference Manual. While 4D's web serving capabilities make presenting information from a database on the internet simpler than any other product on the market does, the flexibility 4D provides in doing this requires careful attention to how these facilities are used, if you are to take maximum advantage of them.

Using 4D's web serving facilities requires a choice between automatic conversion of 4D forms and transmission of external HTML files which must then be linked into 4D methods. These techniques can also be used together.

4D forms converted to the web can only employ a subset of the full programming features of 4D forms. On the other hand, external HTML files must be linked into 4D methods. These methods will generally either receive information from the path passed in the link, or will receive process variables from a form in the HTML file, or both. You must carefully ensure that these connections don't result in memory leaks.

By understanding these options and carefully designing your web links, you can achieve sophisticated web connections from your database easily and flexibly.