DPT Language Reference

Database Programmer's Toolkit

Language Reference and Compatibility Guide



The terms "Model 204" and "204" are trademarks of Rocket Software Inc., and that fact is acknowledged wherever those terms are used in this document. Likewise "Sirius xxxxxx" (anything) are trademarks of Sirius Software.

Contents


Notation Conventions in this Document

In simple cases, the common backus-naur style symbols are used to denote the required syntax of commands and statements, similar to the Model 204 manuals, viz:

Sometimes, especially where more formal notation would be very complicated, examples are used instead. MONITOR is a good example, where there are a number of "flavours" of the command, each with individual options.



Part 1: Commands


General Command Compatibility and Processing Notes

Line continuations
All command lines can be continued with a hyphen on DPT. This differs from M204, where only some commands can be continued.

IN filename
As with User Language, IN clauses can not include the "AT location" phrase, since there are no scattered groups on DPT.

Quote characters in commands on DPT
Some commands take options which can be surrounded by quotes. On Model 204 the only accepted quote character anywhere (including User Language) is the single-quote, and this is respected. However, with some commands that interact with the operating system, the double-quote character is also allowed by DPT since it is commonly used, sometimes even compulsory, on many platforms. For example:

ALLOCATE F1 'MY FILES/F1.DPT'    /? usual M204 convention        ?/
ALLOCATE F1 "MY FILES/F1.DPT"    /? equivalent on DPT            ?/
ALLOCATE F1 'MY FILES/F1.DPT"    /? no good - must not mix types ?/
This can allow an alternative to the usual convention of repeating quotes when you need to put quotes within quotes, since a value starting with one type of quote can only be ended with the same type. E.g
=MIME DEFINE .txt 'text/plain; charset="UTF-8" '
=MIME DEFINE .txt "text/plain; charset=""UTF-8"" "   /? equivalent but less neat ?/
Note that in the above example the semicolon in the MIME value would normally have indicated a line-split in the usual M204 way, and didn't only because it was enclosed in quotes. Terminal line input case translation (controlled by *UPPER/*LOWER) can however not be subverted like this.

Custom options
Some of the M204 commands have DPT custom options. Many of these can be disabled using the CSTFLAGS parameter, but a few are essential for system operation and cannot be disabled. In any case all custom options are covered in the by-command discussions below.


Individual M204 Commands - Processing and Syntax Differences

Model 204 commands that do not feature in this section can be assumed to either work pretty much the same on DPT, or not be emulated at all (see cross-reference). When an unsupported Model 204 command is issued, the DPT message usually gives some idea of the reason, and in some cases suggests a similar alternative. By default such messages are information level (CLASS=I) and processing is allowed to continue.

ALLOCATE

ANALYZE

BROADCAST

CHECKPOINT

COPY DATASET

COPY PROC

COPY STREAM

CREATE FILE

CREATE GROUP

DEFINE FIELD

DISPLAY FIELD

DISPLAY FILE

DISPLAY GROUP

DISPLAY GTBL

DISPLAY PROCEDURE

EDIT

FREE

HALT

INITIALIZE

LOGCTL

This command is partially supported. See also the
equivalent $function.

Nearly all the supported options have a wider range of allowable values than on Model 204, as follows.

LOGKEY

LOGWHO

MONITOR

This is actually several different commands with some shared behaviour between them. The basic form is: Variations of the command are of two main types, namely statistics monitoring variations and miscellaneous other variations. Let's take the former: The USERS variation has a couple of special options in addition to the STATS option mentioned above. The FSTAT variation can also be restricted in scope: Other miscellaneous variations of the MONITOR command are:

MSGCTL

OPEN

On DPT this command is largely the same as on Model 204, except that is has a special variation of the parameterized format to open a file in deferred update mode.

Examples

*One-step:
OPEN SALES, 1STEPLOAD                        //Default
OPEN SALES, 1STEPLOAD, 50%                   //Invalid from V2.19 - use LOADMEMP parameter

*Multi-step:
OPEN SALES, TAPEN, TAPEA                     //use default formats
OPEN SALES, TAPEN, TAPEA, 6                  //stringize numbers - pad to length 6
OPEN SALES, TAPEN, TAPEA, , NOPAD            //stringize unpadded - length irrelevant
OPEN SALES, TAPEN, TAPEA, , NOCRLF           //binary format numbers, length byte strings
OPEN SALES, TAPEN, TAPEA, , NOPAD, NOCRLF    //length byte on both strings and stringized numbers

PROCEDURE

REDEFINE FIELD

RENAME FIELD

RENAME PROCEDURE

REORGANIZE

This command is more comprehensive than the Model 204 REORGANIZE OI command, but does include its functionality. On DPT the command is used for both index-only and full-file reorgs.

The other main difference from M204's REORG OI command is that on DPT the unload and the load phases of the reorg happen with a single invocation of the command, whereas on M204 output files are created that must be sorted and then loaded by the Z command. The DPT scheme saves all the hassle of configuring JCL and tracking separate job steps at run time.

A third, more low-level, difference is that on Model 204 the output of the REORG IO command consists of sequential files identical to those from a deferred update run. On DPT the intermediate sequential files are a special custom format used by this command. The older deferred update file format is completely different.

DBAs often want to make file or field changes as part of a full-file reorg (see recipes in the DBA guide), and in such cases using explicit =UNLOAD and =LOAD commands with an intervening INITIALIZE gives greater control than a single REORGANIZE command. The underlying processing is virtually identical, but it may be slightly slower overall because DPT is given less chance to take shortcuts, cache data in memory, etc. For example when using separate unload+load, the intermediate data will always be written out to disk, even for a smallish file which would all fit in RAM.

RESET

RESET COMMAND

STATUS

TABLEB

UTABLE

USE

DPT supports USE PROC and USE {sequential file} correctly, but the nature of options such as PRINTER, $JOB and $PUNCH is intrinsically mainframey so there is much less support. The only commands of this type currently available on DPT are USE $PRINT and USE $EMAIL. Even with these the subobtions do not match M204's.

Depending on feedback this is something that can be enhanced in later versions. For example USE PRINTER {printer name} ... etc. might be handy where users have several printers, since use $PRINT selects the default Windows printer. More detailed control of colour and font etc. are also available via the Windows print API, if not on all printers, On that subject, options only take effect if the selected printer supports them.

VIEW

VIEW COMMAND

Z

Loads post-sort deferred intex updates for a file, as per the Model 204 command. The file whose indexes are to be loaded must be open, and should have been opened with a "basic" OPEN command, that is without specifying the TAPEx DD names. See the load job in the DPT download for an example.

The index load process performed by this command works by collecting deferred updates for each field/value into a bitmap in memory before storing them in a single trip to the index on disk. This contrasts with early DPT versions where repeated index retrievals and bit-flippings were performed. With most types of field this in-memory method has a dramatic effect on load times. It can be thought of as performing a small number of User Language "FILE RECORDS UNDER" statements rather than a large number of "ADD" statements, although that is not quite accurate as FILE RECORDS would replace any existing set, whereas Z augments it.

The improvement is only effective if the input file is well sorted, and DPT provides a command option which can help ensure that is the case.

See also the DBA guide for more about loading files,


DPT Custom Commands

Many of these commands have been left in the system from when they were used during development and testing, like the custom options on actual M204 commands discussed above. They have been left in in case somebody finds them useful. The discussion (given below the table) is sketchy in some cases - further details are available if anything sounds interesting and you'd like to know more.

Most are prefixed with equals signs, to prevent conflict with existing APSY names ('=' can not be used in APSY names). It has been suggested that asterisk would have been a better choice and maybe that's true. In any case if M204 compatibility is an issue most of them can be disabled using the CSTFLAGS parameter.

List of custom commands (usage notes follow for many)

CommandDescription
=BUFFDUMP [from-page [to-page]] Shows assorted low level info about file pages currently in the buffer pool
=BUFFTIDY Shorten the buffer pool LRU queue
=CHKDUMP A listing of records currently in the checkpoint file
CREATE SUBSYS {name} Required because no SUBSYSMGMT on DPT
DELETE SUBSYS {name} See CREATE SUBSYS
=DELG [pattern] See =SETG
D[ISPLAY] AUDIT Scan the audit trail
D[ISPLAY] SUBSYS [(option)] pattern Option is BRIEF or VERBOSE. Output is smart-edit-enabled. See also CREATE SUBSYS
=ECB [option [new value]] Maintain "ECB" information
=ECHO [any-text] Prints to the current output device. Default is USE=RESPECT.
=EMAIL Sends an email.
=FUNCLIST Lists the current set of installed $functions, with their return value and parameter types (see also $FUNCINFO)
=ILINFO Scans table D to analyze inverted list (e.g. bitmap) population levels.
=IOSTATS IODev7 message counts (the approximate reverse of the figs shown on the client)
=LISTALC List allocated files, like the TSO command
=LOAD Invoke a fast load.
=MIME options Maintain web server related reference data.
MONITOR RESOURCE Show information about internal resource locks
=NEWSL activity Similar to the $SLSTATS function
=PAI [BLOB-option] [n|ALL] A pre-written PAI loop
=PAV field A configurable pre-written FRV loop
=PSTxxxx Pseudo-user commands.
=POST Wrapper for Windows PostMessage API.
=RESxxx Manipulate internal resources and locks
=SEQREAD dd [n|ALL] Prints the first 'n' records in allocated sequential file 'dd'
=SETG gname [=] value Allows global manipulation from command level. Name and value may be quoted. See also =DELG
=SHELL Start an OS process independent of DPT
=SOCKET Start and stop the socket interface
=SPAWN Start a daemon thread
=SPAWNINFO Show info about a daemon thread
=SPAWNPRINT Show daemon thread output
SPY Start and stop the UL debugger
=UNHALT Similar to M204 EOJ
=UNLOAD Invoke a fast unload.
=VSTAT View statistic values
=Z1 Force flush of 1-step deferred update info.

Notes about some custom commands

=BUFFTIDY

This command manually invokes the buffer tidy-up processing normally controlled by the custom parameter, BUFAGE. Most commonly it would be issued periodically by a system daemon (see =PSTTIDY).

CREATE SUBSYS

Goes along with DELETE and DISPLAY. This command is provided to avoid the necessity of recoding SUBSYSMGMT, and also because there is no equivalent of CCASYS where subsystem definitions would be stored. An online would therefore have a number of CREATE SUBSYS commands in the User 0 input stream, much like people often do with file allocations, printer definitons etc., and which would recreate the subsystems anew as the online is started each time. The System Config Guide has more details about subsystem handling.

DISPLAY AUDIT

Issuing this command causes DPT to scan through the current audit file, printing lines to the user's terminal (or USE destination). All lines can be shown, or more often lines would be excluded by giving one or more selection criteria.

This command is particularly useful when DPT is running in NOGUI mode, and/or the user is working on a remote machine from the host. In the first case the simple alternative is notepad, and in the second case there may be no simple alternative (short of writing your own audit file scan program using e.g. $FREAD and $TOKENIZE).

Examples:
D AUDIT                                         /? Show first 1000 lines             ?/
D AUDIT MAXLINES -1                             /? Show whole file                   ?/
D AUDIT FROM -300                               /? The last 5 minutes                ?/
D AUDIT LIKE ' 05 '                             /? Activity for user 5               ?/
D AUDIT LIKE ERR FROM 9 TO 10:25                /? Errors today from 9am to 10:25am  ?/
D AUDIT FROM 2010/09/10/18:37:12                /? Since a very specific time        ?/

=ECB

This is just a command line way of maintaining the same information as the thread synchronization $functions. There are three items of information associated with each ECB, viz a flag (denotes posted/unposted), a numeric value (the post code) and a textual field (can be used for any purpose).

=EMAIL

Hooks into the same underlying send mechanism as the User Language $EMAIL function and the USE $EMAIL command. See notes there for more details and background.

Consider using this command for short ad-hoc messages entered by hand, like where you might use the MSG command on Model 204.

=LISTALC

Like the TSO command. Shows all the OS files that are allocated to the system, of all types or a selected type.

=LOAD

This command efficiently loads field descriptions and/or data and/or index information into a DPT file, so long as that information is presented in one of the acceptable formats. Typically that means that it was either generated during an earlier unload of the same file during a reorg, or from a Model 204 file on the mainframe.

Brief load processing overview:

Stage 1. Preparation

Stage 2. Loading field information:

Stage 3. Loading record data:

Stage 4. Loading indexes:

Stage 5. Completion:

Examples:

=LOAD
IN SALES =LOAD ALL INFORMATION
=LOAD (PREVIEW)
=LOAD (DIR = "Fastload Inputs\Sales\June 2010", CLEANAFTER)

=MIME

This command operates on the internal cross-reference table of file extensions and MIME content types. MIME is the standard scheme for describing the type of data blocks transferred over the internet in various situations. In theory the MIME types are unrelated to file extensions, but often they are simply more elaborate forms of the file extensions. For example in the case of ".gif" the MIME standard type is "image/gif". Likewise ".html" = "text/html" etc. The logic here, assuming there is any, is beyond the scope of this discussion.

DPT maintains a lookup table keyed by extension, with each entry corresponding to a MIME type. This both enables the system to serve fully-described static resources like images and applets, as well as allowing the User Language programmer to use the extension as shorthand in code when specifying a MIME type.

When the system starts, the table is empty, so DPT would be leaving everything to the browser. However, you may find it useful to set up some standard types, if only to be able to use the UL shorthand when creating dynamic content that isn't HTML (the default). The web server daemon initialization procedure is a good place to place such commands (as on the demo system).

MONITOR RESOURCE

Displays a list of the resource control objects and details of who is currently holding them and waiting for them. See also =RESxxx. Users are identified by their OS thread ID. Consult the main MONITOR display to cross-reference this to a user name or M204-style user number if required.

=PAI

Quicker than typing B;FR;PAI;END all the time, and provides more information. This command can also be issued from the DPT client navigator.

=PAV

This is a partner command to =PAI. Both are provided just to save typing the same old 3 line requests over and over again. Both can be issued after right clicking on the navigator.

=PSTCHECK or =PSTTIDY

These two commands are what the checkpointing and buffer tidying pseudo-users (daemon threads) issue when they log on. An infinite loop is set up whereby the issuing thread periodically wakes up to perform some work. =PSTCHECK wakes up every CPTIME seconds and issues a CHECKPOINT command. =PSTTIDY wakes up every BUFAGE seconds and issues a =BUFFTIDY command.

Normal users can issue these PSTxxx commands if they really think there's a good reason to do so. Normally there would not be any good reason.

=PSTFUN

This command turns the calling thread into a $function-support daemon, most likely for handling one of the emulated emulated Sirius $functions but potentially also any other DPT custom $functions requiring background processing. Normally a daemon thread would issue this command, but terminal users can do so as well if they like.

=PSTWEB

This command turns the calling thread into a web server daemon. Normally a daemon thread would issue this command, but terminal users might often do so for debugging purposes.

=POST

A straightforward wrapper for the Windows API PostMessage() function. This command can be used for GUI manipulation, or any other general application communication purposes. PostMessage() is an asynchronous API, so each call is guaranteed to complete. In other words unlike SendMessage() which is synchronous and might hang. There is no DPT wrapper command for SendMessage().

=RESxxx

This is a set of commands for manipulating the resource control objects that DPT uses in various situations where something is shared between users. They can be displayed using MONITOR RESOURCE (see above), and manipulated using =RESDEF (define), =RESDEL (delete), =RESFIX (kill all locks), =RESGET (acquire with wait), =RESINFO (display), =RESTRY (make 1 attempt to acquire), and =RESREL (release). Messing with the system-defined resources too much is probably not advisable, but you may find use for some of these, perhaps with custom $functions. The full syntax is omitted here to save bandwidth, but is available if anybody wants it.

=SHELL

This command kicks off an OS process completely independent of the DPT host system. The new process can of course subsequently connect to or otherwise interact with DPT or its client applications. =SHELL is always asynchronous, in other words it does not wait for the new process to complete.

See also =SPAWN.

=SOCKET

This command starts and stops the host application sub-threads which listen for incoming socket-based connections. There are two such "listener" threads working on two different TCP/IP port numbers, one accepting terminal connections (IODevs 7 and 15) and one accepting HTTP (web) requests. The main reason for using this command during a run would be to change the port numbers without having to bump everyone off.

If the listener threads crash it may also be possible to use the SOCKET command to restart them without bouncing the whole system.

=SPAWN

This kicks of an IODev99 daemon thread to perform background processing within the DPT host system. Except for the parameter order it is processed the same as the $SPAWN custom UL function. Each call to =SPAWN starts a new daemon user which runs, perhaps briefly, until its input is processed.

Daemon threads are operationally very similar to IODev3 threads, and allow all the same commands, parameter settings etc.

Note the difference from =SHELL which kicks off an OS process outside the host, and also the difference from UL $functions which make brief use of existing support daemon threads rather than spawning fresh ones.

=SPAWNINFO

Shows summary information about a daemon thread, such as the input line, output file name, and details of the most serious error message issued by the daemon.

=SPAWNPRINT

Reads and prints the contents of a daemon thread's output file.

SPY

This command affects the run-time control users have over User Language programs. NB. No equals sign because presumably people are familiar with SPY to invoke SoftSpy.

Generally you would not need to issue this command by hand, as Ctrl+F1 in the GUI toggles the debugger on and off. The following notes however may be useful to explain the meaning of some of the GUI debugger options.

=UNHALT

This command is a cross between Model 204's EOJ, and the concept of replying to the HALT message on the MVS operator's console. It tells user 0 to wake up and proceed with its processing. This will often mean the system comes down, unless there is a significant number of extra commands after the HALT, or even another HALT. The System Config Guide gives full details on starting and stopping the system.

=UNLOAD

Efficiently extracts all or part of the information in a file. The results might be the halfway stage of a reorg, or be used used for any other purpose such as loading into another DPT file or passing to an external system. By default the command processes all records in the file and/or index values for all records, but if you want to specify a subset you can use the partner $UNLOAD User Language function, or the GUI interface in the File Wizard utility.

The output of the processing is information in the standard layouts as used by various internal operations as well as user-invoked functions like REORGANIZE and =LOAD. If extract data is required in other significantly different formats, apart from the simple options shown below, you will have to either fall back on a UL/image program, or start with =UNLOAD and then reformat the output using external tools of your choice. Put another way, DPT has no equivalent of the complex Sirius 'FUEL'.

The ordering of information extracted is as it is in the database: data records in ascending record number order, and index values in whichever collating order the field is defined as (string or numeric). There is currently no way to get the =UNLOAD command to change these extract orderings.

Examples:
=UNLOAD
IN SALES =UNLOAD ALL INFORMATION
=UNLOAD FDEFS (FNAMES DIR='C:\Mystats')
=UNLOAD INDEXES FOR FIELDS SALES_Q1, SALES_Q2
=UNLOAD (REPLACE, CRLF) ALL
=UNLOAD FDEFS AND DATA (EBCDIC FNAMES) EXCLUDING AGENT.COMMISSION

=VSTAT

Allows individual stat values to be examined without issuing a MONITOR command. However, it differs in processing from MONITOR in a few other ways. Firstly, zero-value statistics are shown. Secondly, file level statistics can be shown. Thirdly, statistics for other users can *not* be shown. The UL $STAT and $VIEW functions hook into the same low-level functionality as =VSTAT.

=Z1

This command is somewhat related to the regular Z command, but it applies to the single step deferred update process. The function of the command is to trigger the same processing that usually happens when DPT detects a low-memory condition, namely to offload deferred update information to sequential files and free up the memory they were using. Table D is not written - that only happens when the last user closes the file, regardless of any =Z1 commands issued beforehand.

The main time this command would be useful is if one file is in DU mode and hogging a lot of memory with its deferred updates but neither getting closed nor receiving any more updates to trigger the offload. To other actively-updating files in DU mode it would then appear as if available memory was low, and they would keep flushing far too soon and spoiling some of the benefit of their DU mode. If it's not possible to get all users to close the first file, issue the =Z1 command on it.



Part 2: User Language


General UL Compatibility and Processing Notes

Where possible every effort has been made to get User Language working the same as Model 204. Otherwise (usually only obscure situations) an educated guess has been made with the intention of correcting it as soon as possible later on. Usually the decicision taken was to err on the side of leniency if the alternative was possibly making the compiler too strict, since this offers the best chance for existing code to run successfully.

In some cases there are small intentional differences for the various reasons mentioned in the emulation strategy document. More details on some of these, plus other issues of general interest, are given below.

Model 204 statements that aren't supported are discussed in the compile-time messaging section below.

Version compatibility and obsolete M204 features

Statement numbers are not allowed, regardless of the FOPT parameter setting.

Unbalanced brackets may not be used to continue lines. Since there is no INCCC parameter, continuation can not be implied with a non-blank character in this column. Hyphens, unfinished quoted strings and unfinished block comments are the only way to continue lines. Note that the FIND expression

FD
    (AGE > 18
     SEX = M) OR -
     RANK = CORPORAL
END FIND
is not using the unbalanced brackets to continue a line. The newline is real, and means an implied AND, which because of the brackets will be given higher priority than the explicit OR, rather than the usual lower priority.

The "pre V2.1" issues mentioned in the UL manual appendix are taken into account, so that the "previously-allowed" statement forms are not allowed on DPT.

Number handling

Floating point
This is a complex subject, but one thing we can say is that completely emulating Model 204's (and by implication MVS's) handling of floating point numbers is outside the scope of what we are trying to do. DPT's floating point handling is based on the IEEE 754 standard functionality commonly supported by compilers and co-processor chips. This standard is available on S/390 nowadays but M204 (being an old application) almost certainly uses the earlier "hexadecimal floating point" (HFP) format. This means the underlying notions of accuracy and precision are going to be subtly different between the two platforms.

The question is, how far should we go in making things work like they do on M204? The happy answer is that in many cases things should work the same because of the UL convention of *rounding* to 15 significant decimal digits for many operations. DPT follows this convention, as described in some detail in the UL manual, and since both IEEE 754 and long HFP are guaranteed to convert reversibly at this precision, any differences get glossed over when these roundings occur.

Rounding occurs in nearly all situations, including but not limited to: Storing in FLOAT variables, in the PRINT statement for any numeric value, storing in both table B and table D for FLOAT/ORD NUM fields, intermediate expression results after addition/subtraction and all situations where strings are used as numbers (e.g. in the compiler and when STRING variables are used in arithmetic). Rounding is skipped in very few places, namely after multiplication/division and before/after $function calls, since in these cases the extra precision might often be useful. This should be close to how Model 204 behaves, but may not be exact (interestingly the UL manuals themselves aren't completely consistent on what they say). Further refinements can be made in future releases if required.

There may be subtle differences in what is considered a valid numeric literal when exponents are used (e.g. 1.5E3 / 1.5 E 3 / 1.5E+3 etc).

This topic is also relevant to the math $functions, where the potentially different expansions used by Microsoft C and the mainframe libraries could conceivably add further rounding differences in the small digits. It's also relevant to the bit string generated by $FLOAT.

Fixed point arithmetic in UL
When doing arithmetic with FIXED and STRING variables (and image items) that have DP specifications, DPT should behave the same as M204, as per the rules set out in the UL manual 10-23...25, since in this case the behaviour is based on truncation at the point specified by the DP specification, which maxes out at 9. DPT obeys all the rules concerning scale of intermediate and final results, except one. It does not look ahead to see if the next operation is a subtraction (which in any case as the manual says ends up giving an incorrect result).

Pattern matching

DPT's pattern matcher handles all the things the Model 204 one does, including the M204 Version 5 relaxation on the rules about whether you need the escape character before special characters (it is optional). In certain Sirius functions the slightly different Sirius format is recognized too.

However, without knowing how the M204 matcher works it's likely that highly complex patterns will work slower on DPT, but they might be faster, you never know! In particular the common cases of patterns with a leading and/or trailing wildcard and no other specifications such as '*ABC', 'ABC*' or '*ABC*' all work very quickly, being internally translated into the equivalent of $SCAN.

Context specifications

IN clauses on DPT can not include the "AT location" phrase, since there are no scattered groups.

Unlike M204, global foundsets and positions can be created in ad-hoc group context on DPT, but they are deleted at end-of-request, and so do not realistically give rise to a situation where an application will work on DPT but not on Model 204. Ultimately the same result is achieved and it was much easier to implement this way internally.

The following are not allowed in FIND and FR WHERE statements, because the corresponding features are not themselves supported.

Find Processing

The DPT file system applies a certain amount of intelligence to the evaluation of multi-criterion searches. For example powerful criteria like FIND$ and FILE$ are applied first, then indexed equality criteria are performed before indexed range criteria, and the table B search, if required, is done last of all. Of course there is always scope for more ingenious techniques, and only the CCA tech-heads will know exactly what M204 does. DPT does the obvious well-known M204-like things such as that mentioned above, and applies some tricks of its own too. What all this means is that DPT will not behave *exactly* the same as M204 in terms of logical IO counts, BXNEXT etc., (given the different nature of DPT file structures they could never match exactly). However, it still should be a reasonable assumption that if you try two different ways of coding a FIND, the stats from DPT will show the same one is best as if you tried it on Model 204.

It's also possible that DPT does things in a slightly different way when evaluating finds in group context. DPT basically just performs the same find once for each member, which means for example that you could get more than one record locking conflict or MBSCAN-exceeded message during the same find. This is something that will be easy to correct if required in future releases.

List Processing

If DPT runs out of memory when placing records on a global list, the list is cleared and will be found empty by the next request that uses it. Lists are modified in place for performance reasons. Alternatives which would allow a list to reliably retain its pre-modification state in the event of memory problems were deemed too much of a computational and/or memory overhead. This issue is only relevant to global lists because an out-of-memory condition (see elsewhere) always cancels the current request.

Subroutines

CALL statements do not implicitly declare subroutines like they do on M204. If a CALL statement is encountered before the corresponding subroutine is declared, compilation of the CALL is deferred until the end of the request, much in the same way as with JUMP statements. This means that messages are not always issued in the same order as they would be on M204, although it should not affect the situations in which a request will or will not compile successfully.

In some cases it's possible there may be a difference in the handling of file context where list parameters are allowed to be implicitly defined (the M204 manuals don't make it clear what's supposed to happen). In any case if contexts don't match there will be a compile-time error and this will not cause subtle misbehaviour.

On Units

On units are the work of the devil (when he was breifly on the User Language team at at CCA in the 70s). Nothing wrong with the idea in principle, but as implemented in User Language they allow you to write code with highly obscure, even unpredictable error handling behaviour. The major cause of problems is the way ON units from one scope remain in effect during subroutine calls to another. Then consider that in some cases it is allowed to jump out of ON units into the code surrounding them, but that this may not be the same scope as where the condition occurred. This gives rise to a whole page of work-around rules the compiler has to enforce (check out the Model 204 UL manual!) Even then it's not possible to cover every case at compile time, and extra checks are required at run time to insure against inadvertent return from subroutines. All very messy, and hard to guarantee DPT will work exactly the same way.

Secondly, consider that ON units can invoke and also install/deinstall themselves and other ON units, both in their own scope and the scope where the condition was raised. This means it's not always easy to see which ON units are in effect at any given time.

Combine the above points with the fact that activation and de-activation of ON units across calls to subroutines (and across "calls" to ON units) is not completely consistent in practice on M204, and things can get really confusing. DPT should work the same as M204 in all simple cases and most if not all complex cases.

When the BYPASS statement is used after ATTENTION in the test element of an IF or loop construct, M204's behaviour seems a little strange (try it!) DPT is more sensible and bypasses the entire loop or IF block.

If an ON unit is defined inside a FR loop, and later invoked from a point outside that loop, field references in the ON unit give a "no current record" error on DPT. This may be slightly different behaviour to M204, which if memory serves shows the value from the last record processed by the loop. Hopefully an obscure situation though.

Callable $functions

The following three statements can all be used on DPT:
%X = $SETG('A', 'B')
CALL $SETG('A', 'B')
$SETG('A', 'B')
As Sirius discovered a long time ago you can implement new UL features as statements or as $functions, with a variety of pros and cons in each case. Generally speaking $functions are preferable when there are parameters, because of the more structured syntax (brackets, commas etc.), but calling them has historically been clunky because you have to do something with the return code even if it's of no interest. This means if more and more new parts of the language were implemented via $functions, code was going to look bad, and explains the new trend for callable Sirius $functions. The DPT approach is that all $functions are now callable if you're happy to discard the return code. What's more you don't even have to say 'CALL' - a stand-alone $function name is a valid statement, and implies CALL.

This feature can be disabled for stricter M204 compliance using the CSTFLAGS parameter. See also some related notes about syntax-colouring of $function names in the DPT client front end.

Variables

Variables cannot be redefined on DPT - the second definition is rejected by the compiler. This applies regardless of whether the request contains complex subroutines, and also applies to continued requests - the variable from the original request can not even be defined with the same attributes in the continuation. To run existing code where this is done, use MSGCTL to set the "duplicate variable definition ignored" message to CLASS=I.

COMMON variables are an exception to this of course. You have the option of fully defining COMMON variables in each scope they are used, as was required pre-M204-V5, in which case each part of the definition must match the original, or simply declaring the variable name as COMMON, in which case the other characteristics get picked up from the original definition if there was one. The "only one INITIAL clause" error can also be turned on or off as desired.

LVARTBL is registered once per scalar variable, once for every array element, and once for any subroutine parameter of either type (since dimensions are not given on array parameters). This is all regardless of data type (STRING/FLOAT/FIXED).

DPT supports arbitrary expressions as INITIAL values (that is, the Model 204 V5 feature). The expressions are evaluated at the start of running a request, before anything else happens. By default the debugger doesn't stop on the declarations while this happens, but you can make it with an option.

Images

Images as working storage
When using image items in place of %variables, the following general points appliy on DPT.

The dummy sign character used in UNSIGNED PACKED numbers is 'F' for visual familiarity to mainframe users (incidentally $PACK and $UNPACK are handled the same way). However, with an UNSIGNED ZONED item, the character is '3', which is to say no dummy character. This is more natural in an ASCII environment where the decimal digit "zone" of the character set is X'30' to X'39' instead of X'F0' to X'F9'.

Structure of images
Like ON units, User Language images offer the chance to create infinitely complicated structures, using for example large numbers of staggered redefines in conjunction with UNKNOWN or DEPENDING arrays. Also like ON units, the degree of confidence to place in the accuracy of DPT's emulation of Model 204 depends on the complexity of the situation. DPT should work fine in most realistic scenarios.

DEPENDING and UNKNOWN arrays are two ways Model 204 offers to provide mapping of variable structures using what is essentially a fixed structure, namely an image. With an UNKNOWN array certain item position checks are deferred till run time: a classical approach which simply puts some limits on what can be checked at compile time. A DEPENDING array on the other hand corrupts the clean "mapping" function of an image by inserting/removing fillers into/from external records during READ/WRITE, so that for example byte 25 of an incoming record might end up at byte 35 in the image. In this sense it is an ugly solution to the problem, and poses some tricky questions to the compiler. There are a number of restrictions M204 puts on the positioning of other items in an image which contains one or more DEPENDING arrays, presumably kluges intended to minimize the effect of the filler insertion/removal on those other items. Not all these restrictions are implemented on DPT (I don't even know what they all are, and in any case some would probably be unnecessary given the different underlying implementation). So, while existing code written on M204 should work, there is a small window of opportunity to write incompatible new code.

Screens

Screens as working storage
When using screen items in place of %variables, the following general points appliy on DPT. Screen item variables
The following general points apply on DPT.

Display considerations
The ways in which the view on DPT Client differs from a 3270 are now discussed in the IDE User's Guide.

Screen processing on non-IODev 7 devices
Full screen related statements compile on any thread type, but READ and REREAD fail at run time unless the user is running on an IODev 7 (i.e. there is no support for the clunky line-at-a-time screen read processing you can do on M204). PRINT SCREEN on a non-IODev 7 device generates a formatted text-only version of what the screen would look like, minus colours, underline, blink etc. You can also force this to come out on an IODev 7.

Miscellaneous screen issues
The MUST FILL autovalidation option is treated similarly to REQUIRED (i.e. implicit AND, not OR). This disagrees with the manual but seems more logical.

Redefining Labels

Using the same label name several times at different points in a request is very bad progamming style, but it is a style enjoyed by some. After all, which version of the label is the "real" label? It's never really seemed clear what M204's official behaviour is, and it's certainly not explicitly laid out in the manuals.

DPT does have internal support for redefined labels, but it is currently disabled because its behaviour may be sufficiently different from M204 to be confusing. The following notes are left in here for interest, and to make it clear what a potentially confusing area it is. The ability to redefine labels on DPT may be reinstated if and when it is possible to confirm exactly how M204 handles things.

-----------Start of provisional notes about the disabled feature

When the current request just contains just one occurrence of a label, but there is another identical label in the global table or remaining from an earlier portion of a continued request, there is hopefully no confusion over what should happen, either with jumps to the label or with references by other statements.

However, when a label is redefined in the same request (or more specifically the same request continuation portion) it is allowed so long as the statement type (FD, SORT, COUNT etc.) is the same in all cases, and the file context is the same too where appropriate. This means other statements referring to the label can be confident of the type of "contents". The contents of the label as extracted by other statements reflect the contents as set by the most recent statement executed at run time (loops set the VALUE IN or OCC IN at the top of the loop). Jumps to redefined labels on the other hand are resolved at compile time. If the label has already been defined (backward jumps) the jump is to the most recently-encountered version. Otherwise (forward jumps) the jump has to be resolved in a final pass of the compiler, and is therefore always to the last version of the label encountered in the code. To illustrate these points:

B
A: JUMP TO V                       //invalid jump (final V is nested)

V: FRV SURNAME
     PRINT VALUE IN V              //current value of outer loop

V:   FRV CITY                      //redefinition OK - FRV
       PRINT VALUE IN V            //current value of inner loop
     END FOR

     PRINT VALUE IN V              //final value of inner loop
  END FOR

  JUMP TO A                        //back to the top

A:                                 //redefinition OK - "vanilla" statement
  PRINT VALUE IN V                 //final value of final run of inner loop
  JUMP TO A                        //previous statement
END

-----------End of provisional notes

Compile-time messaging

One of the key objectives DPT was written to achieve was to provide much more accurate and helpful compiler messages, so the first thing to point out here is that in most cases the message text doesn't match that on Model 204. Having said that, in some cases the M204 text is so familiar that it is used but with extra information added. In many cases the message is somewhat longer - a example typical of the DPT philosophy here is:

M204:
FIELD NAME REFERENCE NOT WITHIN A FOR LOOP

DPT:
Field name SOMESTRING does not exist in the file context active at this point (TEMP GROUP MYDATA). If you meant a string literal, put quotes round it.

In any case, broadly speaking there are three things that can happen if a statement doesn't compile.

Firstly, as with Model 204, there are a handful of situations where the compiler will issue a message but otherwise compile the statement in question successfully. They can be thought of as helpful advice from the compiler, although you can upgrade them to errors - see below. Most examples of this are listed here.

Secondly there is the most common situation where a statement is invalid. The compiler issues an error message, ignores the line in question, and then carries on compiling. Typically the request as a whole will fail compilation with these because by default the messages are CLASS=E. In other words they are counting errors, which means the error count will be non-zero at the end of compilation, which in turn means the evaluator will not kick in. Compilation may also be aborted prematurely if the error count reaches ERMX. If these messages are made CLASS=I, the request will be evaluated as if the source code line was not present (it will be highlighted in the debugger source code view as having failed compilation). As on Model 204, this is not usually advisable, and may cause unpredictable behaviour.

Finally there are errors which for various reasons mean the compilation cannot continue at all. In such cases the error message is followed by "Compilation aborted" and nothing is run. This category covers things like memory exhaustion and (hopefully rare) internal bugs.

DPT by default should issue messages in roughly the same situations as Model 204. The handling of individual messages in the first two categories above can be altered using the MSGCTL command, for example to force a failed compile by changing an "informational" message to TERM CLASS=E.

Messaging for unsupported M204 statements
In most cases where a Model 204 statement is not emulated, the compiler by default issues an informational message and compiles a "stub" operation into the executable program, so that during execution another informational message gets issued if that operation is hit. The intention of this is to allow existing User Language programs to be debugged and tested. If control doesn't go near the unsupported statement at run time, there is no problem. Even if control does pass through a stub operation, you might use the debugger to break there and tweak variables and image items etc. to pretend that the unsupported statement had been executed. You can upgrade either the CT or RT message to CLASS=E, to force an appropriate error response if desired.

Run-time messaging

Informational messages are issued at run time in the same situations as M204 (divide by zero, variable too small for result, etc.).

A whole family of error messages, namely those issued from inside $functions (e.g. negative parameter to $SQRT to take an obvious example), deserve special mention. On DPT most of these $function parameter-validation messages just use the same message code with differing message text, unlike on M204 where there is a range of separate codes around M204.0745 onwards. Clearly this reduces the precision with which MSGCTL can be applied, and if this is something that causes problems for people, separate message codes can easily be reintroduced.

Additionally, DPT addresses its development and unit-testing mission by issuing messages in a few additional situations where it might highlight a potential UL coding error. A good example of this is when a string %variable is truncated on assignment - a M204 feature that can be overused and sometimes leads to nasty bugs. Similarly when assigning invalid numeric literals to FIXED and FLOAT variables. These considerations also apply to intermediate results when, for example, concatenated strings are truncated or numeric overflow occurs.

In all cases run time messages can be upgraded or downgrded to CLASS=E/I or suppressed using MSGCTL. CLASS=E messages invoke the usual chain of events for a run time error (ON ERROR unit if possible, otherwise request cancellation; then APSY error proc with error global = BUG).

As with compile-time messaging, the default should be for messages to cause errors and be printed on the terminal in roughly the same cases as on M204. All run time messages are by default printed to the audit trail at least.

See also the above comment about messaging for unsupported M204 statements.

Reserved words (and characters)

This section only concerns field names and values, not procedure names. It is an area where the Model 204 manuals are, at the time of writing this, incomplete and/or inaccurate and/or unclear. DPT aims to allow similar usage as on Model 204, but overall is probably somewhat more forgiving, which is to say it requires quotes in a smaller range of situations. The following notes describe DPT's behaviour.

In general, the approach to the treatment of reserved words is to assume the programmer knows what they're doing, and where unquoted reserved words are used to assume the reserved meaning. For example the following command does not contain any ambiguity and requires no quotes.

DEFINE FIELD GIN AND TONIC
But this UL statement would be assumed to refer to two separate fields:
PRINT GIN AND TONIC
So the keyword would have to be "hidden", as in these equivalent alternatives:
PRINT G'IN AND TONIC'
PRINT GIN' AND 'TONIC

The set of reserved words and characters that can cause problems depends on the circumstances. For example the following statement will not work since the keyword WITH introduces the field attributes, (CHEESE would get flagged as an invalid attribute).

DEFINE FIELD BURGER WITH CHEESE
So this time we need quotes to define the field, for example:
DEFINE FIELD 'BURGER WITH CHEESE'
With this field the keyword WITH has a special meaning in UL, and usage of the field might or might not require quotes depending on the situation. For example WITH has no meaning to FIND or ADD, so these are OK:
FD BURGER WITH CHEESE = 3.99
ADD BURGER WITH CHEESE = 3.99
But in expressions WITH means concatenation, so quotes are required, for example:
%X = B'URGER WITH CHEESE'
PRINT BURGER 'W'ITH CHEESE

When defining fields the following points also apply, not for the benefit of the DEFINE command parser, but as sanity checks to help reduce later ambiguous situations in User Language.

When using quotes in field names they can go anywhere as long as they hide or break up the reserved component. In the specific cases of use in arithmetic and print expressions, the quotes cannot enclose the first character, since then it looks like a quoted string. In other cases such as commands like the one above and record-access statements like ADD and CHANGE, the entire field name can be quoted. A commonly-used convention on M204, which works in all situations regardless of whether there are any reserved words, is to leave the first character of a field name unquoted and quote the rest.

Similar considerations apply with field values in statements such as ADD, CHANGE and FIND, which do not usually need quotes, and as with field names the set of keywords causing problems will be different in each situation. For example here is a common situation where the keyword OR should obviously take its operator meaning:

FD NAME = SMITH OR JONES
But we can change the meaning with quotes, as in these two equivalents which search for a single database value rather than two:
FD NAME = 'SMITH OR JONES'
FD NAME = SMITH' OR 'JONES
On the other hand in something like an ADD statement the value is assumed to be the entire line after a certain point, and no quotes are required because there are no further possible keywords or terminator characters. So these two are equivalent:
ADD NAME = RANDALL AND HOPKIRK (DECEASED)
ADD NAME = 'RANDALL AND HOPKIRK (DECEASED)'

ASCII/EBCDIC

During normal operation, string comparisons like in IF tests and database range finds are based on the ASCII collating sequence. This may be a subtle source of different behaviour from the mainframe (e.g. A<a in ASCII but A>a in EBCDIC). It is also relevant to various other situations, such as the $C2X and $X2C functions, PACKED/ZONED image items etc.

From DPT version 2.12, the $ASCII and $EBCDIC functions provide the capability in User Language to convert between character sets if data is being exchanged with a mainframe, and it isn't otherwise converted during transit.

Case handling in User Language

Traditionally UL has been an all-uppercase language, at least as far as language keywords are concerned (user-named components like variables could be in mixedxcase). From DPT version 2.12, this restriction has been eased by emulating the Sirius conventions, as detailed more fully in the section on Sirius features.

Top


M204 Statements - Processing and Syntax Differences

Model 204 statements that do not feature in this section can be assumed to work pretty much the same on DPT. As a rule there are far fewer deviations and "embellishments" with the syntax and operation of UL statements than there are with commands.

As with commands, custom options can be disabled using the CSTFLAGS parameter.

FOR RECORD NUMBER

PRINT

The print statement does all that the M204 equivalent does. In addition, you can print the term *RECINFO, which is treated exactly the same as *RECORD but simply shows more detailed information about the current record.

PRINT ALL INFORMATION [INTO]

The
BLOB-related extensions are supported, but the output produced varies slightly from Model 204, reflecting the underlying storage differences. For example the PAI INTO result-array-3 need only be STRING LEN 10. The default value is LOB_DATA, not LOB_FLOD, thus making BLOB fields work the same as STRING fields by default.

PRINT SCREEN

On an IODev7 the PRINT SCREEN statement works much like READ SCREEN, which is to say it renders the screen in a coloured fullscreen display. On other IODevs, PRINT SCREEN is also allowed, but the screen is displayed simply as text lines, much like a series of PRINT statements. You can force this behaviour on an IODev7 using the DPT custom variation PRINT SCREENLINES.

SORT VALUES

Is this statement supposed to do an in-place sort on the value set, or create a fresh sorted copy in another label in the same way as SORT RECORDS? Memory fails, so it's currently available both ways, as follows. When the truth comes to light these two may have to be switched round (note the 'B' on 'VALUESB'):
V2: SORT VALUES IN V        * V is unchanged, V2 contains fresh sorted set

SORT VALUESB IN V           * Sorts set V in place - doesn't need a label


M204 $function Differences

General notes

The following notes describe non-trivial differences in behaviour of Model 204 $functions on DPT. Most of these differences reflect differences in file handling, system config issues, etc. A catalog of supported $functions is given in the features cross-reference document.

A few of the $functions have been embellished with custom options. As with all custom options these can be disabled by resetting the CSTFLAGS parameter.

In one or two cases (e.g. $EDIT), the $function is listed here as an admission that it may not work right in 100% of cases.

In many cases Model 204 handles errors in $functions by setting a default return code (often zero or null string) and then continuing. In these situations DPT often issues an informational message before continuing. The same message code is used for all $functions so it is easy to suppress it with MSGCTL if you don't like it. In other cases it's possible that Model 204 may issue an informational message and DPT does not. If any of these are a problem for people, please say so.

Bit string data mapping functions
($BINARY, $FLOAT, $FLOATD, $UNBIN, $UNFLOAT)

When using these functions remember that the assumed byte order of the bit string reflects the architecture of the platform on which DPT is running. At the time of writing that will mostly be little-endian (e.g. an Intel PC). Most (all?) mainframes use big-endian byte ordering.

The DPT versions of $BINARY/$UNBIN and $FLOAT/$UNFLOAT interact correctly with each other, and in that sense they can be said to "work correctly". In addition, these functions use the same data formats as the BINARY and FLOAT image item types.

There are two main implications worth noting however. Firstly if you browse the data in a file generated using say $BINARY, it will not look how you expect based on mainframe experience. For example the number 255 written out as a two-byte unscaled binary value will usually read as FF00 little-endian and 00FF big-endian. The second implication is that these functions can not be used when transferring data between platforms with different endianism.

Other specific notes about these functions:

Text case processing functions
($ALPHA, $ALPHNUM, $LOWCASE, $UPCASE)

These functions accept the "language" parameter 2, but ignore it. The local language settings in effect on the OS are used.

Thread synchronization functions
($ECBDGET, $ECBDSET, $ECBTEST, $POST, $UNPOST, $WAIT)

Date functions

All internal handling of dates on DPT is based on the underlying date-handling facilities of the local operating system. What this means that there are almost certainly differences in the ranges of dates which are regarded as valid compared to Model 204. To be specific, the date functions on DPT handle dates in the range Jan 1 0100 to Dec 31 9999.

All the date functions on DPT have the (disablable) option to specify the date format as one of two special values. These formats are mainly intended for use with the web server functionality, and are denoted by the format strings 'UTC' and 'HTTP'. To work properly these must be supplied as the entire format parameter, and not mixed with other components. For example:

B
PRINT $DATECNV('DD MON YYYY', 'UTC', '01 JAN 1970')
PRINT $DATECNV('UTC', 'DD MON YY', 0)
PRINT $DATECNV('DD MON YYYY', 'UTC', '02 JAN 1970')
PRINT $DATECNV('DD MON YYYY', 'HTTP', '02 JAN 1970')
PRINT $DATECNV('DD MON YYYY', 'DD UTC YY', '02 JAN 1970')
PRINT $DATECNV('DD MON YY', 'UTC', '30 FEB 70')
END
0
01 JAN 70
86400
Fri, 02 Jan 1970 00:00:00 GMT
02 UTC 70
***
UTC format
The 'UTC' format interprets the date as an integer value which is the number of seconds since midnight on 1/1/1970. Strictly speaking this is 'Unix Time' and not the more complex and celestially-precise 'UTC' (co-ordinated universal time). However, it is a very common approximation in the C programming world and is used by DPT since it's written in C. UTC usually implies a GMT (Greenwich Mean Time, or London winter time) value - more on this below.

This simple numeric format is quite an efficient way of manipulating dates as it is often the "halfway house" in conversions between more readable formats. It is also ideal for comparisons, or doing non-whole-day DATECHG style processing such as adding 1 hour. If the date is given as a standard M204 format (i.e. no time component) the result in this format is always a multiple of 86400 since that's how many seconds there are in a standard day - the time of day is assumed to be midnight. Negative numbers represent times before 1/1/70.

HTTP format
This is the standard date+time format used in HTTP messaging, and any time you put a date or time value into an HTTP header it should be in this format. It is also a good format to use for display in an international setting such as on web pages. When converting from this format or using it as input to, say, DATEDIF, it is not case sensitive, so you can type values in UL code without worrying about case. It can also be in any of the 3 accepted HTTP 1.1 date/time styles. On the other hand all function outputs in this format are given in the standard mixedcase style, as shown above.

The time of day resolution of DPT's internal date-storage format means that for dates before 1899, not every second of every day can be represented, and 'HTTP' format dates will show the closest second they can. This should be no inconvenience in realistic applications.

GMT and local time
Unlike 'UTC', the 'HTTP' format explicitly professes to be GMT, although of course you could set it to any value you like. You should therefore take care that when requesting the time from the local machine and converting it into this format you request GMT and not local time. The standard M204 $DATE and $TIME functions return local time, so there is a DPT custom $function, $NOW, which you can use not only to get GMT, as adjusted for your local time zone and daylight savings, but also to get a combined date+time instead of the two things separately.

Character set functions

(See also General notes on character sets).

These functions make use of the character code pages definded in the CODESA2E and CODESE2A parameters.

The supplied DEMOPROC contains a short example request showing these functions in action.

Other Functions

$ALPHA
$ALPHNUM

  • These two functions have a DPT custom disablable extra parameter 3. A value of anything except zero supplied for this parameter tells the $functions to allow lowercase characters through. Leaving the parameter off gives the default M204 operation, namely only allowing uppercase.

    $BLDPROC

  • In debug mode $BLDPROC fails if the target procedure was included as part of the running request. This is because it is kept locked during execution for debugging purposes.

    $CHKPINF

  • Options 6 and 7 return zero (no extended quiesce)
  • Option 5 means only pre-images (CPMAX is always 1)
  • Thousandths of a second in the date time formats is irrelevant (and always shows as 99 or 00)

    $DSN and $DSNNUM

  • Files can not be multi-part on DPT, so $DSNNUM always returns 1
  • $DSN works as normal

    $EDIT

  • This is a complex function, for which the Model 204 manuals are not totally clear on its precise operation. The current DPT implementation seems to give pretty good results but there may well be cases where they are different from M204. Please report any you come across!
  • As of version 1.2, the "¬" (hook) character can also be supplied as "^" (hat).

    $ENCRYPT

  • Like $HSH (added in V2.0) $ENCRYPT was previously left out because it was not known what encryption algorithm Model 204 used. That is still the case, but like $HSH, some support is better that none. $ENCRYPT is supported from V2.16.
  • Note therefore that the resulting encrypted data will not be the same as if the same parameters were given to Model 204.
  • DPT allows two extra custom parameters, allowing longer strings to be encrypted (parameter 3) and longer resulting encrypted values (parameter 4). Both default to 8 which are what Model 204 uses.
  • On DPT the encryption is an SHA1 hash function, applied to a concatenation of the input string (parameter 1) and the modifier (parameter 2). The SHA1 algorithm produces 20 byte result values, so if the requested result length from $ENCRYPT is less than that (e.g. the default of 8), the SHA1 result is simply truncated. Requesting a longer result value than 20 characters just returns 20.

    $ENTER

  • Truncation.rounding etc. happens as per normal variable assignments, but no messages are issued.

    $HSH

  • This function has the same purpose as on Model 204, but the hash key generated for any given input string will *not* be the same number.
  • There are two DPT custom disablable options in addition to the string to be hashed. These let you change the hash algorithm if the default seems not to be giving a good hash table distribution when used on your data.
  • The first custom option identifies a basic algorithm from a set of around 6, for example number 1 is the Unix ELF hash. The second custom option activates small modifications to the basic algorithm, such as processing the input string in reverse order.
  • The default gives a good spread of hash keys for typical input data strings, even when they are very similar.
  • More information on this fascinating stuff is available if required.

    $ITSOPEN

  • Returns true if either a proc directory or a data file is open with the given name (see procedure handling).
  • Custom option: An optional second string parameter of ['DBFILE' | 'PROCFILE'] to target the query specifically at that entity type.

    $JOBCODE

  • This is handled somewhat differently to M204, mainly reflecting the fact that on DPT there is no difference between an "online" run and a "batch" run. It is something that will be easy to alter in the next release if needed.
  • The return code from a DPT host run is the highest message code experienced by any user during that run. As on M204, calls to $JOBCODE can increase that high water mark, but not reduce it, and a $JOBCODE call with no parameter simply reports the current value. There is also a DPT custom custom parameter, RETCODE, which is a system-wide parameter accessing the same internal variable as $JOBCODE. RETCODE can be viewed and reset, and *does* allow a change to a lower value.
  • User-specific message code high water marks are maintained for various purposes, but to clarify the slight difference in meaning from Model 204, they are accessed by a separate function, $THREADCODE.
  • Note that none of these codes currently make it back to the client in a BATCH2 style configuration.

    $LOWCASE

  • In addition to the general comment above about the language parameter, $LOWCASE accepts a DPT custom disablable extra parameter 3. A value of anything except zero supplied for this parameter tells $LOWCASE to additionally lowercase the first letter of the string. Leaving the parameter off gives the default M204 operation, namely an uppercase first letter.

    $LSTPROC

  • Less info is returned by this function than the M204 equivalent, since (see also D LIST) we have to make do with what the information kept by the operating system. The same image can be used as on Model 204 to accept the results but some fields will not be populated.
  • Security class is always zero, and user name is always null.
  • The alias option can be used in the same way, but will only ever return one procedure since there are no aliases.
  • The "cursor initiating" call takes a directory snapshot, as per the supplied pattern if one was given. Therefore new/deleted/changed procs will not be reflected, and RESET PRSUFFIX will not take effect, until the next such call (i.e. with loopvar set to 0).
  • By default $LSTPROC now returns the proc modified dates as YYDDD rather than CYYDDD, but you can get the C by enabling a fudge switch. On Model 204 this behaviour (if it exists at all) is obviously controlled some other way (e.g. CUSTOM parameter 2 setting?) so this is a temporary fix.

    $FDEF and $LSTFLD

  • As with $LSTPROC there are many items in the image that are irrelevant on DPT, and are not touched by these $functions.
  • $LSTFLD does not use return code 3.

    $RDPROC

  • LINEND returns a semicolon in all cases
  • Unlike INCLUDE, $RDPROC does not "suck up" the latest version from the client if it's open for edit

    $READ, $READINV, $READLC

  • Responses can't be continued with '-'
  • They can however be queued with ';'
  • Responses are simply truncated at 255 characters (behaviour as described in the messages manual for M204.0730 does not happen)
  • $READINV only results in an invisible input area on an IODev7

    $SLSTATS

  • The M204 manual does not say what the return value of this function is. On DPT for the time being it is a line of stat values for the previous SL period, as per T REQUEST (e.g. CPU=10 DKPR=500 ...)

    $STAT and $VIEW

  • AT LOCATION is not supported
    Custom options:
  • $STAT parm 2 can also be FILE, SYS or SL
  • Similarly in $VIEW the stat level can be FILSTATS
  • If file stats are requested the current file context is used in both cases
  • Both can view CPU and CNCT which according to the manual, M204 cannot

    $STRIP

  • Accepts two DPT custom parameters (disablable for M204 comaptibility).
  • The first indicates which characters to strip instead of, or as well as, zeros. The default is just '0', which yields the same results as the M204 version of the function.
  • This parameter can be a string of any number of characters, and any combination of these in any order is removed.
  • The second inditates where in the string to strip the specified character(s) from, as follows:
  • See also the emulated Sirius function $SUBREP later which is more generalized but could be used in similar situations.
  • Examples:
    B
    PRINT $STRIP('00123')
    PRINT $STRIP('  0123', ' 0')
    PRINT $STRIP('0.12500',, 'RIGHT')
    PRINT $STRIP(' "c:/config.sys" ', ' "', 'ENDS')
    PRINT $STRIP('&$% ABC +*/ DEF []}', '&$%+*/[]{} ', 'ALL')
    END
    123
    123
    0.125
    c:/config.sys
    ABCDEF
    

    $SUBSYS

  • If the subsystem is defined the return code is 1 (i.e. always effectively started). This is because subsystems don't need to be started or stopped on DPT.
  • Return codes 2 and 3 are not used.
  • DPT also allows an extra custom disablable second parameter, enabling you to retrieve information about the subsystem. The values this can take are:

    $VNUM

  • The SORT and SORTKEY options are not allowed


    DPT Custom Statements

    As with the custom commands these are left in here from development in case they may be of some use or interest to anybody. Again more details are available if desired, and again they can all be disabled with CSTFLAGS.

    Custom statements

    StatementDescription
    DEBUG IMAGE image-name See note below
    FD_DIAG diags-level See note below (also FR_DIAG)
    FREE [ALL PAGES | PAGE IN label] Release buffer enqueues. See LOOK PAGE note
    LOOK PAGE IN label INTO %variable See note below
    REQUEST PAGE number Must be labeled. See LOOK PAGE note.
    WEBPRINT print-expression See web coding guide.
    ZAP PAGE IN label from %variable Hack a buffered file page. See LOOK PAGE note.

    DEBUG IMAGE
    This statement produces a breakdown of the item positions within the named image. It can help you to achieve the correct alignment of items during development of a complex image. It might also be useful if you have apparent problems with compatibility of images that you know work on Model 204 but seem not to work, or work differently, on DPT.

    FD_DIAG
    Use the sequence 'FD_DIAG n' in place of 'FD' or 'FR_DIAG n WHERE' in place of 'FR WHERE', and the system will display its workings as the find takes place. This is handy for technical support if the system crashes during a database search, but might also help with tuning by the User Language programmer. The value of n is a set of bit switches. Use 255 or 'ALL' for max detail. Specific bit values can be documented if there is any interest.

    LOOK PAGE etc.
    This small suite of statements lets UL code access buffer page data similarly to the *LOOK command which is in turn similar to M204's equivalent. Typically you would have to reset STRINGMAX to get much use from these statements. There is a short example program in DEMOPROC.


    DPT Custom $functions

    All of these three groups are installed in the demo version of DPT host. All can be disabled using the CSTFLAGS parameter.

    Note also that custom $functions can be dynamically linked (ie. from a DLL) or statically linked, but for simplicity of installation all these are statically linked.

    Set A: Sample $functions

    These are some of the functions which go along with a tutorial document about how to write and install custom $functions on DPT. They therefore range from utterly simple (the first exercise) to quite complex (later exercises). They have been left installed in the distributed executable module in case anybody finds them useful. However, if and when the DPT host is distributed as a semi-built object library this group of functions would not be installed, although they could of course be recompiled and relinked if desired.

    No documentation is supplied for these functions, so you'll have to experiment with them. The editor describes what parameters are expected when you type the left bracket after a function name. Alternatively if you get in touch via the web site, more details and/or the source code are available.

    $FunctionDescription
    $DPT_ARRSORT Sort a 1D %variable (not image) array
    $DPT_CURDIR Analogous to to CURFILE
    $DPT_FUNCINFO Display linked $function info, (See also =FUNCLIST command, and sample program in DEMPROC).
    $DPT_RAND Random number between 0 and 1
    $DPT_RANDOM Allows scaling and shifting {V2: quite similar now to Sirius $RANDOM}
    $DPT_RANDOMNORMAL A normally-distributed value instead of uniform
    $DPT_RANDOMSTRING A string of random capital letters - handy for test data
    $DPT_STATUS2/DPT_STATUSD2/$DPT_SPAWN Now moved to Set B (see below)
    $DPT_C2D/$DPT_D2C Quirky character <-> decimal
    $DPT_X2D/$DPT_D2X Quirky hex <-> decimal. Sirius $D2X/$X2D usually more useful.


    Set B: Extensions to the "standard" set of $functions

    These are functions which are more important than those in Set A to the business of using DPT for realistic applications on the PC, and they are linked in along with the M204 "standard" set ($SUBSTR, $LEN etc.). So all distribution formats for DPT host will include these. At the time of writing this group contains functions which are either interfaces to local platform facilities like files, sockets and threading services, or are tangentially related to the web server functionality.

    Unlike Set A, these functions are fully documented here.

    $FunctionDescription
    $BLOB BLOB maintenance facility
    $EMAIL Send an email
    $FILE_xxx and $xxDIR Interfaces to the local file/directory system
    $LOAD $Function equivalent for =LOAD
    $LOGXXX $Function equivalents for LOGCTL, LOGLST etc.
    $LZW String compression
    $NOW Returns GMT date+time
    $SHA1 Create SHA1 digest of input string
    $SOCKxxx Interface to the socket stack
    $SPAWN Start a daemon (NB used to be $DPT_SPAWN in set A)
    $STATUS2/STATUSD2 Used by some of these $functions
    ($STRIP) Significant extensions to the standard M204 function (see earlier)
    $THREADCODE Like a user-specific version of $JOBCODE
    $TOKEN/$TOKENS/$TOKENIZE Similar to $WORD/$WORDS
    $UNLOAD $Function equivalent for =UNLOAD


    $EMAIL

    This $function is a simple interface to the "Blat" email sender utility. Depending on user feedback, extra options could be added, or it could be expanded it to work via other email clients, or it might operate as a stand-alone SMTP client itself not needing any external tools. Anyway, to use this function currently requires you to get hold of Blat - see its website. Note that Blat is not and does not try to be a perfect email client, and is open source and therefore not officially supported. However it's free, and is well-known and widely-used, and has plenty of documentation on its website and on various user groups around the internet.

    The same underlying email interface can also be invoked on DPT via the USE $EMAIL and =EMAIL commands.

    DEMOPROC in the DPT download contains examples of various ways to send an email.

    How it works summary
    Blat is a Windows command line tool which takes what you type in as the email title, content, recipients, attachments etc., assembles them all into "SMTP" (low level email format), and finally passes the package on to an SMTP mail server which distributes the email.

    All DPT does is take the parameters supplied on the $EMAIL function and makes up a command line which it passes to Blat as if entering it at the DOS prompt. Blat then works as normal and DPT captures any messages it issues. The $EMAIL function has separate parameters (see below) for the more commonly-needed elements like subject and recipients, but there are too many Blat options to give them all their own parameter, so everything except the basic ones must be given as "miscellaneous other parameters". You have to do that for example to add things like attachments and blind copy recipients. Refer to the Blat syntax document for the command line format when you need to do that. Also see the DEMOPROC examples.

    Blat set-up

    1. Download Blat if you don't have it already
    2. Issue the blat "install" command if you haven't done it already
    3. Copy blat.dll into a directory where it will be loadable by DPT. For example the Windows SYSTEM32 directory, or anywhere mentioned in your system's "PATH" variable, or even just the DPT host directory (although then you might need to repeat this step every time you installed a new version of DPT).
    4. Email the world
    The blat install command is required to save you having to enter SMTP server and logon details with every email. The format of the command is somewhat complicated with lots of options which you'll probably want to leave as defaults. Here's an example:
    blat -install mail.isp.com john@smithfamily.net - - - john@smithfamily.net sm1th
    
    The parameters are positional, so hyphens are used to specify defaults (the hyphens must be there). In this example there were 4 non-default values given. In order they were: The address of the mail server; the default "from" address for emails if none is given later when each is sent; a login ID to the mail server; and the corresponding password. You should be able to copy those details straight out of the "options" screen of your usual email client. To set a short name/nickname which will appear in the recipient's inbox, use the following syntax:
    blat -install mail.isp.com "John Smith <john@smithfamily.net>" - - - john@smithfamily.net sm1th
    

    Refer to the Blat docs for information about the items which took their defaults above if you're interested.

    $Function syntax

    $EMAIL(subject, message, to, from, misc options, diagnostics)


    FILE_xxx

    This set of functions provide a more direct interface to the OS file I/O facilities than is available via standard User Language IMAGE read and write. Files accessed with these functions do not need to be ALLOCATEd to the system. The functions are intended for applications needing to access files in an ad-hoc manner, such as might be the case in a web application, or any other native DPT application that wasn't just a port of a Model 204 application.

    Error codes
    In all cases error conditions are reported via $STATUS, although in some cases you can infer an error from the return value of the function itself. These are the $STATUS values. In many cases the error is accompanied by some descriptive text in $STAUSD, and on one or two cases some extra-detailed descriptive text in $STATUSD2.

    0 = Success
    1 = Invalid parameter option
    2 = Enqueue failure in OPEN or DELETE
    3 = Miscellaneous errors
    4 = Invalid file handle used

    $FILE_OPEN(os-filename [, disposition])

    $FILE_GETNAME(handle)

    $FILE_READ(handle, length [, offset])

    $FILE_READLINE(handle)

    $FILE_TELL(handle)

    $FILE_SEEK(handle, new-offset)

    $FILE_WRITE(handle, expression [,offset])

    $FILE_WRITELINE(handle, expression)

    $FILE_COMMIT(handle)

    $FILE_GETSIZE(handle)

    $FILE_CHSIZE(handle, new-size)

    $FILE_MODTIME(handle)

    $FILE_CLOSE(handle)

    $FILE_DELETE(os-filename)

    $FILE_SETG(global-name, handle)
    $FILE_GETG(global-name)
    $FILE_IMPORT(handle)


    $FREAD
    A shorthand version way of issuing several of the above calls in one go for more convenient use in some situations, particularly reading entire files at once.

    $FWRITE
    A shorthand version way of issuing several of the above calls in one go for more convenient use in some situations, particularly writing entire files at once.

    $DIR
    Companion to the above functions. Returns a directory listing from the host machine. This is one of the DPT custom functions that uses the very handy Sirius-style $Lists.

    $RMDIR
    $MKDIR
    Companions to the above functions. Named after the DOS commands ("remove directory" and "make directory") and the C function ("get current working directory").


    $LOAD

    This $function is equivalent to the =LOAD command, but offers a small amount of extra functionality, namely that the various options can be dynamically supplied via User Language expressions and variables. Examples:
    B
    %X = $LOAD
    %X = $LOAD (%FILENAME, %OPTS)
    %X = $LOAD ('SALES', 'PREVIEW ' WITH %PCOUNT)
    END
    

    $LOBDESC
    This can be used to retrieve the BLOB descriptor value held in table B if you're interested. There's nothing you can do with it really, and it's easier to use the LOB_NO_DATA option on the PAI statement to see BLOB descriptor values for debugging/tuning purposes if desired.


    $LOGXXX

    This set of functions makes it more convenient to issue the LOGCTL etc. group of commands than it is via $COMMBG. Plus in $LOGPWCHK, DPT provides a way to validate passwords other than at login time.

    $LOGCTL

    $LOGKEY

    $LOGLST
    Returns the output that would come from the LOGLST command into a $List. Pretty much as you could do with $COMMBG really but cutting out the middle man.

    $LOGPWCHK
    Allows you to validate passwords other than at login time, like with e.g. $RACF/$TSS etc. on Model 204.


    $LZW
    Simple string compression and expansion, via a bare-bones C++ LZW implementation by Nikos Bozinis. Compression ratios aren't as good as with a professional utility, but this may be acceptable to avoid the inconvenience of calling outside User Language. For example English text via $LZW typically reduces to 35-45%, vs say 15-25% with 7Zip.


    $NOW
    This function is most likely to be useful in a web application. It is the only realistic way to access GMT within User Language on DPT, and is also a convenient way to get a combined date+time value.


    $SHA1
    This is not implemented as an algorithm option on $HSH simply because $HSH returns a number not a string. Also, this function is separate from $ENCRYPT (which also uses SHA1 on DPT) because $ENCRYPT performs various extra operations in addition to just a straight hash.


    $SOCKxxx

    This set of functions make up a fairly straightforward interface to the socket stack for "connection-oriented" sockets. The operations available correspond closely to underlying function calls, and you should consult your winsock or general sockets documentation for more details if required.

    DEMOPROC in the DPT demo installation contains an example of simple use of this function to conduct a socket conversation between two users. Depending on when you're reading this the sample web site in WEBPROC may also contain an example of using this function to listen for returning connections from a served applet.

    Note that firewall settings in effect on your machine may block socket traffic, even via the loopback address.

    Error codes
    In all cases error conditions are reported via $STATUS, although in some cases you can infer an error from the return value of the function itself. These are the $STATUS values. In many cases the error is accompanied by some descriptive text in $STAUSD.

    0 = Success
    1 = Invalid paremeter option
    2 = The socket was bumped during the requested operation (most common while waiting in ACCEPT and RECEIVE but potentially also during a large SEND).
    3 = ACCEPT or RECEIVE timed out.
    4 = Invalid socket handle used
    5 = Any other errors

    $SOCK_LISTEN(port-number)

    $SOCK_ACCEPT(listening-handle [, timeout])

    $SOCK_CONNECT(address, port-number)

    $SOCK_SEND(handle, expression)

    $SOCK_SENDLINE(handle, expression)

    $SOCK_RECEIVE(handle, length [,timeout])

    $SOCK_RECEIVELINE(handle, length [,timeout])

    $SOCK_POLL(handle)

    $SOCK_SHUTDOWN(handle [, 'SEND' | 'RECEIVE' | 'BOTH'])

    $SOCK_CLOSE(handle)

    $SOCK_IMPORT(handle)

    $SOCK_SETG(global-name, handle)

    $SOCK_GETG(global-name)

    $SOCK_BUMP(handle)

    $SOCK_UNBUMP(handle)


    $SPAWN
    This is how you can start a fresh thread and get it to do some work. See the discussion of the
    =SPAWN command for general notes on kicking off daemon threads.

    Note that this function starts an entire fresh user thread, and does not make use of any existing $function support daemons. You might therefore use this function to perform larger background tasks and e.g. $COMMBG for smaller tasks.

    Note also the difference between a DPT user thread and an external OS task, which you can kick off using =SHELL.

    The $SPAWN function has all the same functionality as the =SPAWN command, plus the additional ability to specify multiple "real" input lines via a $list. The parameters are in a different order to the command because here the trailing parameters might often be left blank, whereas in the command the daemon's input line comes last as it might contain spaces and this saves you having to give quotes. The the default is also a synchronous call rather than asynchronous.

    When building daemon initiation into applications (rather than the occasional one-off manual use) it is probably best to choose a specific 5-character code and stick to it. This helps to identify in the audit trail what's going on, and also keeps down the amount of output file detritus in the #DAEMON directory from ever-increasing system-generated IDs.

    The return code is numeric and indicates:

    With negative (error) return codes the request continues with an informational message issued. With all nonzero return codes, $DPT_STATUSD contains the text of the corresponding message. If the return code was positive this will be a message that was issued on the daemon thread. The calling code can choose to regard return codes greater than zero as errors or not as required.

    There is currently no direct $function equivalent to the =SPAWNINFO and =SPAWNPRINT commands, but you can easily $COMMBG one of those commands.

    $THREADCODE
    This function works similarly to $JOBCODE, but affects the calling user's own high-water-mark instead of the system-wide one. One reason a separate function is used is to reflect the fact that, unlike Model 204, DPT only has one executable for both batch and online mode (see also differences in messaging).

    This function might often be used within a web application, since the web server's automatic error processing depends on the internal variable accessed by this function. It can be useful to know in advance whether that processing is going to kick in, or to deliberately cancel or trigger it.


    $TOKEN
    This and its variants $TOKENS and $TOKENIZE are general purpose functions, particularly useful when manipulating string data formats such as comma-separated records and text containing CRLF sequences. They are something like more elaborate versions of the Model 204 $WORD and $WORDS functions (with differences covered below).

    In all cases, the "tokens" within the input string must each be less than 64K characters long. The input string itself can however be any length up to the current STRINGMAX, so for example you can parse text lines out of BLOB field values or files read in using $FREAD.

    $TOKENS

    $TOKENIZE
    Following on from the above, this function does a one-off pass through a string and places the tokens into $LIST items. This is more efficient than a lot of scans to locate individual tokens. This function might be thought of like a generalized version of Sirius $PROCDAT.


    $UNLOAD

    This $function is equivalent to the =UNLOAD command, but offers some extra functionality, the most important part of which is that you can unload record subsets. Examples:
    B
    %X = $UNLOAD
    %X = $UNLOAD (%FILENAME, %WHAT, %FIELDSPEC, %OPTS, %SETNAME)
    %X = $UNLOAD (,'ALL INFORMATION')
    %X = $UNLOAD ('SALES',,'AGENT_NAME')
    %X = $UNLOAD (,'INDEXES','EXCLUDING AGENT_NAME, AGENT_DISTRICT')
    %X = $UNLOAD (,'DATA',,'PAI REPLACE DIR="C:\MYEXTRACTS"')
    %X = $UNLOAD (,'FDEFS AND INDEXES',,'EBCDIC NOFLOAT','LIST JULY.SALES')
    END
    


    Set C: Web server related $Functions

    Finally the set of functions used specifically for writing web server scripts. Like Set B, these are also installed as standard, and fully documented. They are grouped separately just for clarity of documentation.

    Request information $functions

    $WEB_GET_URI_PARM
    Retrieve the values of parameters passed in after the question mark on the request URI. Note that the browser will replace spaces in the parameter names/values with a '+' and/or other special characters like quotes and ampersands with %hexhex (e.g. %20 = space). These are re-converted by DPT before they hit your code and you need not worry about them. DPT can also optionally case translate both the parameter names and values.

    $WEB_GET_POST_PARM
    Returns information from the posted data block, if that method is used instead of URI parameters. The function works almost identically to the one above, so if an input page using URI parameters switches to the POST style, the host-side procedure could switch to this $function with no other changes. The browser handles the encoding, and DPT handles the decoding.

    This function also extends the URI version with a couple of extra parameter options reflecting the fact that POST parameters might not be simply name=value pairs. Specifically this means when a "TYPE=file" control is used as an HTML input form element, in which case DPT tries to obtain some extra information from the multipart encoded block.

    $WEB_GET_HEADER
    Access header information that came with the current HTTP request. See also the background discussion on the format of HTTP message headers, and the specialized cookie function below.

    $WEB_GET_CLIENT_ADDRESS
    Gives the TCP/IP address to which the response will be sent for the current HTTP request - usually the browser address. Depending on the make of browser, this information may also be available in one or more of the HTTP headers. Under certain conditions a web script might use this address to open another socket connection back to the browsing computer to achieve application functionality outside the browser.

    $WEB_GET_COOKIE
    This function can be thought of as a specialized version of the above, for retrieving values from the "Cookie" header(s) (see general cookie notes). It saves you having to write parsing code to unpick the name=value pairs from the header, as well as shielding you from the somewhat variable incoming formats this header can take.

    $WEB_PATCH_URI_PARM
    $WEB_PATCH_POST_DATA
    $WEB_PATCH_HEADER
    These functions are all used similarly, and are provided to support coding and testing activities. Under certain circumstances it's also possible you might find a use for these even in live code, perhaps to set up default parameters. Place the calls at the top of the handler procedure that would normally be expecting parameters. For example:

    B
    $WEB_PATCH_URI_PARM('NAME', 'SMITH')
    END
    
    * Main handler code
    B
    %NAME = $WEB_GET_URI_PARM('NAME')
    
    * etc.
    END
    

    Response information setting $functions

    $WEB_PUT_RESPCODE
    Changes or reports the code which will be sent in the HTTP response message title line.

    In an error situation, if your code has retained enough control to be calling this function, it should arguably not be considered an HTTP error at all, and the default response of 200 should be left in place. The way in which browsers handle the code you set must in any case be kept in mind, as discussed in the general coding notes.

    This function would then be reserved for more appropriate and specialized situations like redirects, not-modified, locking problems, etc., where specific browser behaviour is actually desired.

    $WEB_PUT_DATATYPE
    This is the recommended way of accessing the response's "Content-Type" header. It also allows you to use a file extension as shorthand:

    $WEB_PUT_DATATYPE('text/css; charset=UTF-8')   /? wordy  ?/
    $WEB_PUT_DATATYPE('CSS')                       /? tidier ?/
    

    $WEB_PUT_HEADER
    Add or change an HTTP header on the response message currently being constructed.

    Note that some headers are automatically added to your response message, and you should not set those yourself (the browser would probably ignore any you might set anyway). "Content-Type" and "Set-Cookie" are probably the two headers you would most often have good reason to tinker with, but they are catered for by $functions with specialized options (see below). This basic function is therefore most relevant to the occasional use of of headers such as "Modified-Date" (e.g. to control browser caching) or "Location" (e.g. during a redirect).

    $WEB_PUT_COOKIE
    Adds a new "Set-Cookie" header to the response currently being built (see also general cookie notes).

    Like $WEB_PUT_DATATYPE above this function is one you could do without if you tried hard enough, but your code will be neater and more readable if you use it. In a single call it constructs the potentially awkward value for the "Set-Cookie" header and adds it to the response.


    Response data buffer access $functions

    $WEB_PUT

    $WEB_PUT_FILE
    Using this function allows standard page elements to be combined in a slightly different way to using server side includes.

    $WEB_BUFFER_SIZE


    Control and miscellaneous $functions

    $WEB_FINISH

    $WEB_USE

    $WEB_AUTO_ERROR
    Enables or disables the automatic error processing.



    Part 3: Sirius Features

    Firstly a brief note about what is meant here be "Sirius Features". In the same way as DPT is code written to work like Model 204 rather than copying or stealing the Model 204 code, these "Sirius" features are not based on actual Sirius code. And none of them have, at the time of writing, been contributed to the DPT open-source project by anybody related to Sirius Software, so fans of their stuff will unfortunately have to make do with these imitations!

    All these features are considered part of the core product for emulation purposes, so are not disablable with the CSTFLAGS parameter.


    Sirius $functions

    General notes

    > > > > List of emulated functions > > > >

    The Sirius functions are considered part of the core M204 emulation, and are not disablable with the CSTFLAGS parameter like all the other custom functions.

    They are all, like all $functions in general on DPT, callable.

    $Lists

    DPT will by default allow list items of any length, but for Sirius compatibility it can artificially impose a limit of 6124 bytes if you set the CSTFLAGS X'04' bit. The $LIST_MAXIL function, as on M204, tells you which maximum is currently in effect. Some functions test or otherwise use the current maximum, for example many return code -7 to indicate an invalid length parameter. In all cases the processing that takes place corresponds to the maximum currently in effect.

    A $list on DPT is implemented by mapping more or less directly to the C++ standard library type "vector<string>", which in classical data structure terms means a simple array of pointers to strings. An alternative might have been to use a linked list (as suggested by the name "$list"), but since the most common type of accesses are to append items to the end and to read them by index number, an array is much more suitable. Only the less-common operations of inserting and removing items would be helped by using a linked list. On Sirius a $list is a custom data structure, and something (judging by the documentation) of a hybrid between array and linked list. Without more information it's hard to say more about its performance characteristics, except that there will be differences from DPT. Where the Sirius manuals discuss CCATEMP page fragmentation after using some $functions, the same issues will not apply on DPT.

    "sdaemons"

    One or two of the Sirius $functions call out to special background users which sit there permanently waiting to service such requests. On Model 204 they are called "sdaemons" ("s" presumably for "Sirius"). On DPT they are IODev 99 daemon threads very similar to those used for e.g. checkpointing and the web server.

    These daemons were introduced for the same reasons as the web server daemons, namely that in most cases each item of work they are asked to do is small in comparison to the overheads of setting up a fresh user thread. This should be kept in mind when submitting a task via e.g. $COMMBG: if you expect the task to be long, using $SPAWN instead to create your own dedicated thread would often be more considerate to other users, and better for the throughput of the system in general.

    The number of these daemons running at any time is governed by the FUNNDAEM parameter. The control loop is run by the =PSTFUN command.

    Long strings

    There is no need for an explicit "longstring" type on DPT, since any string variable can have any length so long as you set the STRINGMAX parameter high enough.

    This factor is also relevant when it comes to determining "valid" results in some of the $functions below. For example $SUBREP is officially documented to fail if the result would exceed 255 characters long, but on DPT the failure point is the current STRINGMAX value.

    Emulated Sirius $function differences/comments

    The following comments about a few of the functions may be of interest. You should consult your Sirius documentation for details of standard operation.

    Return codes
    In most cases DPT creates the same return codes as the Sirius equivalent functions. One difference is that DPT never gives a specific code for "CCATEMP full", since CCATEMP is not being used.

    Sirius wildcard parameters
    Whenever a function takes a Sirius-style pattern (e.g. $LIST_GLOBAL_DEL), DPT allows standard M204-style patterns, and in fact converts any Sirius-style to M204-style before processing. Basically this means query = plus and doublequote = pling, so for example 'J*".SM?TH' is processed as 'J*!.SM+TH'.

    Character set (ASCII/EBCDIC)
    Unless a function explicitly relates to ASCII and/or EBCDIC (like say $E2A where the meaning is clear), the characters are assumed to be ASCII in the Sirius functions. For example $HEXA converts a hex string (X'616263') to ASCII ('abc'). See also general UL notes on this issue.

    $COMMxx

  • The output of an asynchronous call goes not only to the audit trail but to the daemon's output file (see general daemon notes), which can save unpicking it from other system activity if you want a copy for some reason later.

    $IMGOVL (and $LISTFINDI/$LISTOVLI when a value is given)

  • See CSTFLAGS fudge switch X'8000' for how errors during the assignment are handled.

    $LIST_CAPTURE

  • OUTMRL *is* respected, but because of the difference in meaning on DPT, this should not cause problems.
  • For a note on pagination behaviour see CSTFLAGS fudge switch X'2000'.
  • Output capture is cancelled if the list is global and is deleted via $LIST_GLOBAL_DEL.

    $LISTADDI, $LISTINFI and similar

  • These functions directly access data in the image buffer, rather than applying all the complexities you get during READ and WRITE IMAGE, such as the population of default item values and the hokey behaviour of DEPENDING arrays. The use of $lists with images is mainly intended to provide an easy way to work with columnar data, rather than to simulate file I/O to $lists as an alternative to files.

    $LISTINF

  • To help cater for the ambiguous error-reporting convention of this function (list items containing e.g. the string "-9" might be confusing) $STATUS2 is also set to the same negative number whenever there is an error, or zero otherwise. This is a DPT custom extension.

    $LISTSORT

  • FLOAT 16 keys are treated as FLOAT 8. This agrees with image behaviour where the upper 8 bytes of a FLOAT 16 item are undefined.
  • The FI key type is assumed to mean SIGNED BINARY data, but will work for UNSIGNED so long as the most significant bit is not used. E.g. LEN 4 only values up to 2^31.
  • The PD and ZD key types will work with either SIGNED or UNSIGNED data or any mixture.
  • List items containing invalid data in the positions for sort keys of numeric formats like packed decimal always sort "high" regardless of the specified direction. This also applies whenever a sort key lies entirely outside the list item data. This behaviour was chosen to be like the SORT RECORDS statement.

    $PROCGET

  • On DPT this function also sets $STATUS2 to get round the ambiguity between documented meanings for the null string result (is it EOF or just a blank line in the proc?) $STATUS2 is set to 0 (OK), 1 (EOF) or 2 (error).

    $RANDOM

  • The result is by default an integer, but see CSTFLAGS fudge switch X'4000'.
  • See also custom variations DPT_RANDOMXXX later.

    $RANDOM_SEED

  • The variable need not be 140 bytes long. Any numeric variable, or string capable of holding a 10-digit integer is OK.


    Sirius compiler directives

    Sirius directive
    The only option currently supported is the case option for temporarily overriding the active mixedcase UL mode until the end of the current procedure.
    Sirius Case {ToUpper|Leave}
    
    This directive is entirely case-insensitive. Note also that although the DPT editor will recognize these keywords, they do not affect colour-coding or uppercasing of code.

    Precompiler macro language (!...)
    Pling directives are not currently supported on DPT.


    Mixedcase User Language

    The following Sirius-style features are supported on DPT to enable mixedcase User Language programming.

    Editing mixedcase code on DPT
    When writing mixedcase User Language in the DPT client editor, you will almost certainly want to turn off the option which makes it uppercase whole lines as they're entered (as you would enter "*LOWER" mode) on M204.

    On the other hand, if you're sticking with the convention of uppercase language keywords (OCC, SCREEN, $function names etc.), leave that option active, since having the editor auto-uppercase and colour them is a good way to encourage adherence to that coding standard. Remember this is just a graphical feature of the editor, and having it active does not affect how the compiler on the host might treat mixedcase keywords at any given moment. BEGIN/B is recognized in mixedcase regardless of editor settings.

    See also: full details of case-handling options in the DPT editor.