DPT Language Reference
Database Programmer's Toolkit
Language Reference and Compatibility Guide
The terms "Model 204" and "204" are trademarks of Rocket Software Inc., and that fact is acknowledged wherever those terms are used in this document. Likewise "Sirius xxxxxx" (anything) are trademarks of Sirius Software.
Contents
Notation Conventions in this Document
In simple cases, the common backus-naur style symbols are used to denote the required syntax of commands and statements, similar to the Model 204 manuals, viz:
- Optional elements are enclosed in square brackets
TIME [REQUEST]
- Alternatives are separated with the "|" sign, with any default underlined
CREATE [TEMP | PERM] GROUP groupname
- Compulsory elements are enclosed in curly brackets
BUMP {user-number | user-name | ALL}
- Keywords are in capitals, variable components are in lowercase, and single characters are sometimes enclosed in quotes where there might be confusion over whether it's part of the notation or part of the command.
MONITOR ["(" user-number ")"]
- Options which are accepted for readability reasons but do not have an effect they might be expected to have are in italics.
BROADCAST [URGENT] [message]
Sometimes, especially where more formal notation would be very complicated, examples are used instead. MONITOR is a good example, where there are a number of "flavours" of the command, each with individual options.
Part 1: Commands
Line continuations
All command lines can be continued with a hyphen on DPT. This differs from M204, where only some commands can be continued.
IN filename
As with User Language, IN clauses can not include the "AT location" phrase, since there are no scattered groups on DPT.
Quote characters in commands on DPT
Some commands take options which can be surrounded by quotes. On Model 204 the only accepted quote character anywhere (including User Language) is the single-quote, and this is respected. However, with some commands that interact with the operating system, the double-quote character is also allowed by DPT since it is commonly used, sometimes even compulsory, on many platforms. For example:
ALLOCATE F1 'MY FILES/F1.DPT' /? usual M204 convention ?/
ALLOCATE F1 "MY FILES/F1.DPT" /? equivalent on DPT ?/
ALLOCATE F1 'MY FILES/F1.DPT" /? no good - must not mix types ?/
This can allow an alternative to the usual convention of repeating quotes when you need to put quotes within quotes, since a value starting with one type of quote can only be ended with the same type. E.g
=MIME DEFINE .txt 'text/plain; charset="UTF-8" '
=MIME DEFINE .txt "text/plain; charset=""UTF-8"" " /? equivalent but less neat ?/
Note that in the above example the semicolon in the MIME value would normally have indicated a line-split in the usual M204 way, and didn't only because it was enclosed in quotes. Terminal line input case translation (controlled by *UPPER/*LOWER) can however not be subverted like this.
Custom options
Some of the M204 commands have DPT custom options. Many of these can be disabled using the CSTFLAGS parameter, but a few are essential for system operation and cannot be disabled. In any case all custom options are covered in the by-command discussions below.
Individual M204 Commands - Processing and Syntax Differences
Model 204 commands that do not feature in this section can be assumed to either work pretty much the same on DPT, or not be emulated at all (see cross-reference). When an unsupported Model 204 command is issued, the DPT message usually gives some idea of the reason, and in some cases suggests a similar alternative. By default such messages are information level (CLASS=I) and processing is allowed to continue.
ALLOCATE
- ALLOCATE filename {osfilename | TEMP} [DIRECT | SEQ[UENTIAL] | PROCDIR] [OLD | NEW | COND | MOD | READONLY] [LRECL [=] value [PAD [=] value]] [MAXSIZE [=] value] [ALTNAME [=] value] [PRSUFFIX [=] [.]extension]
- See usage notes in the DBA Guide for some comments re. the huge range of missing options compared to Model 204.
- Quotes are only required on the osfilename if it's a path containing spaces. The quotes may be single or double. The osfilename is passed to the OS exactly as specified, so can be, and often will be, just be a relative path to the host's working directory.
- MOD, READONLY, TEMP, LRECL, PAD and MAXSIZE are only valid on sequential files. Some are DPT custom options, covered here in the DBA guide.
- The default disp is OLD and not NEW like on M204.
- With PROCDIR you must, naturally enough, specify a directory and not a file.
- ALTNAME and PRSUFFIX are special options for use with proc directories.
- To specify a null string for PRSUFFIX, use a dot, as you would when resetting the global PRSUFFIX parameter.
ANALYZE
- ANALYZE [(DESCENDING)] [[LEAF]PAGES] [fieldname]
- This command reports information about the b-trees and inverted lists maintained for indexed fields, as on Model 204. There are some custom extensions to the command which add functionality, as covered below.
- Firstly the basic command. This is the same as on Model 204, with the difference in output format really just reflecting differences in low-level details of the index data structures.
- PAGES and LEAFPAGES are DPT custom options which cause a different and very interesting report to be generated. Information is shown for every node in the b-tree, giving a comprehensive picture of current node utilization: space available, range of values present, the active compression level, etc. For string nodes where prefix compression is active, the prefix values not actually stored are shown in brackets. LEAFPAGES is the same as PAGES except that nodes below leaf level are not shown.
- The DPT custom DESCENDING bracketed option does not affect the output but merely tells the system to perform the analysis starting at the high end of the b-tree. This option is really for technical diagnostic purposes, to ensure that descending-cursor processing such as used by descending value loops, works correctly.
- The PAGES option could be useful for anyone on a file-management training course or learning about b-trees for the first time, especially when used in conjunction with the custom BTREEPAD file parameter, which is mainly there for such purposes. Setting this parameter to a non-zero value causes DPT to think b-tree pages are fuller than they are, and you can make node splits occur at will, observing the effects on the tree structure, string compression, BX**** statistics produced by UL requests, etc.
- Likewise see the custom =ILINFO command later.
BROADCAST
- BROADCAST [FILE] [URGENT] message
- The URGENT option has no effect. It can be given but is ignored.
- Non-file broadcasts are received be users the next time they are at command level about to issue a command. A user currently about to issue a command receives the message after their next command.
- File messages are received when the file is opened, from a closed state. That is, a user who currently has the file open does not get the message if they issue the OPEN command without an intervening CLOSE command.
CHECKPOINT
- CHECKPOINT [cpto-override]
- The difference from Model 204 here is that you can supply an override to the CPTO parameter. The default (-2) is to use the current value of CPTO. A value of -1 means abort immediately if quiesce fails.
- See the System Config Guide for more on checkpointing and CPTO.
- Most commonly this command would be issued periodically by a system daemon (see =PSTCHECK).
COPY DATASET
- COPY [DATASET | STREAM] dd1 TO [DATASET | STREAM] dd2 [COUNT=n | UNFORMATTED]
- The STREAM and DATASET keywords are optional and interchangeable. Using one or the other does not affect the operation of the command at all.
- Both files must be allocated as SEQUENTIAL.
- If the DPT custom option UNFORMATTED is specified, the file copy is performed a great deal faster, in 64K blocks, but any LRECL, RECFM and MAXSIZE specifications given on the two files at ALLOCATE time are ignored. If the destination file is allocated as MOD, this is respected. COUNT=n is incompatible with the UNFORMATTED option.
- Without the UNFORMATTED option, data is copied a "record" at a time, obeying the allocation specifications of the two files, truncating and padding the records as necessary and stopping when either COUNT records have been processed, or DD2's MAXSIZE is reached.
- A per-record copy involves a lot of processing overhead, and if you know the two files have the same format can be a waste of time. (For interest, the system takes this view when performing COPY PROC - see below - and does an unformatted-style copy of one proc to the other).
- See also the custom command =SEQREAD.
COPY PROC
- COPY PROC procname1 TO context-specification [REPLACE] [OLDDATE] [NEWNAME=procname]
- There is no ALIAS option, since we are limited by the information maintained for files be the OS.
- On which subject, if you use the OLDDATE option on an OS (e.g Windows) which maintains file created-time, you can easily create a situation where a proc appears to have been modified before it was created.
- For multiple copies (the LIKE option is also unsupported) OS facilities are probably more appropriate - e.g. a command-line COPY or Windows Explorer drag-and-drop. You can easily SHELL a COPY command.
- To copy to a temporary procedure use e.g. COPY PROC -1 TO TEMPPROC NEWNAME -4. I.e. 'TEMPPROC' stands in for the destination file name.
COPY STREAM
CREATE FILE
- CREATE [FILE] filename
- ... parameters ...
- END
- There is no option to create a file from multiple OS files like with M204, hence no FROM option.
- The FORMAT/NOFORMAT option is not provided.
- The supported file organisations are entry order (FILEORG=0) and unordered, which must also be RRN (FILEORG=X'24').
- Only one parameter may be specified per line (i.e. much like "CCAIN"). Of course semicolons can be used to denote newlines if desired for readability.
- Usage notes.
CREATE GROUP
- CREATE [TEMP|PERM] GROUP groupname FROM f1 [[,] f2...]
- ... parameters ...
- END
- OPTIONAL/MANDATORY only applies to PQO on M204, and is not valid on DPT.
- All groups are potentially procedure file groups, so the PROCFILE parameter is not valid (it is always effectively "*").
- BLDGFT is always effectively YES, so that parameter is not valid either.
- All the security-related parameters are invalid.
- In short, the only parameter you can specify is UPDTFILE. If this is not specified, you won't be able to use STORE RECORD in group context, and $UPDATE will not work either, (but IN GROUP MEMBER %xxx and IN $CURFILE will work as normal).
- See the DBA Guide for more on groups, and for more on procedure processing in group context.
DEFINE FIELD
- DEFINE FIELD fieldname [(attribute [[,] attribute...])]
- DEFINE FIELD fieldname WITH [attribute [[,] attribute...]]
- The rules for valid field names are pretty much the same as on Model 204. Specifically they must begin with a capital letter and be from 1 to 255 characters long. (Note that the M204 manuals are not totally accurate in this area - DPT aims to agree with actual behaviour). See also the section on using of reserved words in field names.
- An additional DPT-specific restriction is that field names containing the ASCII characters 0-31 are disallowed, even if quoted. The issue of ASCII control characters is discussed elsewhere.
- Field attributes: In all cases the equivalent M204 abbreviations can be used, or the full keywords. Only the following attributes are allowed.
- Storage attributes: [STRING | FLOAT | INVISIBLE]. FLOAT always means FLOAT LEN 8, namely a floating point value as per the usual UL rules (see UL number handling).
- FLOAT and STRING fields can be: [UPDATE IN PLACE | UPDATE AT END]
- STRING fields can also be: [BLOB | NBLOB]
- FLOAT and ORD NUM fields will not accept non-numeric values like Model 204 sometimes can (FILEMODL allowing). They are either rejected or converted to zero.
- Indexing attributes: [NORD | ORD CHAR | ORD NUM]. One of the latter is required if the field is invisible.
- Model 204 disallows the combination FLOAT + ORD CHAR, but you can do it on DPT if you really must.
- The only M204 b-tree control parameter for ORDERED fields is [SPLITPCT[=]S] where S defaults to 50. Use 99 when loading a file which will be read-only.
- DPT also has a custom field attribute NO MERGE (abbreviation NM) which affects the single-step load process. This attribute is one that can be REDEFINEd, in which case use "DU MERGE" to revert to the default.
DISPLAY FIELD
- D[ISPLAY] FIELD {etc...all M204 options}
- The main difference with this command from Model 204 is that it is allowed in group context. The group field table is reported as if it were a normal file. Field attributes that can differ across files in a group are shown as [grp] rather than a normal value.
DISPLAY FILE
- D[ISPLAY] FILE {[(option [[,] option...])] filename [[,] filename...] | ALL | ALLG}
- This command on DPT has to cope with the fact that the user may have two types of “files” open, namely procedure directories and data files. The display can be limited to one or the other by using the new options PROC and DB. The default is to show both, and using either option turns off the other one.
- For data files, the PARMS option shows all parameters and values, (amalgamating the FPARMS and TABLES parameter categories), whereas for procedure directories it shows the number of procedures in the directory. In both cases the OS file path is shown.
- In both cases, the custom option GROUPINFO shows information about whether the file is open in single file and/or group context. FULL is a shorter way of specifying both PARMS and GROUPINFO.
- Specifying ALL instead of a list of file names shows all open files and/or directories. The custom option ALLG includes ALL, but also shows a list of the user’s open groups.
- To see a list of all files allocated to the system, but not necessarily open, see =LISTALC.
- When a proc directory has an ALTNAME, the file is listed by this command under the name it was opened as, which may be either the real DD name or the ALTNAME.
DISPLAY GROUP
- D[ISPLAY] [TEMP | PERM] GROUP {groupname [[,] groupname...] | ALL | ALL2}
- No bracketed display options are allowed with this command on DPT (NAMES/DEFINITIONS). The display is always as per DEFINITIONS on M204 (the default).
- There is also a DPT custom option to include ad-hoc groups (which might be hanging around after END MORE). These are otherwise suppressed from the display - use ALL2 instead of ALL to include them.
DISPLAY GTBL
- D[ISPLAY] GTBL {LIST [LIKE pattern] | [NAME=] globalname [[,] globalname...]}
- This is pretty much the same, except that since the globals always come out in alphabetical order with LIST, there is no IN ORDER option.
- The global names may be surrounded with single quotes to cater for names which might contain spaces, commas or the words LIST, LIKE and NAME.
- D[ISPLAY] GTBL OBJECTS
- This is a custom variation of the command which lists all the global objects currently in the global table. The report shows each object's name, type and permanence, plus a small amount of object-specific information such as for example with an image whether it's defined or just declared.
DISPLAY PROCEDURE
- D[ISPLAY] [PROC[EDURE]] [(option [[,] option...])] LIST [ALL | TEMP | LIKE pattern]
- Of the M204 options, BRIEF, COMPACT or VERBOSE are supported, and ALIAS, HDR and NOUSE are not. HDR is always on for BRIEF and VERBOSE. VERBOSE is the default, as on M204.
- Only files with an extension equal to the current setting of PRSUFFIX are shown, unless PRSUFFIX is set to a null string (C''), in which case all files are shown.
- D[ISPLAY] [PROC[EDURE]] [(option [[,] option...])] {[NAME [=]] {name|number} [[,] {name|number}...] | ALL | TEMP}
- The only option supported on the "text" form of the command is LABEL. ALIAS, COMPACT, NOUSE, NUMBER and PRIVILEGES are unsupported for reasons outlined elsewhere. The display is always effectively COMPACT, since most commands do not invoke output paging on DPT. With LABEL, the IN FILENAME clause prefixes the procedure name only in group context.
- TEMP is a DPT custom alternative to ALL on both flavours of the command, and displays temporary procedures, starting at 0 and working backwards.
EDIT
- EDIT [ (ASYNCH | SYNCH) ] procname
- This command is only allowed on the IODev7 client.
- No second "destination" procedure name can be specified. Users sometimes do this on M204 to effect a procedure copy programmatically - use COPY PROC instead then. Otherwise use "Host Save Copy" in the editor.
- A more important difference, especially when using EDIT programmatically, is the ability to specify synchronous or asynchronous processing of the command, and this option is a custom extension. The default is ASYNCH when EDIT is issued from the command line and SYNCH when EDIT is issued from inside a procedure. ASYNCH is invalid from within a procedure, since the editor needs access to the command line to work.
- If the procedure is already being edited on the IODev7 client, focus is simply switched to it. In such cases the existing edit must have been ASYNCH, but if the new command specifies SYNCH, this is respected.
- Since there is no line editor, the Model 204 TERM[INAL] and LINE options are redundant. TERM and SCREEN are ignored, LINE is disallowed.
- Editing issues are convered in more detail in the IDE User Guide.
FREE
- FREE filename
- The difference here is that discontinuities are never allowed, so FREEing a file is not allowed if a file has been updated but not checkpointed. This restriction is implemented by simply taking an implied checkpoint as part of the FREE command. This is unlike a normal checkpoint in that it "times out" if all updates can not be quiesced immediately. This is similar to issuing "CHECKPOINT -1" (see above).
HALT
- HALT
- There is no need to specify a console message or the desired reply on DPT, and those options are disallowed.
- The System Config Guide gives full details on starting and stopping the system.
INITIALIZE
- INITIALIZE [KEEPDEFS | NOFLD]
- The KEEPDEFS option leaves the field definitions in place. It *does not* reload them, so field IDs remain the same after the command. However, it *does* move all field attribute pages to the base of table D if they weren't already.
- NOFLD is just an old synonym for KEEPDEFS, since DPT had this option before Model 204 implemented it with the latter name.
- On DPT, the INITIALIZE command also cleans up any table B/D fragmentation caused by earlier use of INCREASE, so that Table B and D are afterwards each composed of a single extent. Note that "extent" here means as perceived by the DPT file system - there may still be OS level fragmentation.
LOGCTL
This command is partially supported. See also the equivalent $function.
Nearly all the supported options have a wider range of allowable values than on Model 204, as follows.
- First line:
- LOGCTL {A | C | D} userid
- No VM/CMS options (P etc.) are supported.
- File (:) and group (,) variations of the command are not supported.
- User IDs can be up to 30 characters long on DPT.
- Second line:
- password, privileges
- Priority is not supported.
- On DPT passwords can be any non-zero-length string.
- Privileges are held as a 32-bit number, unlike Model 204's 8 bits. You can use the extra bits for any custom processing.
- The privileges can be given in decimal form, such as 255, or hex form, when X'FF' would be the equivalent.
- Third line:
- Terminal list is not supported, so DPT will not even request a third line.
LOGKEY
- LOGKEY
- This command is supported only for the purpose of reporting the user ID and time of the last update of the password file.
- No "key" is actually stored anywhere so there is no parameter on the command.
- See also the System Config Guide section on this.
LOGWHO
- LOGWHO [option [[,] option...]]
- This command shows both less and more than the equivalent M204 command.
- The "more" is with the options, which are all custom extensions. The command can take any combination of the options NOPROC, NODB and DEVICE.
- NOPROC means *don't* show the user's open procedure directories. The default is to show them, and they will be indistinguishable from data files (see below). Files which are open in both "ways" only feature once in the output.
- NODB means *don't* show the user's open files. The default is to show them. They are displayed in the same way as on M204, namely that if a file is only open as part of a group, there is an asterisk after the name. Files open in their own right as well as via a group do not have the asterisk.
- The DEVICE option means show the values the user has for the parameters IODEV, INPUT and OUTPUT. The default is not to show them.
- These are the only things that LOGWHO shows on DPT. In other words there is no information about dead terminals etc., as described in the M204 Command Reference Manual.
MONITOR
This is actually several different commands with some shared behaviour between them. The basic form is:
- MONITOR [subcommand-type-and-options] [[EVERY] n]
- In other words all monitor commands can be given an EVERY value, which makes the display repeat after that number of seconds. If omitted, a single display is generated. This is much the same as on M204, in that on file-based devices like User 0 or a daemon, the repeated display forms an ever-lengthening report until the thread is bumped, and on an interactive device like the DPT client, the user has to press enter to get the next display.
- On the DPT client (IODev7) a useful option is to get the front end to "press enter" automatically at these new page prompts, so you can get a self-updating monitor display.
- The default if no subcommand is given is USERS. That is, something similar to Model 204 (see below for more details).
Variations of the command are of two main types, namely statistics monitoring variations and miscellaneous other variations. Let's take the former:
- MONITOR [SYS[TEM] | USER[S] | FSTAT] [STATS]
- If the keyword STATS is also given it means present the statistics in an unformatted line as per the audit trail. The default is to present a subset of those statistics in a formatted columnar report. Specifying STATS therefore gives more information, but is less readable in the most common cases where CPU, DKPR etc. are of interest.
- Examples:
- MONITOR
- MONITOR USERS (same meaning as MONITOR)
- MONITOR FSTAT STATS EVERY 5 (repeating unformatted file statistics)
The USERS variation has a couple of special options in addition to the STATS option mentioned above.
- MONITOR [USER[S]] [(usernum [,usernum]...)] {SL | FILE[S]}
- The SL option means show since-last stats instead of logout stats.
- When displaying SL stats, each user's current activity and/or procedure name is shown, if known.
- The default is to show all users, but a list of user numbers can be given just like on M204.
- The FILE option actually generates a display much like LOGWHO, and perhaps should not be a MONITOR command at all. Just copying Model 204 though.
- Note that in the formatted report, the column showing "Thread ID" is sometimes referred to by low-level DPT diagnostics. See the system configuration guide for more details on thread scheduling on DPT.
- Likewise, the WT/FLGS column shows estimated "nearest equivalents" to the M204 values. Because of the total difference in scheduling philosophy on DPT, these can not always, if ever, mean exactly the same thing. . In some cases, the custom command MONITOR RESOURCE (see later) will give more detailed information than WT/FLGS on who is holding and waiting for resources.
- Examples:
- MONITOR USER SL STATS EVERY 5 (repeating unformatted since-last stats for all users)
- MONITOR (5,6) (formatted logout stats just for those users)
- MONITOR FILES (open files for each user)
The FSTAT variation can also be restricted in scope:
- MONITOR FSTAT [(filename [,filename]...)]
- With the obvious meaning.
Other miscellaneous variations of the MONITOR command are:
- MONITOR {CHECKPOINT | CHKP}
- This command shows the same information as on M204, plus the information that M204 would present in response to the CHKMSG command.
- When a checkpoint is currently being inhibited, only the user threads are shown, not the procedure names they are executing. Use MONITOR SL (see below) to get that information if desired.
- The "extended quiesce" information alluded to in the M204 manuals is irrelevant on DPT, as checkpointing is somewhat simpler.
- MONITOR CFR
- This is implemented using the more general custom MONITOR RESOURCE command.
- MONITOR DISKBUFF
- Similar to M204 (more detail actually).
- There is also a custom command, =BUFFDUMP, which gives a different lower-level spin on the buffer contents.
- MONITOR SUBSYS
- This is implemented using the more general custom MONITOR RESOURCE command.
MSGCTL
- MSGCTL DPT.number [TERM | NOTERM] [AUDIT | NOAUDIT] [CLASS[=]{E|I}] [RETCODE[O|B]=number]
- Sets up a user-specific override to the system settings for the message. These are the only options supported.
- Any options not specified on the command are left as they are.
- TERM and NOTERM can effectively be managed for all messages at once using the MSGCTL parameter, which is user-specific.
- The CLASS setting (E) controls what is considered a "counting error" for all purposes, namely the aborting of a compilation after ERMX; whether to run a request at the end of compilation; invocation of subsystem error procedures (CNT); retaining message text for access via $FSTERR, $ERRMSG and VIEW ERRORS. As with M204, the error count is zeroed whenever the user returns to command level, at the start of a new or continued request at any include level, and when a new subsystem procedure is included.
- There is no difference between "batch" and "online" processing in the current release of DPT, so the RETCODEO and RETCODEB options are interchangeable. Just RETCODE can also be used.
- Remember that DPT's message codes are not the same as on Model 204, so any MSGCTL commands in ported applications will need attention.
- The message with the highest RETCODE setting issued during a run is the return code that's set by the host application when it ends. All messages issued to the audit trail that have non-zero a RETCODE setting are highlighted as such to make them easier to find. A message need not be issued to the audit trail to set a non-zero return code however, (and also need not be issued to the user terminal, or even be set as an error).
- User-specific message code high-water-marks are also kept, for use in some situations. For example the return value from synchronous $SPAWN and to control the web server automatic error processing.
OPEN
On DPT this command is largely the same as on Model 204, except that is has a special variation of the parameterized format to open a file in deferred update mode.
- OPEN file-context
- OPEN file-context, ddn, dda [, num-pad-length [, format-options]]
- OPEN file-context, 1STEPLOAD
- General notes:
- With the plain command, the file context can be a group or single file context, as on M204. With the parameterized (deferred update) variations the context must be a single file.
- This command will open both a procedure directory context and a data file context, if there is one of each with the supplied name. The command is only considered to have failed if both these attempts fail.
- Format 2 for multi-step deferred updates:
- The optional parameters are the deferred update file names which need not begin with "TAPE" on DPT. If these are specified, the context may not be a group. Both file names are required, and both must be sequential files that are currrently allocated. {Note about defer DD names: The Z command which will be required later assumes the names "TAPEA" and "TAPEN", so it's simplest to just use those. Unless you want to run several files at the same time with deferred updates turned on.}
- You should not specify LRECL on the allocations for these files, as that could truncate the deferred update records and thereby corrupt the indexes.
- The optional "num-pad-length" parameter affects the format of the deferred update records for ORD NUM fields, if there are any. Any positive value here causes these numeric values to be stringized for the sort and padded on the left (i.e. right-justified) to the specified length, rather than the default which is floating point representation (see also NOPAD option below). Field values encountered which exceed the specified length cause load failure. Try to give the lowest value possible when using this option in order to save disk I/O during the sort. At the end of a load DPT issues a message saying the longest value encountered, so you can reduce the allowance next time if appropriate.
- Further formatting options for the deferred update sequential files are keywords and can be added, separated by commas, after the above 3 values. Accepted keywords are:
- NOCRLF:
With the variable-lengthed formats (ORD CHAR or stringized ORD NUM) field values are preceded with a length byte instead of using a CRLF separator at the end of the record. Allows X'0D' and X'0A' in values, at the cost of a more unconventional record format.
- NOPAD:
Stringized ORD NUM values are written unpadded, which saves disk I/O if your sort utulity can handle numeric string values laid out like this. The NOPAD option triggers the use of stringized format for the ORD NUM values, in the same way as does a positive value for the "num-pad-length" parameter above. If both are given, the pad length is ignored.
- Format 3 for one-step deferred updates:
- After the file name the keyword "1STEPLOAD" must be given.
- The values of the LOADMEMP and LOADDIAG parameters at the time of issuing the OPEN command are used throughout the deferred update processing.
- Note: the two optional parameters that used to go with this command are no longer valid - their job is done by the above LOADxxxx parameters.
Examples
*One-step:
OPEN SALES, 1STEPLOAD //Default
OPEN SALES, 1STEPLOAD, 50% //Invalid from V2.19 - use LOADMEMP parameter
*Multi-step:
OPEN SALES, TAPEN, TAPEA //use default formats
OPEN SALES, TAPEN, TAPEA, 6 //stringize numbers - pad to length 6
OPEN SALES, TAPEN, TAPEA, , NOPAD //stringize unpadded - length irrelevant
OPEN SALES, TAPEN, TAPEA, , NOCRLF //binary format numbers, length byte strings
OPEN SALES, TAPEN, TAPEA, , NOPAD, NOCRLF //length byte on both strings and stringized numbers
PROCEDURE
- PROCEDURE procname
- Aliases and security classes are not supported, so those options are disallowed.
- This command is not allowed in group context, because of the differences in procedure handling from M204.
- The rules for valid proc names also differ slightly.
REDEFINE FIELD
- REDEFINE FIELD {options as per define field}
- The following do not require structural changes to existing data, and redefining them takes next to no time. Any number of these options can be redefined on the same command.
- UP/UE for visible fields
- SPLITPCT for ordered fields
- [NO] MERGE for ordered fields (DPT custom option)
- The following involve changes to information throughout the file and may take a long time to perform. Only one of these changes may be performed per command.
- FLOAT to STRING or vice versa (reads all of table B, possibly writes all too, probably slow)
- FLOAT or STRING to INVISIBLE (as above)
- NBLOB to BLOB or vice versa (as above, plus probably table E work too, so even slower)
- INVISIBLE to FLOAT or STRING (reads entire index, retrieves and writes to perhaps every record, perhaps several times, probably very slow)
- ORDERED to NON-ORDERED (quickish - just freeing up pages in table D)
- ORD CHAR to ORD NUM or vice versa (extracts existing b-tree, resorts and builds a new one, inverted lists are touched little if at all, shouldn't take too long)
- NON-ORDERED to ORDERED (reads all of table B, builds b-tree and inverted lists, probably quite slow)
- Other notes:
- The processing takes place as a single, non-backoutable update, so for non-trivial changes a file backup beforehand is a good idea.
- When changing the storage or index format from string to numeric, or adding an ORD NUM index to a STRING field, there might be issues with number validity, possibly causing REDEFINE to fail, in which case the file will be left physically inconsistent.
- Likewise, when changing a BLOB STRING field to a plain STRING field, if any value is longer than 255 bytes, the command will fail in the same way.
STRING to BLOB will cause no such problems of course.
- Any change between STRING and BLOB will be moving a potentially large amount of data between table B and table E (table D), so it's a good idea to check their sizes beforehand and INCREASE if appropriate.
- SEQOPT will often be beneficial to this type of processing, and is worth experimenting with, especially for the commands which walk through table B (NON-ORDERED to ORDERED, any table B format change). {Note: incidentally, the same consideration applies to DELETE FIELD.}
- The final option above (create a new index from scratch) works under the covers by using the same fast index-build code used in the single-step deferred update process, and many of the same discussion points regarding memory chunks etc. apply. Likewise some of the internal processing can be affected by the LOADxxxx group of parameters.
- In cases where several changes are to be made, a reorg (i.e. unload and freshly reload the whole file) might save a lot of IO etc. compared to multiple passes over the same data pages with several REDEFINE commands.
RENAME FIELD
- RENAME FIELD oldname, newname
- The difference here is that the command is much quicker, and therefore somewhat more useful.
- It seems that on Model 204 the new name is hashed into table A, which means a new field code is allocated, which must then be rippled through tables B, C and D. On DPT, since the field attributes area is not a hash table, the same field code can be retained, and the command is near-instantaneous.
- See DEFINE FIELD above for notes about valid field names.
RENAME PROCEDURE
- RENAME [PROC[EDURE]] oldname, newname
- Since no procedure aliases are stored, this command translates into a straight OS file rename.
- Unlike M204 it *may* be issued from within a procedure, as long as (obviously) it's not the one being renamed.
REORGANIZE
This command is more comprehensive than the Model 204 REORGANIZE OI command, but does include its functionality. On DPT the command is used for both index-only and full-file reorgs.
The other main difference from M204's REORG OI command is that on DPT the unload and the load phases of the reorg happen with a single invocation of the command, whereas on M204 output files are created that must be sorted and then loaded by the Z command. The DPT scheme saves all the hassle of configuring JCL and tracking separate job steps at run time.
A third, more low-level, difference is that on Model 204 the output of the REORG IO command consists of sequential files identical to those from a deferred update run. On DPT the intermediate sequential files are a special custom format used by this command. The older deferred update file format is completely different.
DBAs often want to make file or field changes as part of a full-file reorg (see recipes in the DBA guide), and in such cases using explicit =UNLOAD and =LOAD commands with an intervening INITIALIZE gives greater control than a single REORGANIZE command. The underlying processing is virtually identical, but it may be slightly slower overall because DPT is given less chance to take shortcuts, cache data in memory, etc. For example when using separate unload+load, the intermediate data will always be written out to disk, even for a smallish file which would all fit in RAM.
- REORG[ANIZE] [OI [[FOR] fieldname]]
- If the OI keyword is not given, the entire file is reorganized, including the data area (table B) too.
- The OI keyword with no field name means just reorganize the indexes (don't touch table B), and do it for all indexed fields.
- Giving a field name further restricts processing to that field's index. The keyword "FOR" is optional and does nothing. To process more than one field, but not all, issue a separate command for each.
- There is no way to just do a table B reorg on a file which also has indexes. (Doing so would usually involve lots of changes to inverted lists that would make an index reorg advisable anyway).
- Other notes:
- The table B part of a reorg results in efficiently-packed records as per the new file parameters and field definitions. Free record slots are reclaimed, etc., etc., etc. Note that internal record numbers may well change during this, although record order is preserved.
- Reorganizing field indexes reloads their b-trees according to the current SPLITPCT value, and repacks inverted lists (bitmaps etc.) appropriately according to the current population density of the field values.
- A full-file reorg has an implicit INITIALIZE at the halfway stage, before reloading, which includes the table-defragmentation processing invoked by the INITIALIZE command.
- In most ways the underlying operation of this command is the same as if you'd issued separate =UNLOAD and =LOAD commands, so those doc sections should be consulted for detailed processing notes.
- Any temporary files generated are placed in the #REORG directory, which is cleaned up automatically.
RESET
- R[ESET] parmname [=] newvalue [[,] parmname [=] newvalue...]
- The main difference with this command is that it will not function as a compound command, i.e. you can't just issue RESET and then enter the name=value pairs on successive lines with ENTER in between. It must all be issued on the same line.
- Values can be specified in a variety of ways, as per the following examples:
- String parameters:
R AUTOSYS MAINMENU //Simple ASCII literal
R PRSUFFIX C'' //If null value, or contains e.g. spaces, semicolons
R COMSTART X'f5E5' //Rarely useful
- Numeric parameters:
R ERMX 10 //Simple numeric literal
R LAUDIT X'0003' //Best way to access bit flags
RESET COMMAND
- R[ESET] COMMAND commandname USE=use-option
- The USE option can be one of RESPECT, IGNORE, NOCLOSE, NOOPEN or NEVER
- The privilege option is not allowed, as there are no proc access privileges on DPT.
- Another way this command may differ from M204 is that the command name must be given in its "canonical form", for example "DISPLAY FIELD" rather than any abbreviation "D FIELD". The VIEW COMMAND ALL ALIASES command produces a listing of all the command canonical forms and aliases for each. Note that in some cases such as MONITOR and DISPLAY, different flavours of the command are treated as different commands for the purposes of their USE behaviour.
STATUS
- STATUS [filename]
- The syntax and meaning of this command is exactly the same as on M204. The difference is in the range of values reported. For each file that has been allocated during the run (including during the recovery stage if relevant) a list of the following values is reported:
- Allocated/Not allocated
- Open/Not open
- Updated/Not updated
- Updated since last checkpoint/Not updated since last checkpoint
- Recovered (if appropriate)
- Recovery failed and reason (if appropriate)
- In the case of recovery failure the reason will be one of the following:
- File could not be opened (e.g. dataset is missing)
- Recovery info is obsolete (e.g. file already "mended" by hand, perhaps using CREATE)
- File copy is too old (eg. file has been restored to an old copy by hand)
TABLEB
- TABLEB [options]
- The MASTER and OVERFLOW options are not allowed because DPT has no sorted files. All the other options should work as per Model 204.
UTABLE
- UTABLE {new parameter values}
- On DPT the UTABLE command is simply a synonym for RESET.
- Therefore there is no SIZE option like on M204.
USE
DPT supports USE PROC and USE {sequential file} correctly, but the nature of options such as PRINTER, $JOB and $PUNCH is intrinsically mainframey so there is much less support. The only commands of this type currently available on DPT are USE $PRINT and USE $EMAIL. Even with these the subobtions do not match M204's.
Depending on feedback this is something that can be enhanced in later versions. For example USE PRINTER {printer name} ... etc. might be handy where users have several printers, since use $PRINT selects the default Windows printer. More detailed control of colour and font etc. are also available via the Windows print API, if not on all printers, On that subject, options only take effect if the selected printer supports them.
- USE $PRINT [WITH] [options]...
- Where options can be as follows, in any order, with optional commas separating them. These options override the default ones set up for the printer (font, page layout etc.) and where you don't give an option, the default is used.
- In all cases the equals sign is optional.
- [ORIENT[ATION]] [=] {PORTRAIT | LANDSCAPE}
- The keyword ORIENTATION is optional, as the values are recognizable on their own.
- FONT [=] {font name}
- Font names with more than one word should be enclosed in quotes: FONT = "Times New Roman" otherwise DPT will not know where the next option begins. Usually the case of the font name does not matter, so you don't need to play around with *UPPER to type this command, and e.g. FONT = ARIAL should be fine. Nevertheless if you give an unusual font and it doesn't take effect, it's still worth trying it in mixed case just to be sure it's not supported.
- [SIZE] [=] {point size}
- The SIZE keyword is optional. Any numeric option is taken to be a font SIZE specification. For example USE $PRINT FONT JOKERMAN 14 BOLD.
- Note that it is your responsibility to ensure that UDDLPP and OUTMRL have appropriate values so that the paper is big enough to display everything in the font chosen.
- BOLD, ITALIC, UNDERLINE
- These 3 options are probably of minor usefulness, since you can only apply them to a report as a whole using this command, rather than specific characters or lines.
- USE $EMAIL [WITH] [options]...
- See examples in DEMOPROC.
- This mechanism is provided on DPT to be somewhat similar to the "USE PRINTER ... WRITER=SMTP" scheme used on Model 204.
- The body of the message will be made up of the next item of output, as per a regular USE command.
- The special USE command options SUBJECT, TO, FROM, OPTIONS and DIAGS can be given in any order. They match those on the User Language $EMAIL function and their meaning is discussed there, along with some more general notes on the email facility currently available on DPT.
- Blat *does* have a facility to insert custom SMTP header lines into emails, but it does not work the same as you may be familiar with from M204, and existing code of that kind will need alteration to work. DPT provides a workaround to keep things roughly familiar though: While the USE $EMAIL is active, starting any print line with "^USE^ " followed by the option as it would appear on the USE command causes that line *not* to appear in the message but set an override to whatever was given when issuing the USE command. You can set all the options this way if you like:
PRINT '^USE^ SUBJECT = ' WITH %SUBJECT
- Remember that an email is not actually sent until
the complete message text is ready, so in many cases if there are going to be problems with the combination of options, the error will not be flagged up until the "NUMBER OF UDD LINES OUTPUT" message is issued.
- As ever, if parameter values contain spaces, enclose them in quotes. Sometimes you may need quotes within the quotes. The inner quotes in that case should be double (DOS-style) quotes. For example
USE $EMAIL WITH OPTIONS = '-ATTACH "C:\\A PICTURE.JPG"'
- Note also we had to
use a double backslash there to keep Blat happy.
VIEW
- V[IEW] [parameter | parameter-set] [[,]...]
- The only real difference in this command is the range of parameters and parameter sets that can be used, and which parameters fall into which ranges. See the Master M204 X-reference tables for details on valid parameters.
- The valid parameter sets are ALL (the default if just VIEW is specified), ERRORS, FPARMS, TABLES, FILE (= FPARMS+TABLES), SYSTEM, USER and UNSUPPORTED.
- This this means that CWAIT, STORCUR, STORINIT, STORMAX and UTABLE are not supported.
- ALL does not include the "UNSUPPORTED" group.
- DPT maintains lookup table entries for the "UNSUPPORTED" group so that for example if you VIEW MAXBUF it can give you its lame excuse for not using that parameter, and in some cases suggest a similar custom alternative.
- This group is not comprehensive, since M204 is always adding new parameters.
VIEW COMMAND
- V[IEW] COMMAND {commandname | ALL | UNSUPPORTED} [ALIAS[ES]]
- This command has a couple of extra options over M204
- The ALL option lists all commands that there is a DPT equivalent for, (or a rough equivalent anyway). The UNSUPPORTED option lists those M204 commands that aren't supported at all.
- As with parameters (see above) the UNSUPPORTED group is not necessarily comprehensive.
- The ALIAS option shows any aliases, abbreviations or variations of the command. All the aliases grouped under the same "primary" form of the command will be affected by using RESET COMMAND on the primary form.
Z
Loads post-sort deferred intex updates for a file, as per the Model 204 command. The file whose indexes are to be loaded must be open, and should have been opened with a "basic" OPEN command, that is without specifying the TAPEx DD names. See the load job in the DPT download for an example.
The index load process performed by this command works by collecting deferred updates for each field/value into a bitmap in memory before storing them in a single trip to the index on disk. This contrasts with early DPT versions where repeated index retrievals and bit-flippings were performed. With most types of field this in-memory method has a dramatic effect on load times. It can be thought of as performing a small number of User Language "FILE RECORDS UNDER" statements rather than a large number of "ADD" statements, although that is not quite accurate as FILE RECORDS would replace any existing set, whereas Z augments it.
The improvement is only effective if the input file is well sorted, and DPT provides a command option which can help ensure that is the case.
See also the DBA guide for more about loading files,
- Z [forgivingness]
- The forgivingness option causes the load to fail if the input file is not "sufficiently" sorted.
- A value of -1 means infinitely forgiving and the input file can be in any order for the command still to work.
- A value of zero means totally unforgiving. Higher numerical values specify how many input records are allowed to be out of order before abandoning the load. So for example you may have an input file with a few awkward or missing values that your sort utility cannot completely handle, but that you still wanted loaded for some reason.
- The default is zero, enforcing complete sorting. That is the ideal position that you should start from, and then slacken off if necessary.
DPT Custom Commands
Many of these commands have been left in the system from when they were used during development and testing, like the custom options on actual M204 commands discussed above. They have been left in in case somebody finds them useful. The discussion (given below the table) is sketchy in some cases - further details are available if anything sounds interesting and you'd like to know more.
Most are prefixed with equals signs, to prevent conflict with existing APSY names ('=' can not be used in APSY names). It has been suggested that asterisk would have been a better choice and maybe that's true. In any case if M204 compatibility is an issue most of them can be disabled using the CSTFLAGS parameter.
List of custom commands (usage notes follow for many)
Command | Description |
=BUFFDUMP [from-page [to-page]] |
Shows assorted low level info about file pages currently in the buffer pool |
=BUFFTIDY |
Shorten the buffer pool LRU queue |
=CHKDUMP |
A listing of records currently in the checkpoint file |
CREATE SUBSYS {name} |
Required because no SUBSYSMGMT on DPT |
DELETE SUBSYS {name} |
See CREATE SUBSYS |
=DELG [pattern] |
See =SETG |
D[ISPLAY] AUDIT |
Scan the audit trail |
D[ISPLAY] SUBSYS [(option)] pattern |
Option is BRIEF or VERBOSE. Output is smart-edit-enabled. See also CREATE SUBSYS |
=ECB [option [new value]] |
Maintain "ECB" information |
=ECHO [any-text] |
Prints to the current output device. Default is USE=RESPECT. |
=EMAIL |
Sends an email. |
=FUNCLIST |
Lists the current set of installed $functions, with their return value and parameter types (see also $FUNCINFO) |
=ILINFO |
Scans table D to analyze inverted list (e.g. bitmap) population levels. |
=IOSTATS |
IODev7 message counts (the approximate reverse of the figs shown on the client) |
=LISTALC |
List allocated files, like the TSO command |
=LOAD |
Invoke a fast load. |
=MIME options |
Maintain web server related reference data. |
MONITOR RESOURCE |
Show information about internal resource locks |
=NEWSL activity |
Similar to the $SLSTATS function |
=PAI [BLOB-option] [n|ALL] |
A pre-written PAI loop |
=PAV field |
A configurable pre-written FRV loop |
=PSTxxxx |
Pseudo-user commands. |
=POST |
Wrapper for Windows PostMessage API. |
=RESxxx |
Manipulate internal resources and locks |
=SEQREAD dd [n|ALL] |
Prints the first 'n' records in allocated sequential file 'dd' |
=SETG gname [=] value |
Allows global manipulation from command level. Name and value may be quoted. See also =DELG |
=SHELL |
Start an OS process independent of DPT |
=SOCKET |
Start and stop the socket interface |
=SPAWN |
Start a daemon thread |
=SPAWNINFO |
Show info about a daemon thread |
=SPAWNPRINT |
Show daemon thread output |
SPY |
Start and stop the UL debugger |
=UNHALT |
Similar to M204 EOJ |
=UNLOAD |
Invoke a fast unload. |
=VSTAT |
View statistic values |
=Z1 |
Force flush of 1-step deferred update info. |
Notes about some custom commands
=BUFFTIDY
This command manually invokes the buffer tidy-up processing normally controlled by the custom parameter, BUFAGE. Most commonly it would be issued periodically by a system daemon (see =PSTTIDY).
- =BUFFTIDY [age]
- The parameter given simulates the BUFAGE parameter, and if omitted the value of the actual BUFAGE parameter is used. Note therefore that in the common case where BUFAGE is zero, meaning no pseudo-user is started, and you are using the command occasionally by hand, every deletable buffer page would get deleted by the command with no parameter. This is probably what you want - to free up the maximum amount of physical memory.
CREATE SUBSYS
Goes along with DELETE and DISPLAY. This command is provided to avoid the necessity of recoding SUBSYSMGMT, and also because there is no equivalent of CCASYS where subsystem definitions would be stored. An online would therefore have a number of CREATE SUBSYS commands in the User 0 input stream, much like people often do with file allocations, printer definitons etc., and which would recreate the subsystems anew as the online is started each time. The System Config Guide has more details about subsystem handling.
- CREATE SUBSYS subsysname
- ... parameters in any order ...
- END
- Parameters specified as PARAMETER name[=]value, as with CREATE FILE etc. They're not viewable parameters as such, which raises the question of whether the word PARAMETER is appropriate. It was decided to stick with it, as it's also used in the same way in CREATE GROUP (e.g. UPDTFILE is not a viewable parameter either).
- The command can be aborted with *CANCEL.
- The following parameters are COMPULSORY
- LOGPROC: the login procedure name
- COMMGLOB: communications global
- EXITVAL: exit value of COMMGLOB
- FILES: a comma separated string of file or group names, which will be opened, in order, when invoking the apsy, and closed when exiting. If there are temporary or permanent groups they will be opened in preference to files, as per normal OPEN and CLOSE commands (there are no special considerations for temporary groups). The default context, and the one from which all the procedures are expected to come, is the first one specified in this list (as per SUBSYSMGMT on M204), and therefore must name (at least) a procedure directory. All opens must be successful for the subsystem to be used. If any can't be opened, those already opened are closed, and the user is returned to command level. This is another way of saying that all files are "mandatory" to the subsystem, which also has the effect that CLOSE commands issued for any of these files from one of the subsystem procedures are disallowed. At subsystem exit, the files are closed in reverse order, even if they were open normally before the apsy was entered (this agrees with M204 behaviour).
- The following parameters are OPTIONAL
- ERRGLOB: error global name. There is no default.
- ERRPROC: the error procedure. This has one or two special considerations on DPT. The default is for there to be no error procedure, which means that situations where it would be invoked instead result in the user leaving the subsystem.
- CMDGLOB: command line global (no default - extra text after the subsystem name is ignored)
- MSGERR and MSGINFO: These cause the MSGCTL parameter to be set at the start of execution of the login proc, and to be set back to its previous value when the thread leaves the sybsystem. Specify Y to show messages and anything else non-blank to suppress them in both cases. (Although there's nothing to stop one of the procedures in the apsy setting MSGCTL to something else during processing, to turn messaging back on again). The defaults for these are both Y (i.e. all messaging is active, even if MSGCTL was set to turn it off before the apsy was invoked).
- MSGDISC: whether a message is issued saying that the user has left the subsystem. The default is Y.
- MAXITER: maximum iterations of the same proc in a row before it is considered an error situation, and the error proc included with error global value "INCLUDE MAX".
- AUTOCOMMIT: Whether the system should issue a DBMS transaction commit when chaining to the next procedure. It can be Y or N. The default is Y.
- LOGOUT: Whether the user is automatically logged out of DPT after they leave the subsystem. Does not prevent XFER to another subsystem. Often used in conjunction with AUTOSYS. See also related options on the $SUBSYS function. It can be Y or N. The default is N.
DISPLAY AUDIT
Issuing this command causes DPT to scan through the current audit file, printing lines to the user's terminal (or USE destination). All lines can be shown, or more often lines would be excluded by giving one or more selection criteria.
This command is particularly useful when DPT is running in NOGUI mode, and/or the user is working on a remote machine from the host. In the first case the simple alternative is notepad, and in the second case there may be no simple alternative (short of writing your own audit file scan program using e.g. $FREAD and $TOKENIZE).
- D[ISPLAY] AUDIT [FROM[=]timevalue] [TO[=]timevalue]] [LIKE[=]pattern [CASE]] [MAX[LINES][=]n]
- All parameters are optional. See below for parameter format examples.
- Time and pattern selection criteria are "AND"ed together, so lines must pass all given tests to make it to the final report.
- Time selection:
- If no FROM is given, the start of the file is assumed. If no TO is given, the current end of the file is assumed.
- FROM and TO can be given as a minus sign followed by some number of seconds, meaning relative to the current time.
- Alternatively you can give absolute time values, optionally including the date too. Each date+time value must be a single string containing no spaces, where the date and time components are as they appear in the audit trail. So that's YYYY/MM/DD for dates and HH:MM:SS for times. The date and time should either be concatenated together with a non-blank character, or if a blank is used, enclose the pair in quotes.
- Shorter time specifications can also be given. Omitting the date means the current date is assumed. Leaving the MM and/or the SS off the time means 00 is assumed for either or both. So the shortest possible time specification is just HH for HH-o'clock today.
- Text pattern selection:
- If no LIKE pattern is given, every line is selected.
- The pattern is a normal M204-style pattern and is assumed to have a leading and trailing wildcard, so you don't have to specify those.
- Audit trail line comparisons are not case-sensitive, unless the CASE option is given, when they are. (Entering mixed-case patterns would of course also require a prior *LOWER command unless the DISPLAY AUDIT command was in a procedure).
- Patterns can be quoted. They must be quoted if they contain spaces or equals signs.
- Other notes:
- MAXLINES stops the scan when a certain number of lines have been selected. (Default = 1000, -1 = no limit.)
- If the audit file might be large it's best to give at least a FROM time, and even better a TO time. This gives DPT a chance to avoid scanning the whole file.
- Large reports in any case run more quickly to USE destinations, for example USE PROC and then edit the proc.
- Line selection from the audit file is dependent on the relevant information actually being present in the file, as partially controlled by the AUDCTL parameter. Specifically for example the scan on user number shown below will only work if the X'08' bit is set. In the case of date and time values, the command will tell you if it can't perform the scan, as will be the case for date-specific scans with the default audit trail layout, which does not show the date on every line (X'01' bit). In that situation scans with no explicit date selection component do work, but they assume the file all relates to the same day. For 24/7 DPT host runs, make sure to turn on AUDTCL=X'01' if you anticipate extensive use of this command.
- The millisecond component of the audit trail time, if shown, is ignored.
Examples:
D AUDIT /? Show first 1000 lines ?/
D AUDIT MAXLINES -1 /? Show whole file ?/
D AUDIT FROM -300 /? The last 5 minutes ?/
D AUDIT LIKE ' 05 ' /? Activity for user 5 ?/
D AUDIT LIKE ERR FROM 9 TO 10:25 /? Errors today from 9am to 10:25am ?/
D AUDIT FROM 2010/09/10/18:37:12 /? Since a very specific time ?/
=ECB
This is just a command line way of maintaining the same information as the thread synchronization $functions. There are three items of information associated with each ECB, viz a flag (denotes posted/unposted), a numeric value (the post code) and a textual field (can be used for any purpose).
- =ECB [LIST]
- =ECB DISPLAY ecbname
- =ECB {TEXT | NUM | FLAG} ecbname new-value
- Plain =ECB prints a tabular listing showing all info held for all ECBs.
- =ECB DISPLAY ecbname displays the same information for just the requested ECB.
- The third form of the command changes the specified item to a new value, and prints the result (there is an implied DISPLAY).
- Text can be anything. Num must be an integer from 0 to 16777215 (as per $POST). Flag can be Y/N/POSTED/UNPOSTED/NOT POSTED.
=EMAIL
Hooks into the same underlying send mechanism as the User Language $EMAIL function and the USE $EMAIL command. See notes there for more details and background.
Consider using this command for short ad-hoc messages entered by hand, like where you might use the MSG command on Model 204.
- =EMAIL [options...] message
- The options are exactly the same as the USE $EMAIL command.
- Any text after the options is taken to be the message, unless you use the keyword "MESSAGE", in which case it is treated just like another option and can be in any order. So the following two commands do the same thing:
=EMAIL MESSAGE HELLO TO THEQUEEN@PALACE.GOV
=EMAIL TO THEQUEEN@PALACE.GOV HELLO
- Use pipe ("|") characters in the message to specify line breaks:
=EMAIL TO ANATOLY@GMAIL.COM SUBJECT 'MY MOVE' P-K4|HOW IS THE WEATHER IN MOSCOW?
=LISTALC
Like the TSO command. Shows all the OS files that are allocated to the system, of all types or a selected type.
- =LISTALC [ALL | DB | SYS | PROC | SEQ | PROCDIR]
- Entries shown for PROC refer to the actual procedures themselves, not directories. PROCDIR vice versa.
- This command is how the navigator populates its lists of proc directories and data files.
=LOAD
This command efficiently loads field descriptions and/or data and/or index information into a DPT file, so long as that information is presented in one of the acceptable formats. Typically that means that it was either generated during an earlier unload of the same file during a reorg, or from a Model 204 file on the mainframe.
- =LOAD [ALL INFORMATION] [(options)]
- The keywords ALL INFORMATION are completely cosmetic, and optional.
- The kind of information loaded by the command (field defs, data, indexes) is determined by what input files are present in the #FASTIO directory (or other specified directory - see options below). All input files are optional.
- The format of the incoming data is specified by options in the metadata header of input files themselves, and not specified on this command. The =UNLOAD command notes describe those formatting options.
- Bracketed load options:
- Options can be separated with spaces and/or commas.
- PREVIEW: Tells the command not to make any updates to the database file, but read through the input files and print them to the terminal (or USE destination) for the user to have a look at. This is handy because of the various ways in which the actual file contents might not be easily readable using standard file browser tools, especially if they're in EBCDIC. Numerical values are printed out as literals and EBCDIC translation is applied if appropriate.
The optional subparameter says how many entries to print (fields/records/values/inverted lists). Zero (the default) means preview mode is not active, and the file is loaded. Just "PREVIEW" entered with no parameter is taken to mean "PREVIEW=10", since often you just want to take a glance. Any negative number or "ALL" means show everything (best if sent to a USE file or proc).
- DIR: An override for the directory which will searched for the input files. e.g. DIR=SALESDATA, DIR="C:\TEMP", etc. The default is #FASTIO in the host working directory. Either single or double quotes should be used to surround the name if it contains spaces. There is no way to change the expected file names.
- CLEANAFTER: Deletes the input files at the end if there were no errors. This can be convenient in routine situations.
- NOACTION: Locates all eligible input files, but neither loads nor previews the contents.
- CLEANONLY: The same as (NOACTION, CLEANAFTER).
- Other comments:
- The context in which the command is issued must not be a group. Exclusive access is required to the file.
- The process is a single, non-backoutable, bumpable, update unit. Any error (or bump) leaves the file physically inconsistent.
- Minimal diagnostics are issued by default, but increasing the LOADDIAG to a higher value will probably be useful the first few times you use fast load.
- Indexes are built under the covers using the same fast index-build code used in the single-step deferred update process, and many of the same points regarding memory chunks etc. apply. Likewise some of the internal processing can be affected by the LOADxxxx group of parameters.
- Depending on the FMODLDPT parameter setting the =LOAD command might fail from data conversion errors if field definitions in the incoming data differ from those in the file.
Brief load processing overview:
Stage 1. Preparation
- Search for and open "file_TAPE*.DAT" input files.
- If index information is in a single file, this file is split into separate files, one per indexed field, in the same directory.
Stage 2. Loading field information:
- When field definition processing is performed, any field definitions present in the input file and not present in the database file are added. The simplest thing is often to issue an INITIALIZE command before the load and let the fields be defined again from the input file.
- Any existing definitions are left as they are. That is to say, the =LOAD command does not itself ever perform an implied REDEFINE.
- So if you unload a file, initialize it, then manually DEFINE one or more fields before reloading, they will end up with the manually-added definitions rather than those they had before the unload (see reorg recipes).
- If field codes are supplied, DPT makes a note of the actual codes that fields end up with after they're defined, so that if the record input stage uses field codes, the actual new ones can be substituted in.
Stage 3. Loading record data:
- Input records are read and stored in table B.
- If there were existing records in the file, the new records receive record numbers ascending from the current highest in the file, according to BRECPPG at load time.
- During a reorg if a record had extensions before the unload, they are consolidated at reload time down to a
minimum number, ideally of course a single extent.
- If TAPED contains field codes instead of names, the cross-reference table built in the field stage is used to convert to the correct ones.
- If the file has indexed fields and no index input file is present for any of those fields, the data phase builds index entries as if the file were in single-step deferred update mode. There is no need to OPEN the file into DU mode beforehand: the =LOAD command does that automatically.
- If index input files *are* present for the indexed fields, DPT assumes they will contain correct index information corresponding to the data information in the data input file, and it therefore does not generate indexes from the data during this phase. This applies even if the index input file(s) later turn out to be empty. DPT is willing to trust you here - see comment below about this feature.
- If index input files are present, DPT maintains a log during the data load phase, cross-referencing the record numbers in the input versus those new ones it stores. This allows the index information to be adjusted in the next phase to point to the correct records. This is the only reason the input TAPED file has to contain the old record numbers - loaded records do *not* keep their old numbers.
Stage 4. Loading indexes:
- B-tree entries and inverted lists are stored afresh or added to those existing in the file as appropriate.
- B-trees are built as per SPLITPCT at load time, and inverted lists are stored with the appropriate level of compression taking into account which records now possess each value.
- If there was a data phase in the same overall load operation, the record numbers in incoming inverted lists are changed to the correct new ones as stored. (A sanity check is performed - see
LOADCTL X'02').
- Starting with empty indexes is much faster, unless the incoming value entries are all new and located at the high or low end of the b-tree, in which case no b-tree node splits or inverted list modifications are required.
Stage 5. Completion:
- Regardless of which combination of field/data/index input files is processed, at the end of this command the database file is set to FISTAT=0 to indicate that as far as DPT is concerned it is internally consistent.
- Note: This applies regardless of whether the input files are logically consistent with each other, although DPT does issue warnings in some more obvious cases. This freedom of operation gives a database adminstrator some opportunities for creativity in performing certain file maintenance operations, for example on very large files or where unusual data is involved, although generally speaking such sleight-of-hand should not be necessary.
Examples:
=LOAD
IN SALES =LOAD ALL INFORMATION
=LOAD (PREVIEW)
=LOAD (DIR = "Fastload Inputs\Sales\June 2010", CLEANAFTER)
=MIME
This command operates on the internal cross-reference table of file extensions and MIME content types. MIME is the standard scheme for describing the type of data blocks transferred over the internet in various situations. In theory the MIME types are unrelated to file extensions, but often they are simply more elaborate forms of the file extensions. For example in the case of ".gif" the MIME standard type is "image/gif". Likewise ".html" = "text/html" etc. The logic here, assuming there is any, is beyond the scope of this discussion.
DPT maintains a lookup table keyed by extension, with each entry corresponding to a MIME type. This both enables the system to serve fully-described static resources like images and applets, as well as allowing the User Language programmer to use the extension as shorthand in code when specifying a MIME type.
When the system starts, the table is empty, so DPT would be leaving everything to the browser. However, you may find it useful to set up some standard types, if only to be able to use the UL shorthand when creating dynamic content that isn't HTML (the default). The web server daemon initialization procedure is a good place to place such commands (as on the demo system).
MONITOR RESOURCE
Displays a list of the resource control objects and details of who is currently holding them and waiting for them. See also =RESxxx. Users are identified by their OS thread ID. Consult the main MONITOR display to cross-reference this to a user name or M204-style user number if required.
- MONITOR RESOURCE [LIKE pattern]
- The "MONITOR CFR [LIKE cfrpatt]" command on DPT is simply shorthand for typing "MONITOR RESOURCE LIKE CFR_cfrpatt". In other words it just generates a subset of the full MONITOR RESOURCE display.
- The same goes for MONITOR SUBSYS, where any LIKE pattern given is appended to "SUBSYS_", in the same way.
- In both cases if no pattern is given, '*' is assumed - namely all resources of that type.
=PAI
Quicker than typing B;FR;PAI;END all the time, and provides more information. This command can also be issued from the DPT client navigator.
- =PAI [LOB-option] [count|ALL]
- The LOB options are the same as on the UL statement.
- The only difference in processing from the statement is that the command does not bother trying to avoid table E disk reads for BLOB fields where LOB_NONE or LOB_NO_DATA are used.
Hence a little less efficient in those cases.
=PAV
This is a partner command to =PAI. Both are provided just to save typing the same old 3 line requests over and over again. Both can be issued after right clicking on the navigator.
- =PAV [COUNT | RECNUMS] field [([DESC[ENDING]],[STOP[AFTER][=]n])]
- The basic command "=PAV field" just lists the values from the ordered index for the field.
- =PAV COUNT additionally counts the inverted list associated with each value and prints the count.
- =PAV RECNUMS not only counts the inverted list but prints out all the record numbers in it too.
- The bracketed option DESC[ENDING] means start at the high-value end of the ordered index and work back.
- The bracketed option STOP[AFTER] means only process n values of the field.
=PSTCHECK or =PSTTIDY
These two commands are what the checkpointing and buffer tidying pseudo-users (daemon threads) issue when they log on. An infinite loop is set up whereby the issuing thread periodically wakes up to perform some work. =PSTCHECK wakes up every CPTIME seconds and issues a CHECKPOINT command. =PSTTIDY wakes up every BUFAGE seconds and issues a =BUFFTIDY command.
Normal users can issue these PSTxxx commands if they really think there's a good reason to do so. Normally there would not be any good reason.
=PSTFUN
This command turns the calling thread into a $function-support daemon, most likely for handling one of the emulated emulated Sirius $functions but potentially also any other DPT custom $functions requiring background processing. Normally a daemon thread would issue this command, but terminal users can do so as well if they like.
- =PSTFUN [number]
- The single optional parameter is for use when a terminal user wants to become a $function-support daemon temporarily, and indicates the number of $functions to handle before returning to the chevron.
- In most aspects this command and its parameter are handled the same as =PSTWEB - see below for notes. The relevant control parameter in this case is FUNNDAEM instead of WEBNDAEM.
=PSTWEB
This command turns the calling thread into a web server daemon. Normally a daemon thread would issue this command, but terminal users might often do so for debugging purposes.
- =PSTWEB [number]
- The single parameter is for use when a terminal user wants to simulate the action of a server daemon, usually in order to debug a handler procedure in an "as-live" situation.
- The specified number of HTTP request/response exchanges are performed before the thread returns to its previous non-server-daemon state.
- If a number less than or equal to zero is specified, the thread will enter an infinite loop which can only be terminated by bumping it, or resetting WEBNDAEM to a lower value.
- When issued from an interactive thread (IODev7 or 15) it is assumed that temporary daemon-ness is required, and a parameter value of 1 is the default, causing a single HTTP request to get handled. Otherwise the default is zero.
- Such "simulated" daemons ignore the WEBNDAEM setting as they wait for incoming HTTP requests. In other words you can set WEBNDAEM to zero and all the "real" web daemons will log off, thus ensuring any incoming work gets picked up by your terminal session.
- Both dynamic and static HTTP requests count towards the 'number' value given. Therefore if you are debugging, say a page of HTML which contains 3 <IMG SRC=....> tags, you would need to say "=PSTWEB 4" to serve the entire page.
=POST
A straightforward wrapper for the Windows API PostMessage() function. This command can be used for GUI manipulation, or any other general application communication purposes. PostMessage() is an asynchronous API, so each call is guaranteed to complete. In other words unlike SendMessage() which is synchronous and might hang. There is no DPT wrapper command for SendMessage().
- =POST hwnd [,] msg [,] [wparam [[,] lparam]]
- 4 numeric parameters, separated by spaces or commas, of which only 2 are required. Omitted wparam or lparam are taken to be zero.
- All parameters can be given in decimal or "0x...." hex form.
- hwnd can optionally be given as CONSOLE, which means the console window of the current DPT host. For example one usage of this command is to minimize or otherwise move the console around: =POST CONSOLE 0x0112 0xF020 translates to PostMessage(console-window, WM_SYSCOMMAND, SC_MINIMIZE, 0).
- See Windows documentation for full details.
=RESxxx
This is a set of commands for manipulating the resource control objects that DPT uses in various situations where something is shared between users. They can be displayed using MONITOR RESOURCE (see above), and manipulated using =RESDEF (define), =RESDEL (delete), =RESFIX (kill all locks), =RESGET (acquire with wait), =RESINFO (display), =RESTRY (make 1 attempt to acquire), and =RESREL (release). Messing with the system-defined resources too much is probably not advisable, but you may find use for some of these, perhaps with custom $functions. The full syntax is omitted here to save bandwidth, but is available if anybody wants it.
=SHELL
This command kicks off an OS process completely independent of the DPT host system. The new process can of course subsequently connect to or otherwise interact with DPT or its client applications. =SHELL is always asynchronous, in other words it does not wait for the new process to complete.
See also =SPAWN.
- =SHELL [(verb)] application-name-and-or-command-line
- The value of 'verb' can be one of the operations you typically see on the "right-click" menus in Windows Explorer. The allowable range therefore depends on the file associations defined on your machine, and the type of file specified on the command.
- For example OPEN (supported by most entity types), PRINT (usually only documents), and EXPLORE (directories).
- A special verb, and the default, is APP, which means that the command line is interpreted as an application name followed by an optional set of arguments to be passed to it. For example:
=SHELL NOTEPAD MYPROC.TXT
=SHELL (APP) WINWORD.EXE REPORT.DOC
- Depending on your current OS file associations the above calls might do the same thing as =SHELL (OPEN) ...etc.
- Another special verb is CMD (or synonym DOS) which means the entire line is just passed through to the DOS command interpreter CMD.EXE, and is a tidier way to invoke DOS commands. For example:
=SHELL (CMD) MKDIR TESTRUNS
=SHELL (DOS) DEL TESTRUNS/TEMP*.TXT
- This option is processed by invoking CMD.EXE with the "/c" switch. If you need finer control over the switches passed to CMD.EXE, just invoke it explicitly as an application. For example:
=SHELL (DOS) SOMEJOB.BAT /? might be too quick to see ?/
=SHELL CMD.EXE /T:0A /K SOMEJOB.BAT /? review results when finished ?/
- In some cases where you might think of using a DOS command, the $FILE_xxx and $xxDIR set of UL functions offer more control over what happens and give better diagnostics.
- =SHELL can be used to simulate USE $JOB, by first writing the DOS batch commands to a sequential file with extension ".bat" and then invoking it for example as above.
- There is no direct $function equivalent to the =SHELL command, but it is easy to perform effectively the same thing, using either $SPAWN or $COMMBG and then issuing the =SHELL command from the daemon.
=SOCKET
This command starts and stops the host application sub-threads which listen for incoming socket-based connections. There are two such "listener" threads working on two different TCP/IP port numbers, one accepting terminal connections (IODevs 7 and 15) and one accepting HTTP (web) requests. The main reason for using this command during a run would be to change the port numbers without having to bump everyone off.
If the listener threads crash it may also be possible to use the SOCKET command to restart them without bouncing the whole system.
- =SOCKET {START | STOP | STATUS} [TERM[INAL] | WEB]
- The optional parameter can be used to direct the command at either of the two listeners.
- Note that this command has no effect on User Language applications listening using the $SOCKET function, although you can still BUMP such threads.
=SPAWN
This kicks of an IODev99 daemon thread to perform background processing within the DPT host system. Except for the parameter order it is processed the same as the $SPAWN custom UL function. Each call to =SPAWN starts a new daemon user which runs, perhaps briefly, until its input is processed.
Daemon threads are operationally very similar to IODev3 threads, and allow all the same commands, parameter settings etc.
Note the difference from =SHELL which kicks off an OS process outside the host, and also the difference from UL $functions which make brief use of existing support daemon threads rather than spawning fresh ones.
- =SPAWN [ASYNCH | SYNCH] {IDstring | *} commandline
- The SYNCH/ASYNCH option, if given, must be the first word after the command name.
- The IDstring can be any string up to 5 characters long. If an asterisk is given the system generates a 5 digit value starting at 00001. The spawned thread logs on with a user name of "Daemon_xxxxx" where xxxxx is the ID string.
- The input for the spawned thread is limited when you use the =SPAWN command to a single physical line (but see $SPAWN which can take a $LIST). The single line can consist of a number of logical lines (e.g. B;PRINT 'HELLO';END) which is often a decent substitute for a real multi-line capability. However, since semicolons are interpreted as newlines on the issuing user's device, it is necessary to use an alternative convention to pass them to the daemon if required. The convention used by the BROADCAST command is followed, namely that you should specify slashes instead of semicolons. To actually include a slash, use "!/". For example =SPAWN * B/PRINT 'The new millennium began on 01!/01!/2001'/END.
- More usually a daemon would open a directory and include a procedure, in which case the above issue is not a problem. In any case in version 2 a call to the $SPAWN function giving multiple "real" lines in a $list is possible, and probably more familiar to M204 users.
- There is no need to include a LOGOFF command - this is assumed, unlike on an IODev3.
- The output device of the daemon thread is a sequential file of the form IDstring.txt which is stored in the #DAEMON directory. If the file already exists, it is overwritten.
- In synchronous mode, the issuing thread enters a DPT custom wait state of WT=96 until the daemon finishes.
- Also in synchronous mode the return code of the =SPAWN command is set to the highest return code encountered by the daemon thread, so the "parent" thread can be considered to have had the error too. This is relevant in various situations, such as if a SETGRC command comes next.
- Creation of the spawned thread goes hand in hand with the creation of a system resource (see also =RESxxx) named SPAWN_xxxxx, where xxxxx is the ID string. This resource is automatically acquired by the spawned thread when it starts, and released when it terminates. This is most relevant in asynchronous mode, where for example the calling thread could spawn many background tasks and then wait for them to finish by using some combination of MONITOR RESOURCE, =RESTRY, =RESGET, =RESINFO, $WAIT, $ECBDGET or =SPAWNINFO.
- If the specified ID string corresponds to a still-running daemon thread (i.e. a resource exists and is still enqueued) the =SPAWN command fails. If the ID string has been used before, but the daemon has finished (i.e. a resource exists but it is no longer enqueued) the resource is re-used and the output file overwritten.
- If a thread issues =SPAWN ASYNCH..., closely followed by one of the monitoring commands below, the daemon can fail to start because the spawning thread will be examining the resource. In a programmatic setting, you would usually use synchronous spawn.
=SPAWNINFO
Shows summary information about a daemon thread, such as the input line, output file name, and details of
the most serious error message issued by the daemon.
- =SPAWNINFO [IDstring]
- The ID string defaults to the last one generated by the system (see =SPAWN), which means it would usually not be useful to accept the default if several threads had been spawned.
- If the specified daemon thread is still active, an error message is issued.
=SPAWNPRINT
Reads and prints the contents of a daemon thread's output file.
- =SPAWNPRINT [IDstring]
- The ID string defaults to the last one generated by the system (see =SPAWN), which means it would usually not be useful to accept the default if several threads had been spawned.
- If the specified daemon thread is still active, an error message is issued.
- The corresponding resource is enqueued whilst the file is being read, so that a new daemon thread with the same ID can not be started until =SPAWNPRINT has finished. This is more relevant to the $function context, where the request might wait for unpredictable lengths of time between reading each line.
SPY
This command affects the run-time control users have over User Language programs. NB. No equals sign because presumably people are familiar with SPY to invoke SoftSpy.
Generally you would not need to issue this command by hand, as Ctrl+F1 in the GUI toggles the debugger on and off. The following notes however may be useful to explain the meaning of some of the GUI debugger options.
- SPY [[ON | OFF | LINE] | [spymode] [rttrace] [spyflags]]
- SPY sets the values of the RTTRACE, SPYFLAGS and SPYMODE parameters, and is the only way of affecting those parameters. Most commonly the values for these parameters would be passed as client preferences rather than typing them by hand.
- SPY ON is shorthand for setting the default values for the thread type the user is currently running on, as covered below. SPY is shorthand for SPY ON.
- SPY OFF is shorthand for setting all three parameters to zero.
- SPY LINE can be used on IODev7 to set SPYMODE to 2 instead of the default 3 if you want to use line mode for some reason.
- If ON or OFF is not specified, one, two or all three of the parameters can be specified, in the above order, with missing values taking their defaults.
- SPY ON and SPY OFF are all you will usually need, but details of other possibilities are given in the IDE user's guide.
=UNHALT
This command is a cross between Model 204's EOJ, and the concept of replying to the HALT message on the MVS operator's console. It tells user 0 to wake up and proceed with its processing. This will often mean the system comes down, unless there is a significant number of extra commands after the HALT, or even another HALT. The System Config Guide gives full details on starting and stopping the system.
- =UNHALT
- When issued from a user thread, this command, like M204's EOJ, initiates a "Do you really want to" exchange.
=UNLOAD
Efficiently extracts all or part of the information in a file. The results might be the halfway stage of a reorg, or be used used for any other purpose such as loading into another DPT file or passing to an external system. By default the command processes all records in the file and/or index values for all records, but if you want to specify a subset you can use the partner $UNLOAD User Language function, or the GUI interface in the File Wizard utility.
The output of the processing is information in the standard layouts as used by various internal operations as well as user-invoked functions like REORGANIZE and =LOAD. If extract data is required in other significantly different formats, apart from the simple options shown below, you will have to either fall back on a UL/image program, or start with =UNLOAD and then reformat the output using external tools of your choice. Put another way, DPT has no equivalent of the complex Sirius 'FUEL'.
The ordering of information extracted is as it is in the database: data records in ascending record number order, and index values in whichever collating order the field is defined as (string or numeric). There is currently no way to get the =UNLOAD command to change these extract orderings.
- =UNLOAD [what] [[EXCLUDING|FOR] [FIELD[S]] field [,field...]] [(options)]
- The 'what' clause indicates which parts of the file to extract. Specify it as a series of keywords, optionally separated by commas, spaces or the word "AND". The keywords are FDEFS, DATA, INDEX[ES], and ALL which means all 3 and is the default. ALL INFORMATION is the same as ALL.
- You can unload information for a subset of fields by giving the desired list of field names separated by commas. The list of fields must be preceded by at least one of the introductory keywords above. The default (FOR) includes only the specified fields. The alternative (EXCLUDING) will unload all fields except the ones specified. Fields not participating in the unload get no information put into any of the output files.
- The bracketed options can appear anywhere in the command and affect the formatting/encoding of the output, as follows. Multiple options should be separated with spaces and/or commas. All options are off by default.
- PAI:
Affects TAPED and TAPEI files. Makes the contents of these files human-readable, like the output of simple User Language programs, as covered here (D) and here (I). This format is acceptable to the fast load process, and in many cases can be a more convenient format for both unload and load, particularly loading data extracted from Model 204 if the highest performance is not required.
- EBCDIC: Translate all string values into EBCDIC. Use this option if the extract is going to the mainframe. Note that when the extract contains binary data like field IDs or bitmaps it isn't possible to leave this issue to the file-transfer engine (e.g. FTP), since that would corrupt the binary data. PAI-mode jobs can either use this option or let FTP do it.
- FNAMES: Populate the TAPED file with field names instead of codes. The lookups required to do this mean the unload is slightly less efficient, plus the data extract file is almost certainly larger. This option is always effectively turned on in PAI mode.
- NOFLOAT: Extract numeric field values as strings rather than the 8 byte IEEE 754 FP bit patterns that are stored in the DPT file. This adds some conversion overhead, but the effect on extract size may be positive or negative depending on the data. Remember that Model 204 on the mainframe by default does not understand the IEEE 754 format. This option is always effectively turned on in PAI mode.
- ENDIAN: Causes binary items in the output to be written in reverse-endian order. Affects all integer numbers such as record numbers, other assorted integer control fields, plus floating point values too if included. If byte ordering is treated differently on the target machine for integer and floating point, use IENDIAN and FENDIAN separately to achieve the desired combination. This option is irrelevant in PAI mode.
- CRLF:
Affects TAPED and TAPEI files. Appends an extra CRLF byte pair after each record (TAPED) or each segment inverted list (TAPEI). This can make unloaded data more palatable to e.g. sort utilities later, and likewise can also make it easier to create custom load input from external sources.
- DIR: An override for the directory which will receive the output files. e.g. DIR=SALESDATA, DIR="C:\TEMP", etc. The default is #FASTIO in the host working directory. This directory is created if it doesn't exist already. Either single or double quotes should be used to surround the name if it contains spaces.
- REPLACE:
Since you will usually want to do something with the extract files before creating more of them, the command by default will not overwrite existing output files if they exist. This is intended to prevent earlier extract processing being wasted if you just forget to pass the extracts on to whatever their next destination is. Use the REPLACE option to disable this check. See also the related CLEANxx options on =LOAD.
- Other comments:
- The context in which the command is issued must not be a group.
- File updates are blocked on other user threads for the duration of the unload. In fact the =UNLOAD command itself may fail with record locking conflict if other users have in-flight transactions.
- The processing is bumpable.
- Minimal diagnostics are issued by default.
- DPT can use parallel-processing during unloads. See the LOADTHRD parameter notes for more details.
- By default, index entries which point to nonexistent records *are* unloaded, but see LOADCTL X'04'.
Examples:
=UNLOAD
IN SALES =UNLOAD ALL INFORMATION
=UNLOAD FDEFS (FNAMES DIR='C:\Mystats')
=UNLOAD INDEXES FOR FIELDS SALES_Q1, SALES_Q2
=UNLOAD (REPLACE, CRLF) ALL
=UNLOAD FDEFS AND DATA (EBCDIC FNAMES) EXCLUDING AGENT.COMMISSION
=VSTAT
Allows individual stat values to be examined without issuing a MONITOR command. However, it differs in processing from MONITOR in a few other ways. Firstly, zero-value statistics are shown. Secondly, file level statistics can be shown. Thirdly, statistics for other users can *not* be shown. The UL $STAT and $VIEW functions hook into the same low-level functionality as =VSTAT.
- =VSTAT [ALL | statname} [ALL | SL | USER | FILE | SYSTEM]
- The stat name and level, if given, must be specified in this order, i.e. name followed by level.
- If a named stat is not collected at a named level, an error message results, as it does with $VIEW and $STAT (which also return a null string).
- To view stats at FILE level, the command must be issued in some file context.
- Both "ALL" options will show all stats and/or levels that are collected. For example =VSTAT DKRD ALL would show 4 figures, and =VSTAT ALL FILE would show all the file statistics. The master M204 X-reference document gives details on which stats are collected at which levels.
=Z1
This command is somewhat related to the regular Z command, but it applies to the single step deferred update process. The function of the command is to trigger the same processing that usually happens when DPT detects a low-memory condition, namely to offload deferred update information to sequential files and free up the memory they were using. Table D is not written - that only happens when the last user closes the file, regardless of any =Z1 commands issued beforehand.
The main time this command would be useful is if one file is in DU mode and hogging a lot of memory with its deferred updates but neither getting closed nor receiving any more updates to trigger the offload. To other actively-updating files in DU mode it would then appear as if available memory was low, and they would keep flushing far too soon and spoiling some of the benefit of their DU mode. If it's not possible to get all users to close the first file, issue the =Z1 command on it.
- =Z1
- The file context must be a non-group that is currently in single-step deferred udpate mode.
- The file *remains* in single-step deferred update mode after this command.
Part 2: User Language
General UL Compatibility and Processing Notes
Where possible every effort has been made to get User Language working the same as Model 204. Otherwise (usually only obscure situations) an educated guess has been made with the intention of correcting it as soon as possible later on. Usually the decicision taken was to err on the side of leniency if the alternative was possibly making the compiler too strict, since this offers the best chance for existing code to run successfully.
In some cases there are small intentional differences for the various reasons mentioned in the emulation strategy document. More details on some of these, plus other issues of general interest, are given below.
Model 204 statements that aren't supported are discussed in the compile-time messaging section below.
Version compatibility and obsolete M204 features
Statement numbers are not allowed, regardless of the FOPT parameter setting.
Unbalanced brackets may not be used to continue lines. Since there is no INCCC parameter, continuation can not be implied with a non-blank character in this column. Hyphens, unfinished quoted strings and unfinished block comments are the only way to continue lines. Note that the FIND expression
FD
(AGE > 18
SEX = M) OR -
RANK = CORPORAL
END FIND
is not using the unbalanced brackets to continue a line. The newline is real, and means an implied AND, which because of the brackets will be given higher priority than the explicit OR, rather than the usual lower priority.
The "pre V2.1" issues mentioned in the UL manual appendix are taken into account, so that the "previously-allowed" statement forms are not allowed on DPT.
Number handling
Floating point
This is a complex subject, but one thing we can say is that completely emulating Model 204's (and by implication MVS's) handling of floating point numbers is outside the scope of what we are trying to do. DPT's floating point handling is based on the IEEE 754 standard functionality commonly supported by compilers and co-processor chips. This standard is available on S/390 nowadays but M204 (being an old application) almost certainly uses the earlier "hexadecimal floating point" (HFP) format. This means the underlying notions of accuracy and precision are going to be subtly different between the two platforms.
The question is, how far should we go in making things work like they do on M204? The happy answer is that in many cases things should work the same because of the UL convention of *rounding* to 15 significant decimal digits for many operations. DPT follows this convention, as described in some detail in the UL manual, and since both IEEE 754 and long HFP are guaranteed to convert reversibly at this precision, any differences get glossed over when these roundings occur.
Rounding occurs in nearly all situations, including but not limited to: Storing in FLOAT variables, in the PRINT statement for any numeric value, storing in both table B and table D for FLOAT/ORD NUM fields, intermediate expression results after addition/subtraction and all situations where strings are used as numbers (e.g. in the compiler and when STRING variables are used in arithmetic). Rounding is skipped in very few places, namely after multiplication/division and before/after $function calls, since in these cases the extra precision might often be useful. This should be close to how Model 204 behaves, but may not be exact (interestingly the UL manuals themselves aren't completely consistent on what they say). Further refinements can be made in future releases if required.
There may be subtle differences in what is considered a valid numeric literal when exponents are used (e.g. 1.5E3 / 1.5 E 3 / 1.5E+3 etc).
This topic is also relevant to the math $functions, where the potentially different expansions used by Microsoft C and the mainframe libraries could conceivably add further rounding differences in the small digits. It's also relevant to the bit string generated by $FLOAT.
Fixed point arithmetic in UL
When doing arithmetic with FIXED and STRING variables (and image items) that have DP specifications, DPT should behave the same as M204, as per the rules set out in the UL manual 10-23...25, since in this case the behaviour is based on truncation at the point specified by the DP specification, which maxes out at 9. DPT obeys all the rules concerning scale of intermediate and final results, except one. It does not look ahead to see if the next operation is a subtraction (which in any case as the manual says ends up giving an incorrect result).
Pattern matching
DPT's pattern matcher handles all the things the Model 204 one does, including the M204 Version 5 relaxation on the rules about whether you need the escape character before special characters (it is optional). In certain Sirius functions the slightly different Sirius format is recognized too.
However, without knowing how the M204 matcher works it's likely that highly complex patterns will work slower on DPT, but they might be faster, you never know! In particular the common cases of patterns with a leading and/or trailing wildcard and no other specifications such as '*ABC', 'ABC*' or '*ABC*' all work very quickly, being internally translated into the equivalent of $SCAN.
Context specifications
IN clauses on DPT can not include the "AT location" phrase, since there are no scattered groups.
Unlike M204, global foundsets and positions can be created in ad-hoc group context on DPT, but they are deleted at end-of-request, and so do not realistically give rise to a situation where an application will work on DPT but not on Model 204. Ultimately the same result is achieved and it was much easier to implement this way internally.
The following are not allowed in FIND and FR WHERE statements, because the corresponding features are not themselves supported.
- LOCATION$ (no scattered groups)
- Equals sign in FILE$ (no scattered groups)
- SFGE$ (no sorted files)
- SFL$ (no sorted files)
Find Processing
The DPT file system applies a certain amount of intelligence to the evaluation of multi-criterion searches. For example powerful criteria like FIND$ and FILE$ are applied first, then indexed equality criteria are performed before indexed range criteria, and the table B search, if required, is done last of all. Of course there is always scope for more ingenious techniques, and only the CCA tech-heads will know exactly what M204 does. DPT does the obvious well-known M204-like things such as that mentioned above, and applies some tricks of its own too. What all this means is that DPT will not behave *exactly* the same as M204 in terms of logical IO counts, BXNEXT etc., (given the different nature of DPT file structures they could never match exactly). However, it still should be a reasonable assumption that if you try two different ways of coding a FIND, the stats from DPT will show the same one is best as if you tried it on Model 204.
It's also possible that DPT does things in a slightly different way when evaluating finds in group context. DPT basically just performs the same find once for each member, which means for example that you could get more than one record locking conflict or MBSCAN-exceeded message during the same find. This is something that will be easy to correct if required in future releases.
List Processing
If DPT runs out of memory when placing records on a global list, the list is cleared and will be found empty by the next request that uses it. Lists are modified in place for performance reasons. Alternatives which would allow a list to reliably retain its pre-modification state in the event of memory problems were deemed too much of a computational and/or memory overhead. This issue is only relevant to global lists because an out-of-memory condition (see elsewhere) always cancels the current request.
Subroutines
CALL statements do not implicitly declare subroutines like they do on M204. If a CALL statement is encountered before the corresponding subroutine is declared, compilation of the CALL is deferred until the end of the request, much in the same way as with JUMP statements. This means that messages are not always issued in the same order as they would be on M204, although it should not affect the situations in which a request will or will not compile successfully.
In some cases it's possible there may be a difference in the handling of file context where list parameters are allowed to be implicitly defined (the M204 manuals don't make it clear what's supposed to happen). In any case if contexts don't match there will be a compile-time error and this will not cause subtle misbehaviour.
On Units
On units are the work of the devil (when he was breifly on the User Language team at at CCA in the 70s). Nothing wrong with the idea in principle, but as implemented in User Language they allow you to write code with highly obscure, even unpredictable error handling behaviour. The major cause of problems is the way ON units from one scope remain in effect during subroutine calls to another. Then consider that in some cases it is allowed to jump out of ON units into the code surrounding them, but that this may not be the same scope as where the condition occurred. This gives rise to a whole page of work-around rules the compiler has to enforce (check out the Model 204 UL manual!) Even then it's not possible to cover every case at compile time, and extra checks are required at run time to insure against inadvertent return from subroutines. All very messy, and hard to guarantee DPT will work exactly the same way.
Secondly, consider that ON units can invoke and also install/deinstall themselves and other ON units, both in their own scope and the scope where the condition was raised. This means it's not always easy to see which ON units are in effect at any given time.
Combine the above points with the fact that activation and de-activation of ON units across calls to subroutines (and across "calls" to ON units) is not completely consistent in practice on M204, and things can get really confusing. DPT should work the same as M204 in all simple cases and most if not all complex cases.
When the BYPASS statement is used after ATTENTION in the test element of an IF or loop construct, M204's behaviour seems a little strange (try it!) DPT is more sensible and bypasses the entire loop or IF block.
If an ON unit is defined inside a FR loop, and later invoked from a point outside that loop, field references in the ON unit give a "no current record" error on DPT. This may be slightly different behaviour to M204, which if memory serves shows the value from the last record processed by the loop. Hopefully an obscure situation though.
Callable $functions
The following three statements can all be used on DPT:
%X = $SETG('A', 'B')
CALL $SETG('A', 'B')
$SETG('A', 'B')
As Sirius discovered a long time ago you can implement new UL features as statements or as $functions, with a variety of pros and cons in each case. Generally speaking $functions are preferable when there are parameters, because of the more structured syntax (brackets, commas etc.), but calling them has historically been clunky because you have to do something with the return code even if it's of no interest. This means if more and more new parts of the language were implemented via $functions, code was going to look bad, and explains the new trend for callable Sirius $functions. The DPT approach is that all $functions are now callable if you're happy to discard the return code. What's more you don't even have to say 'CALL' - a stand-alone $function name is a valid statement, and implies CALL.
This feature can be disabled for stricter M204 compliance using the CSTFLAGS parameter. See also some related notes about syntax-colouring of $function names in the DPT client front end.
Variables
Variables cannot be redefined on DPT - the second definition is rejected by the compiler. This applies regardless of whether the request contains complex subroutines, and also applies to continued requests - the variable from the original request can not even be defined with the same attributes in the continuation. To run existing code where this is done, use MSGCTL to set the "duplicate variable definition ignored" message to CLASS=I.
COMMON variables are an exception to this of course. You have the option of fully defining COMMON variables in each scope they are used, as was required pre-M204-V5, in which case each part of the definition must match the original, or simply declaring the variable name as COMMON, in which case the other characteristics get picked up from the original definition if there was one. The "only one INITIAL clause" error can also be turned on or off as desired.
LVARTBL is registered once per scalar variable, once for every array element, and once for any subroutine parameter of either type (since dimensions are not given on array parameters). This is all regardless of data type (STRING/FLOAT/FIXED).
DPT supports arbitrary expressions as INITIAL values (that is, the Model 204 V5 feature). The expressions are evaluated at the start of running a request, before anything else happens. By default the debugger doesn't stop on the declarations while this happens, but you can make it with an option.
Images
Images as working storage
When using image items in place of %variables, the following general points appliy on DPT.
- STRING items behave similarly to STRING %variables.
- FLOAT items behave similarly to FLOAT %variables.
- EFORMAT items behave similarly to STRING %variables for truncation and padding, and FLOATs for computation.
- BINARY, PACKED and ZONED items behave similarly to FIXED %variables - importantly for runtime DP handling (see notes elsewhere).
- Messages are issued for truncation, variable-too-small etc. in similar situations as with %variables.
- However bear in mind the different storage format. For example the "variable too small for numeric result" error is more of an issue than with %variables, since numeric type items may be quite small (e.g. PACKED LEN 1 can only hold a number from -9 to +9). On the other hand the types which remove decimal points for storage thereby offer a "bonus" available digit!
- BINARY is an interesting case where we must decide how many BP equals each DP. This is handled by treating e.g. a BINARY BP 3 the same as FIXED DP 3 and so on. Going from binary to decimal the number of places required is the same (e.g. 110.101 = 6.625) so it's not as ridiculous a convention as it sounds.
- The high 8 bytes of a FLOAT 16 are set to zeros when writing to the item, but ignored when reading it.
- Accessing a numeric item which contains data in the wrong format is a request-cancelling error.
- LEN UNKNOWN items identified at run time as length zero return a null string or zero when accessed.
The dummy sign character used in UNSIGNED PACKED numbers is 'F' for visual familiarity to mainframe users (incidentally $PACK and $UNPACK are handled the same way). However, with an UNSIGNED ZONED item, the character is '3', which is to say no dummy character. This is more natural in an ASCII environment where the decimal digit "zone" of the character set is X'30' to X'39' instead of X'F0' to X'F9'.
Structure of images
Like ON units, User Language images offer the chance to create infinitely complicated structures, using for example large numbers of staggered redefines in conjunction with UNKNOWN or DEPENDING arrays. Also like ON units, the degree of confidence to place in the accuracy of DPT's emulation of Model 204 depends on the complexity of the situation. DPT should work fine in most realistic scenarios.
DEPENDING and UNKNOWN arrays are two ways Model 204 offers to provide mapping of variable structures using what is essentially a fixed structure, namely an image. With an UNKNOWN array certain item position checks are deferred till run time: a classical approach which simply puts some limits on what can be checked at compile time. A DEPENDING array on the other hand corrupts the clean "mapping" function of an image by inserting/removing fillers into/from external records during READ/WRITE, so that for example byte 25 of an incoming record might end up at byte 35 in the image. In this sense it is an ugly solution to the problem, and poses some tricky questions to the compiler. There are a number of restrictions M204 puts on the positioning of other items in an image which contains one or more DEPENDING arrays, presumably kluges intended to minimize the effect of the filler insertion/removal on those other items. Not all these restrictions are implemented on DPT (I don't even know what they all are, and in any case some would probably be unnecessary given the different underlying implementation). So, while existing code written on M204 should work, there is a small window of opportunity to write incompatible new code.
Screens
Screens as working storage
When using screen items in place of %variables, the following general points appliy on DPT.
- User-defined screen items and ITEMNAME behave similarly to STRING %variables.
- The numeric system-defined items (PFKEY, ROW, COLUMN and ITEMID) behave similarly to FIXED DP 0, (except via screen item variables - see below).
- Messages are issued for truncation, variable-too-small etc. in similar situations as with %variables.
Screen item variables
The following general points apply on DPT.
- The "source" variable can be a variable (:%V), an image item (:%IMAGE:ITEM) or other screen item (:%SCREEN:ITEM) or an array element (:%V(2), :%IMAGE:ARRAY(3,7) etc.)
- Unlike field name variables (%%V) screen item variable references can be stacked (e.g. :::%V = the item named in the item named in the item named in %V). This may or may not be allowed on M204, but in any case coding like that is probably not a good idea.
- The source variable of a field name variable can be a screen item variable (%:%V).
- All screen item variables are considered as strings by the compiler when used in expressions, even if they eventually resolve to ROW, COLUMN or ITEMID. This is for internal convenience.
- The DP specification of the screen item ultimately accessed at run time is preserved when used in an expression. With the system-generated items like PFKEY, DP 0 is assumed.
- There may be places M204 accepts SIVs where DPT doesn't, and vice versa. For example the DEPENDING variable in an image cannot be a SIV on DPT.
Display considerations
The ways in which the view on DPT Client differs from a 3270 are now discussed in the IDE User's Guide.
Screen processing on non-IODev 7 devices
Full screen related statements compile on any thread type, but READ and REREAD fail at run time unless the user is running on an IODev 7 (i.e. there is no support for the clunky line-at-a-time screen read processing you can do on M204). PRINT SCREEN on a non-IODev 7 device generates a formatted text-only version of what the screen would look like, minus colours, underline, blink etc. You can also force this to come out on an IODev 7.
Miscellaneous screen issues
The MUST FILL autovalidation option is treated similarly to REQUIRED (i.e. implicit AND, not OR). This disagrees with the manual but seems more logical.
Redefining Labels
Using the same label name several times at different points in a request is very bad progamming style, but it is a style enjoyed by some. After all, which version of the label is the "real" label? It's never really seemed clear what M204's official behaviour is, and it's certainly not explicitly laid out in the manuals.
DPT does have internal support for redefined labels, but it is currently disabled because its behaviour may be sufficiently different from M204 to be confusing. The following notes are left in here for interest, and to make it clear what a potentially confusing area it is. The ability to redefine labels on DPT may be reinstated if and when it is possible to confirm exactly how M204 handles things.
-----------Start of provisional notes about the disabled feature
When the current request just contains just one occurrence of a label, but there is another identical label in the global table or remaining from an earlier portion of a continued request, there is hopefully no confusion over what should happen, either with jumps to the label or with references by other statements.
However, when a label is redefined in the same request (or more specifically the same request continuation portion) it is allowed so long as the statement type (FD, SORT, COUNT etc.) is the same in all cases, and the file context is the same too where appropriate. This means other statements referring to the label can be confident of the type of "contents". The contents of the label as extracted by other statements reflect the contents as set by the most recent statement executed at run time (loops set the VALUE IN or OCC IN at the top of the loop). Jumps to redefined labels on the other hand are resolved at compile time. If the label has already been defined (backward jumps) the jump is to the most recently-encountered version. Otherwise (forward jumps) the jump has to be resolved in a final pass of the compiler, and is therefore always to the last version of the label encountered in the code. To illustrate these points:
B
A: JUMP TO V //invalid jump (final V is nested)
V: FRV SURNAME
PRINT VALUE IN V //current value of outer loop
V: FRV CITY //redefinition OK - FRV
PRINT VALUE IN V //current value of inner loop
END FOR
PRINT VALUE IN V //final value of inner loop
END FOR
JUMP TO A //back to the top
A: //redefinition OK - "vanilla" statement
PRINT VALUE IN V //final value of final run of inner loop
JUMP TO A //previous statement
END
-----------End of provisional notes
Compile-time messaging
One of the key objectives DPT was written to achieve was to provide much more accurate and helpful compiler messages, so the first thing to point out here is that in most cases the message text doesn't match that on Model 204. Having said that, in some cases the M204 text is so familiar that it is used but with extra information added. In many cases the message is somewhat longer - a example typical of the DPT philosophy here is:
M204:
FIELD NAME REFERENCE NOT WITHIN A FOR LOOP
DPT:
Field name SOMESTRING does not exist in the file context active at this point (TEMP GROUP MYDATA). If you meant a string literal, put quotes round it.
In any case, broadly speaking there are three things that can happen if a statement doesn't compile.
Firstly, as with Model 204, there are a handful of situations where the compiler will issue a message but otherwise compile the statement in question successfully. They can be thought of as helpful advice from the compiler, although you can upgrade them to errors - see below. Most examples of this are listed here.
Secondly there is the most common situation where a statement is invalid. The compiler issues an error message, ignores the line in question, and then carries on compiling. Typically the request as a whole will fail compilation with these because by default the messages are CLASS=E. In other words they are counting errors, which means the error count will be non-zero at the end of compilation, which in turn means the evaluator will not kick in. Compilation may also be aborted prematurely if the error count reaches ERMX. If these messages are made CLASS=I, the request will be evaluated as if the source code line was not present (it will be highlighted in the debugger source code view as having failed compilation). As on Model 204, this is not usually advisable, and may cause unpredictable behaviour.
Finally there are errors which for various reasons mean the compilation cannot continue at all. In such cases the error message is followed by "Compilation aborted" and nothing is run. This category covers things like memory exhaustion and (hopefully rare) internal bugs.
DPT by default should issue messages in roughly the same situations as Model 204. The handling of individual messages in the first two categories above can be altered using the MSGCTL command, for example to force a failed compile by changing an "informational" message to TERM CLASS=E.
Messaging for unsupported M204 statements
In most cases where a Model 204 statement is not emulated, the compiler by default issues an informational message and compiles a "stub" operation into the executable program, so that during execution another informational message gets issued if that operation is hit. The intention of this is to allow existing User Language programs to be debugged and tested. If control doesn't go near the unsupported statement at run time, there is no problem. Even if control does pass through a stub operation, you might use the debugger to break there and tweak variables and image items etc. to pretend that the unsupported statement had been executed. You can upgrade either the CT or RT message to CLASS=E, to force an appropriate error response if desired.
Run-time messaging
Informational messages are issued at run time in the same situations as M204 (divide by zero, variable too small for result, etc.).
A whole family of error messages, namely those issued from inside $functions (e.g. negative parameter to $SQRT to take an obvious example), deserve special mention. On DPT most of these $function parameter-validation messages just use the same message code with differing message text, unlike on M204 where there is a range of separate codes around M204.0745 onwards. Clearly this reduces the precision with which MSGCTL can be applied, and if this is something that causes problems for people, separate message codes can easily be reintroduced.
Additionally, DPT addresses its development and unit-testing mission by issuing messages in a few additional situations where it might highlight a potential UL coding error. A good example of this is when a string %variable is truncated on assignment - a M204 feature that can be overused and sometimes leads to nasty bugs. Similarly when assigning invalid numeric literals to FIXED and FLOAT variables. These considerations also apply to intermediate results when, for example, concatenated strings are truncated or numeric overflow occurs.
In all cases run time messages can be upgraded or downgrded to CLASS=E/I or suppressed using MSGCTL. CLASS=E messages invoke the usual chain of events for a run time error (ON ERROR unit if possible, otherwise request cancellation; then APSY error proc with error global = BUG).
As with compile-time messaging, the default should be for messages to cause errors and be printed on the terminal in roughly the same cases as on M204. All run time messages are by default printed to the audit trail at least.
See also the above comment about messaging for unsupported M204 statements.
Reserved words (and characters)
This section only concerns field names and values, not procedure names. It is an area where the Model 204 manuals are, at the time of writing this, incomplete and/or inaccurate and/or unclear. DPT aims to allow similar usage as on Model 204, but overall is probably somewhat more forgiving, which is to say it requires quotes in a smaller range of situations. The following notes describe DPT's behaviour.
In general, the approach to the treatment of reserved words is to assume the programmer knows what they're doing, and where unquoted reserved words are used to assume the reserved meaning. For example the following command does not contain any ambiguity and requires no quotes.
DEFINE FIELD GIN AND TONIC
But this UL statement would be assumed to refer to two separate fields:
PRINT GIN AND TONIC
So the keyword would have to be "hidden", as in these equivalent alternatives:
PRINT G'IN AND TONIC'
PRINT GIN' AND 'TONIC
The set of reserved words and characters that can cause problems depends on the circumstances. For example the following statement will not work since the keyword WITH introduces the field attributes, (CHEESE would get flagged as an invalid attribute).
DEFINE FIELD BURGER WITH CHEESE
So this time we need quotes to define the field, for example:
DEFINE FIELD 'BURGER WITH CHEESE'
With this field the keyword WITH has a special meaning in UL, and usage of the field might or might not require quotes depending on the situation. For example WITH has no meaning to FIND or ADD, so these are OK:
FD BURGER WITH CHEESE = 3.99
ADD BURGER WITH CHEESE = 3.99
But in expressions WITH means concatenation, so quotes are required, for example:
%X = B'URGER WITH CHEESE'
PRINT BURGER 'W'ITH CHEESE
When defining fields the following points also apply, not for the benefit of the DEFINE command parser, but as sanity checks to help reduce later ambiguous situations in User Language.
- Field names must not consist of a single reserved word. This test considers the entire set of reserved words that applies in all situations (see the M204 file manager's guide).
- Reserved characters must be hidden with quotes
When using quotes in field names they can go anywhere as long as they hide or break up the reserved component. In the specific cases of use in arithmetic and print expressions, the quotes cannot enclose the first character, since then it looks like a quoted string. In other cases such as commands like the one above and record-access statements like ADD and CHANGE, the entire field name can be quoted. A commonly-used convention on M204, which works in all situations regardless of whether there are any reserved words, is to leave the first character of a field name unquoted and quote the rest.
Similar considerations apply with field values in statements such as ADD, CHANGE and FIND, which do not usually need quotes, and as with field names the set of keywords causing problems will be different in each situation. For example here is a common situation where the keyword OR should obviously take its operator meaning:
FD NAME = SMITH OR JONES
But we can change the meaning with quotes, as in these two equivalents which search for a single database value rather than two:
FD NAME = 'SMITH OR JONES'
FD NAME = SMITH' OR 'JONES
On the other hand in something like an ADD statement the value is assumed to be the entire line after a certain point, and no quotes are required because there are no further possible keywords or terminator characters. So these two are equivalent:
ADD NAME = RANDALL AND HOPKIRK (DECEASED)
ADD NAME = 'RANDALL AND HOPKIRK (DECEASED)'
ASCII/EBCDIC
During normal operation, string comparisons like in IF tests and database range finds are based on the ASCII collating sequence. This may be a subtle source of different behaviour from the mainframe (e.g. A<a in ASCII but A>a in EBCDIC). It is also relevant to various other situations, such as the $C2X and $X2C functions, PACKED/ZONED image items etc.
From DPT version 2.12, the $ASCII and $EBCDIC functions provide the capability in User Language to convert between character sets if data is being exchanged with a mainframe, and it isn't otherwise converted during transit.
Case handling in User Language
Traditionally UL has been an all-uppercase language, at least as far as language keywords are concerned (user-named components like variables could be in mixedxcase). From DPT version 2.12, this restriction has been eased by emulating the Sirius conventions, as detailed more fully in the section on Sirius features.
Top
M204 Statements - Processing and Syntax Differences
Model 204 statements that do not feature in this section can be assumed to work pretty much the same on DPT. As a rule there are far fewer deviations and "embellishments" with the syntax and operation of UL statements than there are with commands.
As with commands, custom options can be disabled using the CSTFLAGS parameter.
FOR RECORD NUMBER
- FOR RECORD NUMBER {value | IN label}
- In other words there is no OPTIMIZING FNV option. Actually you can put it on there but it won't have any effect.
PRINT
The print statement does all that the M204 equivalent does. In addition, you can print the term *RECINFO, which is treated exactly the same as *RECORD but simply shows more detailed information about the current record.
PRINT ALL INFORMATION [INTO]
The BLOB-related extensions are supported, but the output produced varies slightly from Model 204, reflecting the underlying storage differences. For example the PAI INTO result-array-3 need only be STRING LEN 10. The default value is LOB_DATA, not LOB_FLOD, thus making BLOB fields work the same as STRING fields by default.
- LOB_NONE: BLOB fields are omitted altogether. No table E disk reads are performed.
- LOB_DATA: BLOB fields are printed as if they were STRING fields, only longer. Non-BLOB field values print as normal. Also this option populates PAI mode result-array-3, if given, with the "raw" descriptors for BLOB fields.
- LOB_SHORT_DATA: As LOB_DATA, except truncating to 255 bytes per value.
- LOB_NO_DATA: If the statement is a PAI INTO, nothing is put into result-array-2 for BLOB fields, but the "raw" descriptors are still placed into result-array-3, if given. In non-INTO mode, BLOB descriptors are formatted into a readable format and printed as if they were the data. In neither mode are any table E disk reads performed.
- LOB_FLOD (or LOB_LOAD): Similar to Model 204, but the information is printed in a way DPT fast load will accept, rather than a way Model 204 FLOD will accept. If the statement is a PAI INTO, this option works the same as LOB_DATA.
- LOB_LOAD_ALL: (DPT custom option). As LOB_LOAD but all fields, not just BLOB fields, are printed in the alternate format. Perhaps of interest when regular STRING fields contain newline characters.
PRINT SCREEN
On an IODev7 the PRINT SCREEN statement works much like READ SCREEN, which is to say it renders the screen in a coloured fullscreen display. On other IODevs, PRINT SCREEN is also allowed, but the screen is displayed simply as text lines, much like a series of PRINT statements. You can force this behaviour on an IODev7 using the DPT custom variation PRINT SCREENLINES.
SORT VALUES
Is this statement supposed to do an in-place sort on the value set, or create a fresh sorted copy in another label in the same way as SORT RECORDS? Memory fails, so it's currently available both ways, as follows. When the truth comes to light these two may have to be switched round (note the 'B' on 'VALUESB'):
V2: SORT VALUES IN V * V is unchanged, V2 contains fresh sorted set
SORT VALUESB IN V * Sorts set V in place - doesn't need a label
M204 $function Differences
General notes
The following notes describe non-trivial differences in behaviour of Model 204 $functions on DPT. Most of these differences reflect differences in file handling, system config issues, etc. A catalog of supported $functions is given in the features cross-reference document.
A few of the $functions have been embellished with custom options. As with all custom options these can be disabled by resetting the CSTFLAGS parameter.
In one or two cases (e.g. $EDIT), the $function is listed here as an admission that it may not work right in 100% of cases.
In many cases Model 204 handles errors in $functions by setting a default return code (often zero or null string) and then continuing. In these situations DPT often issues an informational message before continuing. The same message code is used for all $functions so it is easy to suppress it with MSGCTL if you don't like it. In other cases it's possible that Model 204 may issue an informational message and DPT does not. If any of these are a problem for people, please say so.
Bit string data mapping functions
($BINARY, $FLOAT, $FLOATD, $UNBIN, $UNFLOAT)
When using these functions remember that the assumed byte order of the bit string reflects the architecture of the platform on which DPT is running. At the time of writing that will mostly be little-endian (e.g. an Intel PC). Most (all?) mainframes use big-endian byte ordering.
The DPT versions of $BINARY/$UNBIN and $FLOAT/$UNFLOAT interact correctly with each other, and in that sense they can be said to "work correctly". In addition, these functions use the same data formats as the BINARY and FLOAT image item types.
There are two main implications worth noting however. Firstly if you browse the data in a file generated using say $BINARY, it will not look how you expect based on mainframe experience. For example the number 255 written out as a two-byte unscaled binary value will usually read as FF00 little-endian and 00FF big-endian. The second implication is that these functions can not be used when transferring data between platforms with different endianism.
Other specific notes about these functions:
- $BINARY returns -1 ("overflow") when insufficient bits are specified for the result, rather than truncating/masking, which might also make sense. This behaviour may or may not agree with M204 - please someone confirm.
- $FLOAT truncates its 64 bit input value to 32 bits according to IEEE 754 rules, not any specified in the M204 manuals. (NB this function, as well as $FLOATD, is deprecated on M204 anyway).
Text case processing functions
($ALPHA, $ALPHNUM, $LOWCASE, $UPCASE)
These functions accept the "language" parameter 2, but ignore it. The local language settings in effect on the OS are used.
Thread synchronization functions
($ECBDGET, $ECBDSET, $ECBTEST, $POST, $UNPOST, $WAIT)
- Return codes 3 and 5 are never issued. Both fall under RC=4 since if NUSERS=1 no ECBs are created at system start-up time.
- Extended quiesce is not supported on DPT. Return code 9 covers all cases where QZSIG or CPQZ are supplied as the ECB name. There is no $STATUSD information associated with RC=9. RC=11,12,13 are never issued.
- The SWAP/NOSWAP parameter of $WAIT has no effect, as all threads are permanently "swappable" on DPT. RC=6 is therefore never issued. The parameter value, if specified, is however validated as one of these two values, and RC=17 *is* issued if appropriate.
- Like all resource waits on DPT, $WAIT starts off spinning aggressively. However, since it might be called when no other users are logged on, after a second or so the wait becomes less agressive. This is to give your PC's idle process a chance to run.
- The wait interval given to $WAIT is only accurate to plus or minus one second. In other words a value of 2 might wait for 2 seconds, or 1.01 seconds or 2.99 seconds.
- See also the =ECB command.
Date functions
All internal handling of dates on DPT is based on the underlying date-handling facilities of the local operating system. What this means that there are almost certainly differences in the ranges of dates which are regarded as valid compared to Model 204. To be specific, the date functions on DPT handle dates in the range Jan 1 0100 to Dec 31 9999.
All the date functions on DPT have the (disablable) option to specify the date format as one of two special values. These formats are mainly intended for use with the web server functionality, and are denoted by the format strings 'UTC' and 'HTTP'. To work properly these must be supplied as the entire format parameter, and not mixed with other components. For example:
B
PRINT $DATECNV('DD MON YYYY', 'UTC', '01 JAN 1970')
PRINT $DATECNV('UTC', 'DD MON YY', 0)
PRINT $DATECNV('DD MON YYYY', 'UTC', '02 JAN 1970')
PRINT $DATECNV('DD MON YYYY', 'HTTP', '02 JAN 1970')
PRINT $DATECNV('DD MON YYYY', 'DD UTC YY', '02 JAN 1970')
PRINT $DATECNV('DD MON YY', 'UTC', '30 FEB 70')
END
0
01 JAN 70
86400
Fri, 02 Jan 1970 00:00:00 GMT
02 UTC 70
***
UTC format
The 'UTC' format interprets the date as an integer value which is the number of seconds since midnight on 1/1/1970. Strictly speaking this is 'Unix Time' and not the more complex and celestially-precise 'UTC' (co-ordinated universal time). However, it is a very common approximation in the C programming world and is used by DPT since it's written in C. UTC usually implies a GMT (Greenwich Mean Time, or London winter time) value - more on this below.
This simple numeric format is quite an efficient way of manipulating dates as it is often the "halfway house" in conversions between more readable formats. It is also ideal for comparisons, or doing non-whole-day DATECHG style processing such as adding 1 hour. If the date is given as a standard M204 format (i.e. no time component) the result in this format is always a multiple of 86400 since that's how many seconds there are in a standard day - the time of day is assumed to be midnight. Negative numbers represent times before 1/1/70.
HTTP format
This is the standard date+time format used in HTTP messaging, and any time you put a date or time value into an HTTP header it should be in this format. It is also a good format to use for display in an international setting such as on web pages. When converting from this format or using it as input to, say, DATEDIF, it is not case sensitive, so you can type values in UL code without worrying about case. It can also be in any of the 3 accepted HTTP 1.1 date/time styles. On the other hand all function outputs in this format are given in the standard mixedcase style, as shown above.
The time of day resolution of DPT's internal date-storage format means that for dates before 1899, not every second of every day can be represented, and 'HTTP' format dates will show the closest second they can. This should be no inconvenience in realistic applications.
GMT and local time
Unlike 'UTC', the 'HTTP' format explicitly professes to be GMT, although of course you could set it to any value you like. You should therefore take care that when requesting the time from the local machine and converting it into this format you request GMT and not local time. The standard M204 $DATE and $TIME functions return local time, so there is a DPT custom $function, $NOW, which you can use not only to get GMT, as adjusted for your local time zone and daylight savings, but also to get a combined date+time instead of the two things separately.
Character set functions
(See also General notes on character sets).
These functions make use of the character code pages definded in the CODESA2E and CODESE2A parameters.
- $EBCDIC = Returns the input string converted as per the code page CODESA2E.
- $ASCII = Returns the input string converted as per the code page CODESE2A. (Note: before V2.12 $ASCII returned the input string unchanged).
The supplied DEMOPROC contains a short example request showing these functions in action.
Other Functions
$ALPHA
$ALPHNUM
These two functions have a DPT custom disablable extra parameter 3. A value of anything except zero supplied for this parameter tells the $functions to allow lowercase characters through. Leaving the parameter off gives the default M204 operation, namely only allowing uppercase.
$BLDPROC
In debug mode $BLDPROC fails if the target procedure was included as part of the running request. This is because it is kept locked during execution for debugging purposes.
$CHKPINF
Options 6 and 7 return zero (no extended quiesce)
Option 5 means only pre-images (CPMAX is always 1)
Thousandths of a second in the date time formats is irrelevant (and always shows as 99 or 00)
$DSN and $DSNNUM
Files can not be multi-part on DPT, so $DSNNUM always returns 1
$DSN works as normal
$EDIT
This is a complex function, for which the Model 204 manuals are not totally clear on its precise operation. The current DPT implementation seems to give pretty good results but there may well be cases where they are different from M204. Please report any you come across!
As of version 1.2, the "¬" (hook) character can also be supplied as "^" (hat).
$ENCRYPT
Like $HSH (added in V2.0) $ENCRYPT was previously left out because it was not known what encryption algorithm Model 204 used. That is still the case, but like $HSH, some support is better that none. $ENCRYPT is supported from V2.16.
Note therefore that the resulting encrypted data will not be the same as if the same parameters were given to Model 204.
DPT allows two extra custom parameters, allowing longer strings to be encrypted (parameter 3) and longer resulting encrypted values (parameter 4). Both default to 8 which are what Model 204 uses.
On DPT the encryption is an SHA1 hash function, applied to a concatenation of the input string (parameter 1) and the modifier (parameter 2). The SHA1 algorithm produces 20 byte result values, so if the requested result length from $ENCRYPT is less than that (e.g. the default of 8), the SHA1 result is simply truncated. Requesting a longer result value than 20 characters just returns 20.
$ENTER
Truncation.rounding etc. happens as per normal variable assignments, but no messages are issued.
$HSH
This function has the same purpose as on Model 204, but the hash key generated for any given input string will *not* be the same number.
There are two DPT custom disablable options in addition to the string to be hashed. These let you change the hash algorithm if the default seems not to be giving a good hash table distribution when used on your data.
The first custom option identifies a basic algorithm from a set of around 6, for example number 1 is the Unix ELF hash. The second custom option activates small modifications to the basic algorithm, such as processing the input string in reverse order.
The default gives a good spread of hash keys for typical input data strings, even when they are very similar.
More information on this fascinating stuff is available if required.
$ITSOPEN
Returns true if either a proc directory or a data file is open with the given name (see procedure handling).
Custom option: An optional second string parameter of ['DBFILE' | 'PROCFILE'] to target the query specifically at that entity type.
$JOBCODE
This is handled somewhat differently to M204, mainly reflecting the fact that on DPT there is no difference between an "online" run and a "batch" run. It is something that will be easy to alter in the next release if needed.
The return code from a DPT host run is the highest message code experienced by any user during that run. As on M204, calls to $JOBCODE can increase that high water mark, but not reduce it, and a $JOBCODE call with no parameter simply reports the current value. There is also a DPT custom custom parameter, RETCODE, which is a system-wide parameter accessing the same internal variable as $JOBCODE. RETCODE can be viewed and reset, and *does* allow a change to a lower value.
User-specific message code high water marks are maintained for various purposes, but to clarify the slight difference in meaning from Model 204, they are accessed by a separate function, $THREADCODE.
Note that none of these codes currently make it back to the client in a BATCH2 style configuration.
$LOWCASE
In addition to the general comment above about the language parameter, $LOWCASE accepts a DPT custom disablable extra parameter 3. A value of anything except zero supplied for this parameter tells
$LOWCASE to additionally lowercase the first letter of the string. Leaving the parameter off gives the default M204 operation, namely an uppercase first letter.
$LSTPROC
Less info is returned by this function than the M204 equivalent, since (see also D LIST) we have to make do with what the information kept by the operating system. The same image can be used as on Model 204 to accept the results but some fields will not be populated.
Security class is always zero, and user name is always null.
The alias option can be used in the same way, but will only ever return one procedure since there are no aliases.
The "cursor initiating" call takes a directory snapshot, as per the supplied pattern if one was given. Therefore new/deleted/changed procs will not be reflected, and RESET PRSUFFIX will not take effect, until the next such call (i.e. with loopvar set to 0).
By default $LSTPROC now returns the proc modified dates as YYDDD rather than CYYDDD, but you can get the C by enabling a fudge switch. On Model 204 this behaviour (if it exists at all) is obviously controlled some other way (e.g. CUSTOM parameter 2 setting?) so this is a temporary fix.
$FDEF and $LSTFLD
As with $LSTPROC there are many items in the image that are irrelevant on DPT, and are not touched by these $functions.
$LSTFLD does not use return code 3.
$RDPROC
LINEND returns a semicolon in all cases
Unlike INCLUDE, $RDPROC does not "suck up" the latest version from the client if it's open for edit
$READ, $READINV, $READLC
Responses can't be continued with '-'
They can however be queued with ';'
Responses are simply truncated at 255 characters (behaviour as described in the messages manual for M204.0730 does not happen)
$READINV only results in an invisible input area on an IODev7
$SLSTATS
The M204 manual does not say what the return value of this function is. On DPT for the time being it is a line of stat values for the previous SL period, as per T REQUEST (e.g. CPU=10 DKPR=500 ...)
$STAT and $VIEW
AT LOCATION is not supported
Custom options:
$STAT parm 2 can also be FILE, SYS or SL
Similarly in $VIEW the stat level can be FILSTATS
If file stats are requested the current file context is used in both cases
Both can view CPU and CNCT which according to the manual, M204 cannot
$STRIP
Accepts two DPT custom parameters (disablable for M204 comaptibility).
The first indicates which characters to strip instead of, or as well as, zeros. The default is just '0', which yields the same results as the M204 version of the function.
This parameter can be a string of any number of characters, and any combination of these in any order is removed.
The second inditates where in the string to strip the specified character(s) from, as follows:
- 'LEFT' = strip leading only, as per the standard $STRIP (the default)
- 'RIGHT' = strip trailing only
- 'BOTH' = strip both leading and trailing
- 'ENDS' = same as 'BOTH'
- 'ALL' = strip all occurrences throughout the string
See also the emulated Sirius function $SUBREP later which is more generalized but could be used in similar situations.
Examples:
B
PRINT $STRIP('00123')
PRINT $STRIP(' 0123', ' 0')
PRINT $STRIP('0.12500',, 'RIGHT')
PRINT $STRIP(' "c:/config.sys" ', ' "', 'ENDS')
PRINT $STRIP('&$% ABC +*/ DEF []}', '&$%+*/[]{} ', 'ALL')
END
123
123
0.125
c:/config.sys
ABCDEF
$SUBSYS
If the subsystem is defined the return code is 1 (i.e. always effectively started). This is because subsystems don't need to be started or stopped on DPT.
Return codes 2 and 3 are not used.
DPT also allows an extra custom disablable second parameter, enabling you to retrieve information about the subsystem. The values this can take are:
- Null string works the same as normal.
- All the parameter names from the CREATE SUBSYS command. For example $SUBSYS(,'LOGPROC') will return the name of the login procedure of the currently -running subsystem, if any.
- Two special values which work together with the APSY auto-logout feature. These options are intended to provide the basis for users to write their own home-baked login security schemes for websites etc. (See also AUTOSYS, $HSH, $SHA1 etc.)
- 'LOGOUT_DISABLE' = The calling user becomes immune to being logged out by the auto-logout option of subsystems, and is placed at command level on disconnection from any subsystem. The operation of the LOGOUT command is unaffected.
- 'LOGOUT_ENABLE' = Removes any immunity previously imparted by the above.
$VNUM
The SORT and SORTKEY options are not allowed
DPT Custom Statements
As with the custom commands these are left in here from development in case they may be of some use or interest to anybody. Again more details are available if desired, and again they can all be disabled with CSTFLAGS.
Custom statements
Statement | Description |
DEBUG IMAGE image-name |
See note below |
FD_DIAG diags-level |
See note below (also FR_DIAG) |
FREE [ALL PAGES | PAGE IN label] |
Release buffer enqueues. See LOOK PAGE note |
LOOK PAGE IN label INTO %variable |
See note below |
REQUEST PAGE number |
Must be labeled. See LOOK PAGE note. |
WEBPRINT print-expression |
See web coding guide. |
ZAP PAGE IN label from %variable |
Hack a buffered file page. See LOOK PAGE note. |
DEBUG IMAGE
This statement produces a breakdown of the item positions within the named image. It can help you to achieve the correct alignment of items during development of a complex image. It might also be useful if you have apparent problems with compatibility of images that you know work on Model 204 but seem not to work, or work differently, on DPT.
FD_DIAG
Use the sequence 'FD_DIAG n' in place of 'FD' or 'FR_DIAG n WHERE' in place of 'FR WHERE', and the system will display its workings as the find takes place. This is handy for technical support if the system crashes during a database search, but might also help with tuning by the User Language programmer. The value of n is a set of bit switches. Use 255 or 'ALL' for max detail. Specific bit values can be documented if there is any interest.
LOOK PAGE etc.
This small suite of statements lets UL code access buffer page data similarly to the *LOOK command which is in turn similar to M204's equivalent. Typically you would have to reset STRINGMAX to get much use from these statements. There is a short example program in DEMOPROC.
DPT Custom $functions
All of these three groups are installed in the demo version of DPT host. All can be disabled using the CSTFLAGS parameter.
Note also that custom $functions can be dynamically linked (ie. from a DLL) or statically linked, but for simplicity of installation all these are statically linked.
Set A: Sample $functions
These are some of the functions which go along with a tutorial document about how to write and install custom $functions on DPT. They therefore range from utterly simple (the first exercise) to quite complex (later exercises). They have been left installed in the distributed executable module in case anybody finds them useful. However, if and when the DPT host is distributed as a semi-built object library this group of functions would not be installed, although they could of course be recompiled and relinked if desired.
No documentation is supplied for these functions, so you'll have to experiment with them. The editor describes what parameters are expected when you type the left bracket after a function name. Alternatively if you get in touch via the web site, more details and/or the source code are available.
$Function | Description |
$DPT_ARRSORT |
Sort a 1D %variable (not image) array |
$DPT_CURDIR |
Analogous to to CURFILE |
$DPT_FUNCINFO |
Display linked $function info, (See also =FUNCLIST command, and sample program in DEMPROC). |
$DPT_RAND |
Random number between 0 and 1 |
$DPT_RANDOM |
Allows scaling and shifting {V2: quite similar now to Sirius $RANDOM} |
$DPT_RANDOMNORMAL |
A normally-distributed value instead of uniform |
$DPT_RANDOMSTRING |
A string of random capital letters - handy for test data |
$DPT_STATUS2/DPT_STATUSD2/$DPT_SPAWN |
Now moved to Set B (see below) |
$DPT_C2D/$DPT_D2C |
Quirky character <-> decimal |
$DPT_X2D/$DPT_D2X |
Quirky hex <-> decimal. Sirius $D2X/$X2D usually more useful. |
Set B: Extensions to the "standard" set of $functions
These are functions which are more important than those in Set A to the business of using DPT for realistic applications on the PC, and they are linked in along with the M204 "standard" set ($SUBSTR, $LEN etc.). So all distribution formats for DPT host will include these. At the time of writing this group contains functions which are either interfaces to local platform facilities like files, sockets and threading services, or are tangentially related to the web server functionality.
Unlike Set A, these functions are fully documented here.
$Function | Description |
$BLOB |
BLOB maintenance facility |
$EMAIL |
Send an email |
$FILE_xxx and $xxDIR |
Interfaces to the local file/directory system |
$LOAD |
$Function equivalent for =LOAD |
$LOGXXX |
$Function equivalents for LOGCTL, LOGLST etc. |
$LZW |
String compression |
$NOW |
Returns GMT date+time |
$SHA1 |
Create SHA1 digest of input string |
$SOCKxxx |
Interface to the socket stack |
$SPAWN |
Start a daemon (NB used to be $DPT_SPAWN in set A) |
$STATUS2/STATUSD2 |
Used by some of these $functions |
($STRIP) |
Significant extensions to the standard M204 function (see earlier) |
$THREADCODE |
Like a user-specific version of $JOBCODE |
$TOKEN/$TOKENS/$TOKENIZE |
Similar to $WORD/$WORDS |
$UNLOAD |
$Function equivalent for =UNLOAD |
$EMAIL
This $function is a simple interface to the "Blat" email sender utility. Depending on user feedback, extra options could be added, or it could be expanded it to work via other email clients, or it might operate as a stand-alone SMTP client itself not needing any external tools. Anyway, to use this function currently requires you to get hold of Blat - see its website. Note that Blat is not and does not try to be a perfect email client, and is open source and therefore not officially supported. However it's free, and is well-known and widely-used, and has plenty of documentation on its website and on various user groups around the internet.
The same underlying email interface can also be invoked on DPT via the USE $EMAIL and =EMAIL commands.
DEMOPROC in the DPT download contains examples of various ways to send an email.
How it works summary
Blat is a Windows command line tool which takes what you type in as the email title, content, recipients, attachments etc., assembles them all into "SMTP" (low level email format), and finally passes the package on to an SMTP mail server which distributes the email.
All DPT does is take the parameters supplied on the $EMAIL function and makes up a command line which it passes to Blat as if entering it at the DOS prompt. Blat then works as normal and DPT captures any messages it issues. The $EMAIL function has separate parameters (see below) for the more commonly-needed elements like subject and recipients, but there are too many Blat options to give them all their own parameter, so everything except the basic ones must be given as "miscellaneous other parameters". You have to do that for example to add things like attachments and blind copy recipients. Refer to the Blat syntax document for the command line format when you need to do that. Also see the DEMOPROC examples.
Blat set-up
- Download Blat if you don't have it already
- Issue the blat "install" command if you haven't done it already
- Copy blat.dll into a directory where it will be loadable by DPT. For example the Windows SYSTEM32 directory, or anywhere mentioned in your system's "PATH" variable, or even just the DPT host directory (although then you might need to repeat this step every time you installed a new version of DPT).
- Email the world
The blat install command is required to save you having to enter SMTP server and logon details with every email. The format of the command is somewhat complicated with lots of options which you'll probably want to leave as defaults. Here's an example:
blat -install mail.isp.com john@smithfamily.net - - - john@smithfamily.net sm1th
The parameters are positional, so hyphens are used to specify defaults (the hyphens must be there). In this example there were 4 non-default values given. In order they were: The address of the mail server; the default "from" address for emails if none is given later when each is sent; a login ID to the mail server; and the corresponding password. You should be able to copy those details straight out of the "options" screen of your usual email client.
To set a short name/nickname which will appear in the recipient's inbox, use the following syntax:
blat -install mail.isp.com "John Smith <john@smithfamily.net>" - - - john@smithfamily.net sm1th
Refer to the Blat docs for information about the items which took their defaults above if you're interested.
$Function syntax
$EMAIL(subject, message, to, from, misc options, diagnostics)
- None of the parameters are compulsory (see above comments about how they are passed to Blat). Usually however the first 4 should be supplied at least. The send will obviously fail if no recipient is given somehow.
- subject: A string which will become the message subject/title.
- message: Can be specified as a string or a Sirius list. If the parameter value is a number it is assumed to be a $list ID. Lines within the message can be separated using the pipe ("|") character, or if you use a $list, each item represents a line.
- to: A string containing the email address of the recipient. Multiple recipients can be separated with commas.
- from: A string giving the return email address. If not given, this will be the default sender address given at Blat install time. To make a short name or nickname appear in the recipient's inbox, use the format shown above in the second install example, but don't add extra quotes here.
- misc options: A string containing any further Blat command line options required. Anything you give here is appended with no modification to the command line when calling Blat, so it's up to you to quote the appropriate parts if required - use double (DOS-style) quotes. Also try not to put extra spaces between things - Blat sometimes doesn't like it.
- Some of the Blat options let you to pass even more options in files. You might find the FILE_XXXX set of functions covered elsewhere in this document useful to prepare the files in such cases (in particular perhaps the "TEMP" disposition on $FILE_OPEN).
- When specifying miscellaneous option strings which contain file paths (e.g. attachment names), you will certainly have to deal with a feature of Blat where backslashes must be given as two backslashes. This is because backslash is the C language escape character and Blat is written in C. For example use -attach "C:\\pic.bmp". (See use of $SUBREP in the DEMOPROC example).
- diagnostics: DPT captures the Blat output that would normally come to the DOS prompt. This parameter is a numeric value specifying whether, and how much of, this captured output should be printed to the audit trail. Possible values are 0=Nothing; 1=Basic messages; 2=Blat's "-debug" switch is turned on; 3=Blat's "-superdebug" switch is turned on. Setting "1" is sufficient to diagnose most problems.
Return codes.
- 0 = Success
- -1 = Missing or invalid parameter given in UL
- -2 = Error loading the Blat DLL
- -3 = Some other error invoking Blat
- >0 = Blat error codes. Refer to the Blat documentation for more on these, and/or try re-sending the email with the diagnostics parameter turned on.
FILE_xxx
This set of functions provide a more direct interface to the OS file I/O facilities than is available via standard User Language IMAGE read and write. Files accessed with these functions do not need to be ALLOCATEd to the system. The functions are intended for applications needing to access files in an ad-hoc manner, such as might be the case in a web application, or any other native DPT application that wasn't just a port of a Model 204 application.
Error codes
In all cases error conditions are reported via $STATUS, although in some cases you can infer an error from the return value of the function itself. These are the $STATUS values. In many cases the error is accompanied by some descriptive text in $STAUSD, and on one or two cases some extra-detailed descriptive text in $STATUSD2.
0 = Success
1 = Invalid parameter option
2 = Enqueue failure in OPEN or DELETE
3 = Miscellaneous errors
4 = Invalid file handle used
$FILE_OPEN(os-filename [, disposition])
- The 'os-filename' is passed to the operating system as-is, and so can be an absolute value (e.g. 'C:\autoexec.bat') or relative (e.g. 'pics/baby.jpg'). The file name can be left blank if you also give a disposition of 'TEMP'.
- The return value is a "handle" to the file, and can be used for future READ, WRITE etc. calls. In case of an error, the return value is 0 and $STATUS is set as above.
- The handle is actually the address of an internal control object which is created by this function. The lifetime and ownership of this object are the same as for the equivalent socket objects, and are discussed under $SOCK_CLOSE.
- The 'disposition' should be a string value taking one of the familiar values from the mainframe. Those supported are: 'OLD' (file must exist), 'NEW' (file must not exist, and is created, empty), 'COND' (as NEW but file is opened OK if it does exist), 'MOD' (as OLD but all WRITE operations go at the end of the file), 'SHR' (as OLD but WRITE operations disallowed).
- Special non-standard disposition values used by this function are 'CLR' (as OLD but clear down the file when opening), 'CCLR' (COND + CLR), and 'TEMP'. When using the TEMP disposition the given file name is ignored and DPT creates a new file with a system-generated name in the SEQTEMP directory. This directory is cleared down after each run so you are saved the trouble of deleting the file when you finish with it. The file name allocated in this case can be retrieved using $FILE_GETNAME.
- The default disposition is OLD.
- The OS is asked to enqueue the file in exclusive mode, or share mode if disp was SHR. Note that the nature of this enqueueing is different to that used for internal DPT objects such as procedures and database files. With such internally-controlled objects the DPT host applies an extra layer of enqueuing so that different DPT users are protected from each other and, for example, if one user is editing a procedure (holds EXCL), another can't display it (requires SHR). The $FILEIO function works only at the lower level and only the OS enqueueing is applied, with possible different results.
$FILE_GETNAME(handle)
- Returns the OS file name (full path) associated with the handle. Useful when you got a system-generated file name using the TEMP disposition at OPEN time, or when you did give a specific file name but not the full path.
$FILE_READ(handle, length [, offset])
- The length and offset specify which part of the file to retrieve.
- A length of -1 (the default) means read the entire file.
- An offset of zero means the start of the file. An offset of -1 (the default) means read from the current file pointer position.
- The return value is a string containing the requested portion of the file.
- If the file does not contain enough bytes to fill 'length', as many bytes are returned as there were up to the end of the file.
- The file pointer is moved forward by 'length', or to the end of the file, so repeated reads taking the default offset will pass smoothly through the file with the minimum of disk head movement.
- You can increase the value of the STRINGMAX parameter to declare a receiving %variable of sufficient size for big chunks of data.
- READ operations move the file pointer along by the length of the data read.
$FILE_READLINE(handle)
- Similar to 'READ', but reads a "line" from the file, as delimited by the next CRLF sequence. This simplifies application code in the common case of textual data.
- The final line in a file may or may not have a CRLF terminator. Or put another way, if the final two characters in a file are the CRLF, this call does *not* return one last empty line.
- The returned string is stripped of its CRLF terminator (if it had one - see previous point).
- No offset parameter is allowed, since CRLF-terminated lines are usually variable in length. Lines are read sequentially unless you do something clever with SEEK between READ LINE calls.
$FILE_TELL(handle)
- Returns the current position of the file pointer in the specified file.
- This is one way to tell if a sequence of read operations has reached the end of the file (compare with GETSIZE - see below).
- Note that the "offsets" as used by this function are zero-based. When it is opened the file pointer will usually (unless disp=MOD) be at offset zero - the beginning of the file. When the pointer is sent to the end of the file with SEEK or READ, the file pointer is considered to be just past the last byte in the file, so its value will be the same as the size of the file.
$FILE_SEEK(handle, new-offset)
- Repositions the file pointer in the specified file. You can give -1 to go to the end without first having to request the file size.
- The return value is the new file pointer position.
- You can SEEK past the end of the file, in which case the next WRITE will pad the file out with hex 00 characters.
$FILE_WRITE(handle, expression [,offset])
- Writes the value of the string expression to the specified file.
- An offset of zero means the start of the file. An offset of -1 (the default) means write at the current file pointer position.
- If the file pointer is not at the end of the file, existing data is overwritten. There is no "insert" alternative.
- When writing data at the end of the file, it is extended automatically (i.e. there will be no "x37" or "file is too small" type of message as you would get with mainframe files).
- The return value is the number of bytes written.
- The file pointer after this operation is positioned after the last byte of the written data. In other words it advances by the length of the written data.
- This operation is disallowed if the file was opened with disp=SHR.
- If disp=MOD, the data always goes at the end of the file regardless of the supplied offset.
$FILE_WRITELINE(handle, expression)
- Works just like 'WRITE', but adds an additional CRLF sequence to the end of the written data. This simplifies application code in the common case of textual data.
- The return value is the length of the string without the CRLF.
- There is no offset parameter for the same reason as with READ LINE, and this operation would normally be used to write a file from the start after first emptying it.
$FILE_COMMIT(handle)
- Forces any updates to be written through to disk.
- Depending on the platform this may or may not happen with every write anyway, but the WRITE options to this $function do not do so. On Windows in particular the OS will definitely be buffering the file, and is best not to explicitly commit unless you have good reason to (e.g. the file update time may not get set unless you commit).
- The CLOSE option below is the only one which includes an implicit COMMIT.
$FILE_GETSIZE(handle)
- Returns the number of bytes currently in the file.
$FILE_CHSIZE(handle, new-size)
- Truncates or extends the current contents of the file. A new size of zero empties the file.
$FILE_MODTIME(handle)
- Returns the date and time of the last modification to the file, as a UTC style value.
$FILE_CLOSE(handle)
- After this call, the specified handle becomes invalid for all future calls. The underlying control object created by $FILE_OPEN is destroyed.
- By default all files opened in a request are closed anyway at the end of the request, but you can change this - see SETG and IMPORT below.
$FILE_DELETE(os-filename)
- Deletes the named file, which must not be in use.
- A single call to this function can only delete one file. To perform DOS-style pattern deletes, you could first use $DIR and then make a sequence of calls here, or pass it all over to the OS with =SHELL.
$FILE_SETG(global-name, handle)
$FILE_GETG(global-name)
$FILE_IMPORT(handle)
- Support for global and cross-thread usage of files opened by $FILE_OPEN.
- Same meanings as with socket handles.
$FREAD
A shorthand version way of issuing several of the above calls in one go for more convenient use in some situations, particularly reading entire files at once.
- $FREAD(os-filename, length, offset)
- The parameters have the same meaning as in the $FILE_OPEN and $FILE_READ functions. The open disposition is assumed to be SHR.
- The return value is the data read.
- Each call to this function opens the file, positions the file pointer, reads data and closes the file. It is therefore inefficient to call repeatedly to pass through anything but a small file. The function is intended as a convenient syntax most suited to reading entire files in one go, or maintaining fixed-format records which are infrequently accessed.
$FWRITE
A shorthand version way of issuing several of the above calls in one go for more convenient use in some situations, particularly writing entire files at once.
- $FWRITE(os-filename, expression [, disposition, [, offset] ]])
- The parameters have the same meaning as in the $FILE_OPEN and $FILE_WRITE operations.
- The default disposition is 'CCLR', which causes the entire contents of the file to be replaced, or a new file to be created if it didn't exist.
- Certain values of disposition and offset are invalid (disp=SHR) for example.
- The return value is the number of bytes written.
- Each call to this function opens the file, positions the file pointer, writes data and closes the file. The same applicability comments apply as with $FILEREAD.
$DIR
Companion to the above functions. Returns a directory listing from the host machine. This is one of the DPT custom functions that uses the very handy Sirius-style $Lists.
- $DIR($List ID [, filename-pattern [, 0|1]] )
- The pattern given should be just like you would in a DOS DIR command. It is case-insensitive. The default is ('*').
- The third parameter can be given as any non-zero number to make the function list directories instead of files. By default it is zero and lists just files.
- The resulting list is sorted by name. To obtain a list sorted by, say modified time, use the names from this function to get the desired information via $FILE_MODTIME, make another list and sort that. This kind of thing can be added later if anybody wants it.
- The return code is 0 (success), -1 (invalid $list ID) or -2 (any other error).
$RMDIR
$MKDIR
Companions to the above functions. Named after the DOS commands ("remove directory" and "make directory") and the C function ("get current working directory").
- $RMDIR(directory-name)
- $MKDIR(directory-name)
- The return code is 0 (success) or 1 (failure), in which case $STATUSD contains a textual description of the problem.
$LOAD
This $function is equivalent to the =LOAD command, but offers a small amount of extra functionality, namely that the various options can be dynamically supplied via User Language expressions and variables.
- $LOAD (filename, options)
- By default the current file context is used, or you can override it with a filename parameter.
- The options parameter should contain a string expression as per the equivalent part of the =LOAD command, except that no brackets are required round the options.
- Other notes about the processing of this $function are the same as for the command, since at runtime it hooks into the same internal routines.
- Result codes and actions:
- 0 = success
- Positive = Error during load processing - the audit trail will contain more messages saying what. The request will continue if possible although it depends on the seriousness of the error, and in any case the file affected may be unusable by later statements if they try.
- -1 = No current file context, or invalid supplied context. The request continues with an error message issued.
- -2 = Invalid parameter. The request continues with an error message issued.
Examples:
B
%X = $LOAD
%X = $LOAD (%FILENAME, %OPTS)
%X = $LOAD ('SALES', 'PREVIEW ' WITH %PCOUNT)
END
$LOBDESC
This can be used to retrieve the BLOB descriptor value held in table B if you're interested. There's nothing you can do with it really, and it's easier to use the LOB_NO_DATA option on the PAI statement to see BLOB descriptor values for debugging/tuning purposes if desired.
- $LOBDESC(fieldname, occurrence)
- The return value is a string containing the 10 character descriptor, which you could examine using $C2X or by overlaying onto a UL image consisting of BINARY items.
- The function returns null for conditions such as there being no current record loop, or the field not being present on the record.
- If the field is not a BLOB field, the function returns the normal value of the field as a string.
- The fieldname should be given as a string expression, and the occurrence as a number which can default to 1.
- For example:
$LOBDESC('COMMENTS', 2)
$LOBDESC(%FLDNAME, %FLDOCC)
$LOGXXX
This set of functions makes it more convenient to issue the LOGCTL etc. group of commands than it is via $COMMBG. Plus in $LOGPWCHK, DPT provides a way to validate passwords other than at login time.
$LOGCTL
- $LOGCTL('A'|'C'|'D', userid, password, privs)
- See the LOGCTL command notes for details about allowable parameter values on DPT.
- Privileges must be given to the function as a decimal number, but you can work with hex using the $X2D function (e.g.
$LOGCTL('A', 'JOHN', 'PASSWORD', $X2D('00FF'))
- As with the LOGCTL command, when using 'C' (change), the password and/or privileges are left alone if no parameter is coded - e.g. with
$LOGCTL('A', 'JOHN',, 255) no attempt is made to change the password. Supplying an actual default value (null string or zero) is taken to mean $LOGCTL should attempt to use the value.
- Return values:
- 0 = success
- -1 = invalid operation code
- -2 = missing userid
- -3 = invalid userid
- -4 = password required for A (add), or null password supplied for C (change)
- -5 = privileges out of range
- -6 = user already exists (A)
- -7 = user does not exist (C or D)
$LOGKEY
$LOGLST
Returns the output that would come from the LOGLST command into a $List. Pretty much as you could do with $COMMBG really but cutting out the middle man.
- $LOGLST($List ID, user1, user2)
- Return values:
- 0 = success
- -1 = invalid $list ID
- -2 = any other error
$LOGPWCHK
Allows you to validate passwords other than at login time, like with e.g. $RACF/$TSS etc. on Model 204.
- $LOGPWCHK(user ID, password)
- Return values:
- 0 = password is valid
- -1 = password is NOT valid
- -2 = user does not exist
$LZW
Simple string compression and expansion, via a bare-bones C++ LZW implementation by Nikos Bozinis. Compression ratios aren't as good as with a professional utility, but this may be acceptable to avoid the inconvenience of calling outside User Language. For example English text via $LZW typically reduces to 35-45%, vs say 15-25% with 7Zip.
- $LZW( C[OMPRESS] | E[XPAND], string)
- Returns the compressed or expanded version of the input string.
- The algorithm is reversible, so $LZW('E', $LZW('C', 'SOMESTRING')) returns 'SOMESTRING'.
$NOW
This function is most likely to be useful in a web application. It is the only realistic way to access GMT within User Language on DPT, and is also a convenient way to get a combined date+time value.
- $NOW('GMT'|'LOCAL')
- Returns an integer value as per the 'UTC' date+time format.
- When returning GMT, the local machine time is adjusted as per the OS settings for time zone and daylight savings.
- Note that the default is GMT, not local time. This reflects the main intention of this function to be used in an international setting.
$SHA1
This is not implemented as an algorithm option on $HSH simply because $HSH returns a number not a string. Also, this function is separate from $ENCRYPT (which also uses SHA1 on DPT) because $ENCRYPT performs various extra operations in addition to just a straight hash.
- $SHA1(string)
- Returns the SHA1 digest of the input string.
- The input string can be any length. The output string is always 20 characters long.
$SOCKxxx
This set of functions make up a fairly straightforward interface to the socket stack for "connection-oriented" sockets. The operations available correspond closely to underlying function calls, and you should consult your winsock or general sockets documentation for more details if required.
DEMOPROC in the DPT demo installation contains an example of simple use of this function to conduct a socket conversation between two users. Depending on when you're reading this the sample web site in WEBPROC may also contain an example of using this function to listen for returning connections from a served applet.
Note that firewall settings in effect on your machine may block socket traffic, even via the loopback address.
Error codes
In all cases error conditions are reported via $STATUS, although in some cases you can infer an error from the return value of the function itself. These are the $STATUS values. In many cases the error is accompanied by some descriptive text in $STAUSD.
0 = Success
1 = Invalid paremeter option
2 = The socket was bumped during the requested operation (most common while waiting in ACCEPT and RECEIVE but potentially also during a large SEND).
3 = ACCEPT or RECEIVE timed out.
4 = Invalid socket handle used
5 = Any other errors
$SOCK_LISTEN(port-number)
- The return value is a "handle" to a listening socket which should be used in subsequent calls to 'ACCEPT', or zero if there is an error (see $STATUS codes below).
- The handle is actually the address of an internal control object which is created by this function. The lifetime and ownership of this object are discussed under $SOCK_CLOSE.
- When choosing a port number, there will be less chance of clashes with other applications if you go into the thousands or tens of thousands (e.g. the DPT client connects on port 13204). Check out the IANA web site for more information.
$SOCK_ACCEPT(listening-handle [, timeout])
- The 'listening-handle' should be the value returned from a previous successful 'LISTEN' call.
- The calling program enters bumpable wait state 93 until it receives an incoming connection on the port number which was given to 'LISTEN'.
- The return value is a "handle" to a new socket, connected to the client, and can be used for future SEND, RECEIVE etc. calls. In case of an error, the return value is 0, and $STATUS is set as above.
- After a successful call to $SOCK_ACCEPT the calling thread will have handles to (at least) two valid socket control objects, namely the listening socket and the newly-opened connected socket. Depending on the circumstances it might often not close the listening socket to concentrate on the conversation with the client, or perhaps pass the connected socket to a new thread (see $SPAWN) and loop round to accept more new connections. Lifetime and ownership of socket control objects are discussed under $SOCK_CLOSE.
- The timeout value is in seconds, and if the call does not otherwise return in this time, it is aborted ($STATUS=3). The default is never to time out. This parameter is useful if the partner process may or may not respond, allowing you to "give up" and close user/thread resources cleanly without the extra complication of bumping. For example if you serve a web page applet and set up a $SOCK_ACCEPT to service the applet's expected reconnection, waiting for more than a few seconds would be futile.
$SOCK_CONNECT(address, port-number)
- Analogous to $SOCK_ACCEPT, but as called by a "client" process. This function attempts to open a connection with a process which is listening at the specified address and port.
- The address can be given in either "dotted" or literal format, with the latter being resolved in the usual way. For example 'localhost' will usually correspond to the loopback address '127.0.0.1'.
- The return value is a "handle" to a socket, connected to the listener, and can be used for future SEND, RECEIVE etc. calls. In case of an error, the return value is 0, and $STATUS is set as above.
- The handle is actually the address of an internal control object which is created by this function. The lifetime and ownership of this object are discussed under $SOCK_CLOSE.
- At the moment this call returns, the listening end can be assumed to have just completed a $SOCK_ACCEPT call. Either end can start sending and/or receiving now, according to the conventions in your application-level code.
$SOCK_SEND(handle, expression)
- This operation simply sends the specified data down the socket identified by 'handle'.
- The return value is the number of bytes sent, which after a successful call will be the length of the expression. In case of an error the return value is 0, and $STATUS is set as above.
$SOCK_SENDLINE(handle, expression)
- Works just like 'SEND', but adds an additional CRLF sequence to the end of the sent data. This simplifies application code in the common case of a textual conversation with the partner process.
- The return value is the length of the string without the CRLF.
$SOCK_RECEIVE(handle, length [,timeout])
- The calling thread enters bumbable wait state 93 until the partner process has sent 'length' bytes of data, and we have successfully received them at our end.
- The return value is a string containing the received data. You might often have to increase the value of the STRINGMAX parameter (default 255) in order to declare a receiving %variable of sufficient size. In case of an error the return value is a null string, and $STATUS is set as above.
- See $SOCK_ACCEPT above for timeout comments.
$SOCK_RECEIVELINE(handle, length [,timeout])
- Similar to $SOCK_RECEIVE, but reads a "line" from the socket, as delimited by the next CRLF sequence to arrive. This simplifies application code in the common case of a textual conversation with the partner process.
- The returned string is stripped of its CRLF terminator.
- The returned string must not exceed 32K in length.
- See $SOCK_ACCEPT above for timeout comments.
$SOCK_POLL(handle)
- This performs a non-blocking check to see if there is any data in the socket which could be retrieved by a subsequent $SOCK_RECEIVE. The return value is 1 (there is some data) or 0 (there is none). This can be used to prevent your code going into a blocked state if there is other useful stuff it could do with the time instead, or simply to display some kind of wait/progress information to the user.
- Note that DPT does not provide a direct pass-through to the socket MSG_PEEK option, which is easy to use inappropriately, as well as not being universally supported.
$SOCK_SHUTDOWN(handle [, 'SEND' | 'RECEIVE' | 'BOTH'])
- This operation can be used prior to a $SOCK_CLOSE call in order to co-ordinate socket flush and linger effects in the socket infrastructure.
- Explicit shutdown is usually not necessary however.
- After this call, the specified socket will fail in send and/or receive operations depending on which type of operation was shut down.
$SOCK_CLOSE(handle)
- After this call, the specified handle becomes invalid for all future calls. The underlying control object, as created by $SOCK_LISTEN, $SOCK_ACCEPT or $SOCK_CONNECT, is deleted and all associated resources returned to the system.
- If you don't close them explicitly, sockets opened by a request are by default automatically closed at "full" END (i.e. they remain across END MORE).
- Ownership of the control object can be reassigned somewhere else, in which case the request will not delete it at END. The control objects are always owned by something, either a currently-running request, or a user's global table. See $SOCK_IMPORT and $SOCK_SETG below.
$SOCK_IMPORT(handle)
- Transfers ownership of the socket from its current owner to the currently-running request, which will now automatically close it if required when it ends. The previous owner might either be the current user's global table, in which case the IMPORT makes it "non-global" again, or another user entirely, in which case the socket will become inaccessible to that other user.
- This might be used if you have a listening program which farms out incoming connections to daemons using $SPAWN.
- $SOCK_IMPORT and $SOCK_BUMP are the only ones of this set of $functions for which the specified handle may currently belong to another user. This ensures thread-safe usage of sockets.
$SOCK_SETG(global-name, handle)
- Transfers ownership of the socket from its current owner to the global table of the current user, so that it will remain usable until the user logs off. (i.e. across request boundaries). The socket will then be usable by any request the user runs that knows the handle value.
- If the socket is already owned by the global table of the current user, it is reassigned the new name. In other words each global socket only has one name. {In this sense the two parameters of this function might be considered "the wrong way round" since the global table main "key" is the socket handle, but they were defined this way merely for familiarity with the regular $SETG.}
- Names must also be unique. Only one global socket can have a particular name.
- Socket handles saved like this do not get cleared by CLEARGO - they are outside the normal behaviour of the global table. To remove an entry from the global table one, use $SOCK_CLOSE or $SOCK_IMPORT.
- The return code is 0 (OK) or 1 (any error - e.g. name is already in use).
$SOCK_GETG(global-name)
- Locates a socket previously saved by $SOCK_SETG and returns the handle for use in the current request.
- After this call the socket remains owned by the global table (compare $SOCK_IMPORT).
- The return code is zero if there is no global socket with the specified name.
$SOCK_BUMP(handle)
- If the socket is currently waiting in a $SOCK_ACCEPT or $SOCK_RECEIVE call on another thread, calling this function gets it out of that state, and the call that was waiting returns with $STATUS=2.
- Note that this is not as "harsh" as bumping the user with the BUMP command, although that will also interrupt these waits. If you bump a thread which is in a waiting $SOCKxxx call, the call is interrupted so that the thread can be terminated as usual. So in that case, instead of the request continuing with $STATUS=2, it ends entirely.
- Calling $SOCK_BUMP on a socket does not close it, and it can be reused after UNBUMP if required.
- $SOCK_IMPORT and $SOCK_BUMP are the only ones of this set of $functions for which the specified handle may currently belong to another user. This ensures thread-safe usage of sockets.
- Note that this is the only one of this set of $functions that doesn't correspond directly to an underlying winsock function call. It is a DPT custom feature, associated with the user threading infrastructure.
$SOCK_UNBUMP(handle)
- Allows a socket to be reused after it has been bumped.
$SPAWN
This is how you can start a fresh thread and get it to do some work. See the discussion of the =SPAWN command for general notes on kicking off daemon threads.
Note that this function starts an entire fresh user thread, and does not make use of any existing $function support daemons. You might therefore use this function to perform larger background tasks and e.g. $COMMBG for smaller tasks.
Note also the difference between a DPT user thread and an external OS task, which you can kick off using =SHELL.
The $SPAWN function has all the same functionality as the =SPAWN command, plus the additional ability to specify multiple "real" input lines via a $list. The parameters are in a different order to the command because here the trailing parameters might often be left blank, whereas in the command the daemon's input line comes last as it might contain spaces and this saves you having to give quotes. The the default is also a synchronous call rather than asynchronous.
- $SPAWN(commandline [, daemon ID] [,'SYNCH' | 'ASYNCH')
- $SPAWN($list ID [, daemon ID] [,'SYNCH' | 'ASYNCH')
- The second form is assumed if the value given as parameter 1 is numeric.
When building daemon initiation into applications (rather than the occasional one-off manual use) it is probably best to choose a specific 5-character code and stick to it. This helps to identify in the audit trail what's going on, and also keeps down the amount of output file detritus in the #DAEMON directory from ever-increasing system-generated IDs.
The return code is numeric and indicates:
- -1 = an error occurred kicking off the daemon
- -2 = an error occurred retrieving the results after the daemon finished (synchronous mode only)
- -6 = invalid $list ID (-6 used for consistency with some Sirius functions)
- 0 = Success
- >0 = The highest message RETCODE issued during the processing of a synchronous daemon.
With negative (error) return codes the request continues with an informational message issued. With all nonzero return codes, $DPT_STATUSD contains the text of the corresponding message. If the return code was positive this will be a message that was issued on the daemon thread. The calling code can choose to regard return codes greater than zero as errors or not as required.
There is currently no direct $function equivalent to the =SPAWNINFO and =SPAWNPRINT commands, but you can easily $COMMBG one of those commands.
$THREADCODE
This function works similarly to $JOBCODE, but affects the calling user's own high-water-mark instead of the system-wide one. One reason a separate function is used is to reflect the fact that, unlike Model 204, DPT only has one executable for both batch and online mode (see also differences in messaging).
This function might often be used within a web application, since the web server's automatic error processing depends on the internal variable accessed by this function. It can be useful to know in advance whether that processing is going to kick in, or to deliberately cancel or trigger it.
- $THREADCODE(new-high-water-mark [,text])
- The function returns the old value of the user's MSGCTL retcode high water mark.
- The new value can be left off, in which case no change is made.
- Unlike $JOBCODE, this function can reduce the current high water mark as well as increase it, most commonly down to zero.
- An optional text message can be supplied along with the new retcode high-water-mark. This will be reported as if it were the text of a regular error message which had caused the HWM to be set. For example on the web server automatic error page, or with spawned daemon thread results wherever they are shown.
$TOKEN
This and its variants $TOKENS and $TOKENIZE are general purpose functions, particularly useful when manipulating string data formats such as comma-separated records and text containing CRLF sequences. They are something like more elaborate versions of the Model 204 $WORD and $WORDS functions (with differences covered below).
In all cases, the "tokens" within the input string must each be less than 64K characters long. The input string itself can however be any length up to the current STRINGMAX, so for example you can parse text lines out of BLOB field values or files read in using $FREAD.
- $TOKEN(string, separator, index [,quote-option])
- The function differs from $WORD in the following significant ways:
- The separator can be longer than one character
- So for example $X2C('0D0A') is the ASCII CRLF sequence and can be used to process data containing multiple CRLF-separated lines.
- The default separator is a comma instead of a space
- The optional final parameter allows separators within quoted sections of the string to be ignored, and/or quotes to be removed from tokens. The parameter has bit settings as follows:
- 0 = disregard quotes (default - same as $WORD)
- X'01' = consider the single quote character
- X'02' = consider the double quote character
- X'03' = (therefore consider both)
- X'04' = Remove quotes from token
- Separators appearing within quoted sections of the input string do not count
- So if we select quote option 2, $TOKEN('Abc ; Def="Gh;Ij" ; Klmn', ';', 3) returns 'Klmn' and not 'Ij" ' because the semicolon inside "Gh;Ij" did not count.
- When quote option 3 is used each quoted section must begin and end with the same type of quote, but may contain the other type within it.
- Using the X'04' bit means the returned string has surrounding quote characters removed from it, and saves you doing a $STRIP or similar on the results in common cases such as reading a .csv file.
- For example $TOKEN('"JOHN","SMITH"',,2+4) returns 'SMITH' instead of '"SMITH"'.
- Other Miscellaneous notes
- If you are doing a lot of access to the same piece of data using this function it would be much more efficient to read it into a $LIST first - see $TOKENIZE below.
- See also the custom options on $STRIP.
$TOKENS
- $TOKENS(string, separator [,quote-option])
- Differing from $WORDS in the same way that $TOKEN differs from $WORD (see above detailed notes).
- So the return value is the maximum index that could be given to $TOKEN and get anything back.
$TOKENIZE
Following on from the above, this function does a one-off pass through a string and places the tokens into $LIST items. This is more efficient than a lot of scans to locate individual tokens. This function might be thought of like a generalized version of Sirius $PROCDAT.
- $TOKENIZE(string, separator, $list ID [, quote-option])
- See notes above about the handling of separators like CRLF.
- The return value is the number of tokens, or -1 (invalid list ID).
$UNLOAD
This $function is equivalent to the =UNLOAD command, but offers some extra functionality, the most important part of which is that you can unload record subsets.
- $UNLOAD (filename, what, field-spec, options, label-or-list-name)
- By default the current file context is used, or you can override it with a filename parameter and/or a foundset label or list name.
- In case of lists and foundset labels with the same name, you can prefix a list name with the keyword LIST. Otherwise the foundset will be used.
- When a record subset is specified, this determines both the contents of the data extract and the inverted lists in any index extract(s).
- Parameters 2-4 should contain string expressions as per the equivalent parts of the =UNLOAD command, except that no brackets are required round the options, and a keyword is not required to introduce
any field names.
- In the event of a record locking conflict (needs SHR), the function gives a return code and does not invoke the UL ON RLC unit. To make that happen, create a locked found set first and give it to the $function.
- In other respects the processing of this $function is the same as for the command, since at runtime it hooks into the same internal routines. See the command notes for further info.
- Return codes
:
- 0 = success
- Positive = Error during unload processing - the audit trail may contain more messages saying what, especially if LOADDIAG is active. The request will continue if possible although it depends on the seriousness of the error.
- -1 = No current file context, or invalid supplied context. The request continues with an error message issued.
- -2 = Invalid parameter. The request continues with an error message issued.
Examples:
B
%X = $UNLOAD
%X = $UNLOAD (%FILENAME, %WHAT, %FIELDSPEC, %OPTS, %SETNAME)
%X = $UNLOAD (,'ALL INFORMATION')
%X = $UNLOAD ('SALES',,'AGENT_NAME')
%X = $UNLOAD (,'INDEXES','EXCLUDING AGENT_NAME, AGENT_DISTRICT')
%X = $UNLOAD (,'DATA',,'PAI REPLACE DIR="C:\MYEXTRACTS"')
%X = $UNLOAD (,'FDEFS AND INDEXES',,'EBCDIC NOFLOAT','LIST JULY.SALES')
END
Set C: Web server related $Functions
Finally the set of functions used specifically for writing web server scripts. Like Set B, these are also installed as standard, and fully documented. They are grouped separately just for clarity of documentation.
Request information $functions
$WEB_GET_URI_PARM
Retrieve the values of parameters passed in after the question mark on the request URI. Note that the browser will replace spaces in the parameter names/values with a '+' and/or other special characters like quotes and ampersands with %hexhex (e.g. %20 = space). These are re-converted by DPT before they hit your code and you need not worry about them. DPT can also optionally case translate both the parameter names and values.
- $WEB_GET_URI_PARM(parm-name [,index])
- Returns the value of the named URI parameter.
- To retrieve very long values, increase STRINGMAX.
- Since a URI may contain several occurrences of the same-named parameter, you can specify an optional 'index' to retrieve values after the first. The default index is 1, thus giving the value of the first occurrence.
- If there was no URI parameter with the specified name and index, the function returns a null string.
- The special value for index of 'COUNT' returns the number of parameters matching the supplied name. If the name is omitted or null, the total number of parameters is returned. This can be handy to set up a loop when you don't know what parameter names you are expecting.
- $WEB_GET_URI_PARM(index [, 'VALUE' | 'NAME'])
- The function works differently if the first parameter to the function is given as a number. In this mode the parameters are considered as a sequence of name=value pairs referenced by 'index'.
- Depending on what you give as the second parameter, either the name or value of the pair is returned, or a null string if 'index' does not refer to an existing pair.
- The special value of index=0 returns the entire string after the question mark in the URI. This can be handy for information when developing input forms.
- $WEB_GET_URI_PARM(x,y,'SSI')
- An optional third parameter can be given as the string value 'SSI' in either case above to alter the behaviour when the script is part of an SSI page.
$WEB_GET_POST_PARM
Returns information from the posted data block, if that method is used instead of URI parameters. The function works almost identically to the one above, so if an input page using URI parameters switches to the POST style, the host-side procedure could switch to this $function with no other changes. The browser handles the encoding, and DPT handles the decoding.
This function also extends the URI version with a couple of extra parameter options reflecting the fact that POST parameters might not be simply name=value pairs. Specifically this means when a "TYPE=file" control is used as an HTML input form element, in which case DPT tries to obtain some extra information from the multipart encoded block.
- $WEB_GET_POST_PARM(parm-name [, index [, 'VALUE' | 'TYPE' | 'FILENAME' | 'FILELEN'] ])
- $WEB_GET_POST_PARM(index [, 'VALUE' | 'NAME' | 'TYPE' | 'FILENAME' | 'FILELEN'])
- Handling of 'parm-name' and 'index' is exactly the same as the URI function above. The special value of index=0 returns the entire undecoded multipart data block, which might be useful if you are using unusual or custom block layouts.
- The extra options 'VALUE', 'TYPE', 'FILENAME' and 'FILELEN' for the final parameter are for use with "TYPE=file" form controls. 'VALUE' returns the contents of the file. 'TYPE' returns the MIME data type if one was sent. 'FILENAME' is the suggested file name if one was sent (usually the client-local file name). 'FILELEN' returns the number of bytes that were sent as file data, excluding multipart boundary terminators and other control information. (Comparing the 'FILELEN' to the $LEN of the receiving variable for 'VALUE' is a good way to ensure that no truncation occurred when assigning a very long value to the receiving variable).
- With all other input control types apart from "TYPE=file", 'VALUE' has the same meaning as omitting the final parameter. 'TYPE', 'FILENAME' and 'FILELEN' return null strings in those cases.
$WEB_GET_HEADER
Access header information that came with the current HTTP request. See also the background discussion on the format of HTTP message headers, and the specialized cookie function below.
- $WEB_GET_HEADER(header-name [, index])
- $WEB_GET_HEADER(index [, 'VALUE' | 'NAME']))
- Handling of 'header-name' and 'index' is exactly the same as the URI function above, except that the special value of index=0 returns a null string.
- If for some reason you need to access sub-parts of a header which are delimited by, say, semicolons, you can use the $TOKEN and/or $STRIP functions.
- $WEB_GET_HEADER('URL' | 'METHOD' | 'VERSION')
- This function also provides as "pseudo" headers 3 items of information on the request message title line. METHOD returns 'GET'/'POST'/'HEAD'. URL returns the requested resource name, without any '?.....' suffix. VERSION returns the HTTP version - usually '1.1'.
- These three can not be requested by index number since they do not feature in the list of "regular" headers.
- Should you for some reason use a "regular" header with any of these names, that value is returned in preference to the pseudo header.
$WEB_GET_CLIENT_ADDRESS
Gives the TCP/IP address to which the response will be sent for the current HTTP request - usually the browser address. Depending on the make of browser, this information may also be available in one or more of the HTTP headers. Under certain conditions a web script might use this address to open another socket connection back to the browsing computer to achieve application functionality outside the browser.
- $WEB_GET_CLIENT_ADDRESS
- This function returns a null string if it's called from a non-web-script setting, and is therefore one way to test whether that is the case.
$WEB_GET_COOKIE
This function can be thought of as a specialized version of the above, for retrieving values from the "Cookie" header(s) (see general cookie notes). It saves you having to write parsing code to unpick the name=value pairs from the header, as well as shielding you from the somewhat variable incoming formats this header can take.
- $WEB_GET_COOKIE(cookie-name [, index])
- Handling of 'cookie-name' and 'index' is exactly the same as the first form of the URI function above. (Note therefore that technically you could count all cookies either with $WEB_PUT_HEADER('COOKIE','COUNT') or $WEB_PUT_COOKIE(,'COUNT'). The latter is recommended.
- No case translation or other conversion is applied to incoming cookie values, regardless of WEBFLAGS. (See more detailed comment re FUNC_WEB_PUT_COOKIE).
$WEB_PATCH_URI_PARM
$WEB_PATCH_POST_DATA
$WEB_PATCH_HEADER
These functions are all used similarly, and are provided to support coding and testing activities. Under certain circumstances it's also possible you might find a use for these even in live code, perhaps to set up default parameters. Place the calls at the top of the handler procedure that would normally be expecting parameters. For example:
B
$WEB_PATCH_URI_PARM('NAME', 'SMITH')
END
* Main handler code
B
%NAME = $WEB_GET_URI_PARM('NAME')
* etc.
END
- $WEB_PATCH.....(name, value)
- Adds a parameter or header to the internal object representing the current HTTP request. The return code is the number of such objects now defined.
- In all 3 cases, the object names need not be unique, so calling these functions add a new occurrence regardless of whether the named object already exists.
- If 'name' is supplied as null, no action is taken. On the other hand a null 'value' is perfectly reasonable.
- Note that the values patched in will not feature in the untranslated URI or POST data block, so an index=0 call would not include them.
- No uppercasing of names or values is performed by this function, as would normally be invoked according to WEBFLAGS.
- To insure against forgetting to take these function calls out of code when you finish testing, it can be an idea to put them inside a conditional block so they only run at a terminal. For example "IF $VIEW('IODEV') EQ 7 THEN..." or "IF $WEB_GET_CLIENT_ADDRESS EQ '' THEN...".
- As with GET_URI, the PATCH_URI function can also be given the optional string value 'SSI' as parameter 3.
Response information setting $functions
$WEB_PUT_RESPCODE
Changes or reports the code which will be sent in the HTTP response message title line.
In an error situation, if your code has retained enough control to be calling this function, it should arguably not be considered an HTTP error at all, and the default response of 200 should be left in place. The way in which browsers handle the code you set must in any case be kept in mind, as discussed in the general coding notes.
This function would then be reserved for more appropriate and specialized situations like redirects, not-modified, locking problems, etc., where specific browser behaviour is actually desired.
- $WEB_PUT_RESPCODE(response-code)
- The return value is the pre-change response code.
- When called with a negative or missing 'response-code', the function makes no changes (i.e. just reports the current value).
- If the code you use has a conventional meaning, DPT will append the conventional accompanying phrase ("Not Found", "Forbidden", etc.) to the HTTP message title line. Otherwise a generic phrase applying to the range the code is in will be used - in the case of codes above 510 this is "Server Application Error". Otherwise you have no control over the content of the title line phrase. To convey more detailed diagnostics, compose appropriate content into the HTTP buffer.
- If automatic error handling occurs after you call this function, the code decided upon by the handler overrides yours. Therefore it can be a good idea to issue $THREADCODE(0) at the same time as this function.
- If you never call this function the default is 200 ('Success').
$WEB_PUT_DATATYPE
This is the recommended way of accessing the response's "Content-Type" header. It also allows you to use a file extension as shorthand:
$WEB_PUT_DATATYPE('text/css; charset=UTF-8') /? wordy ?/
$WEB_PUT_DATATYPE('CSS') /? tidier ?/
- $WEB_PUT_DATATYPE( {mime-value | file-extension} )
- If you never call this function the default value for the Content-Type header of a dynamic page is 'text/html'.
- The single parameter is interpreted either as a literal MIME type value or a file extension, as per the above two examples. If it contains a slash, it is considered to be the former, and used as supplied. Otherwise the system looks it up in the file extensions vs. MIME types reference table to see if it is a defined extension. If so, the corresponding MIME type is used. If not, the value is assumed to be invalid and the Content-Type header is cleared.
- Specifying a null string (or invalid extension) means that the Content-Type header will not be sent, and it will be up to the browser to do what it can with the data. You might also clear the content type for tidiness if sending no data at all, such as with a 'redirect' or 'not-modified'. It is not recommended to clear the header using this function and then set one the long way round using $WEB_PUT_HEADER, although that does work.
- The extension supplied can have an optional dot prefix (e.g. .txt) and can be supplied in mixed case.
- The return value is 1 (extension lookup successful) or 0 (any other outcome).
$WEB_PUT_HEADER
Add or change an HTTP header on the response message currently being constructed.
Note that some headers are automatically added to your response message, and you should not set those yourself (the browser would probably ignore any you might set anyway). "Content-Type" and "Set-Cookie" are probably the two headers you would most often have good reason to tinker with, but they are catered for by $functions with specialized options (see below). This basic function is therefore most relevant to the occasional use of of headers such as "Modified-Date" (e.g. to control browser caching) or "Location" (e.g. during a redirect).
- $WEB_PUT_HEADER(header-name, value)
- Case-translation is optionally applied 'header-name' and 'value', according to the WEBFLAGS parameter.
- If 'header-name' is supplied as null, no action is taken. On the other hand a null 'value' is perfectly reasonable.
- The return code from this function is the number of headers now defined on the response.
$WEB_PUT_COOKIE
Adds a new "Set-Cookie" header to the response currently being built (see also general cookie notes).
Like $WEB_PUT_DATATYPE above this function is one you could do without if you tried hard enough, but your code will be neater and more readable if you use it. In a single call it constructs the potentially awkward value for the "Set-Cookie" header and adds it to the response.
- $WEB_PUT_COOKIE(name, value, [expires], [path], [domain], [custom-info])
- The return code of the function is the number of Set-Cookie headers now defined on the current response.
- If the name is supplied as a null string, the function does not set a new cookie.
- Each cookie added by this function is a separate instance of the "Set-Cookie" header. That is, DPT does not put more than one cookie in the same header (an alternative technique used by some web servers).
- The value may be a null string (the default).
- Commas and semicolons have special meaning to the HTTP protocol, so they are replaced in cookie values by %HH encodings, which are translated back again by the cookie retrieval $function. This should be transparent to your code.
- The 'expires', 'path' and 'domain' parameters correspond to those standard subparts of the Set-Cookie header, and are all optional. Omitting these parameters may or may not have the same effect as using a null string, as follows.
- expires: There is no default for this parameter - the option is only added to the Set-Cookie header if you supply a value. If you supply a value it must be given in 'UTC' format (e.g. $NOW+24*3600 specifies this time tomorrow). The browser will take a missing value to mean that the cookie should be deleted when the user closes the browser. A value in the past (e.g. 0) can be used to delete a cookie immediately.
- path: This parameter has a default value of the current URI. In other words if you omit the parameter DPT appends e.g. "; path=myscript.dptw" to the Set-Cookie header. This is different from the usual default behaviour of browsers, where omitting this option from the cookie is taken to mean that it applies to the whole site. Making a cookie that applies to the whole site is achieved explicitly with the commonly-used path value of '/' (forward slash).
- domain: There is no default for this parameter - the option is only added to the Set-Cookie header if you supply a value. Explicit null string is an invalid cookie.
- Other cookie options: The final parameter to this function allows you to set any other less common cookie options such as "max-age".
- Since the current version of the DPT web server has no explicit support for secure connections this option is not provided, but you could easily supply "secure" as the custom-info parameter if you saw a use for that.
- Despite the fact that the cookie name, value and options are all technically part of the "value" of the "Set-Cookie" header, this header is treated as a special case and the data is immune to case translation triggerred by the WEBFLAGS parameter. The philosophy here is that this header is entirely under the control of the host-side code in your application, so the information should be set, and later retrieved by FUNC_WEB_GET_COOKIE, exactly as you specify. Note that the browser may be more fussy about the case of the "path" option in cookies than it is about the case of URIs in general, and it is possible to set up links that work but don't pick up their cookies as expected because of a case mismatch.
Response data buffer access $functions
$WEB_PUT
- $WEB_PUT(value)
- The value is a string, and can be of any length. Reset the STRINGMAX parameter to use values longer than 255 bytes.
- It is appended to the current HTTP buffer contents with no translation or additions.
- The return code of this function is always zero.
$WEB_PUT_FILE
Using this function allows standard page elements to be combined in a slightly different way to using server side includes.
- $WEB_PUT_FILE(filename-expression [, sub-delimiter] )
- The file is identified by its OS file name, which can be given as an absolute path (e.g. C:/...) or relative. In the most likely case that you use the latter (e.g. 'stdhtml/banner.html') the file location is assumed to be relative to WEBROOT, (e.g. 'dpthost-base/WEBROOT/stdhtml/banner.html' in this case).
- As an aside, note that this is slightly different to the case where you code an inline image (IMG SRC=....) in HTML, or a SSI #include file, where the location is relative to the location of the current page.
- Symbolic substitution
- If a value is given for the 'sub-delimiter' parameter, a find-and-replace operation is performed on symbolic parameters in the file data before copying it to the HTTP buffer. Otherwise the contents of the file is efficiently copied with no changes.
- Symbolic parameters are indicated in the file by wrapping them before and after with 'sub-delimiter'. For example if 'sub-delimiter' were supplied as '@@', then '@@DATE@@' in the proc would denote a symbolic parameter named DATE. The parameter name can contain any characters, and is simply the string between the two delimiters.
- Substitute text values are retrieved from the global table, where the parameter name (e.g. 'DATE') is used as the global name.
- If the named global does not exist, there is no further fall back to other substitution methods such as prompting, as there would be with dummy string processing. The symbolic parameter is left in place in the text. To get it replaced with a null string you have to explicitly supply a null string global.
- Return codes
0=Success
1=Error (see $STATUSD for details)
$WEB_BUFFER_SIZE
- $WEB_BUFFER_SIZE[(newsize)]
- Returns the current number of bytes in the HTTP response buffer.
- The optional parameter is a number which lets you change the size of the buffer. If the buffer is larger than the number you give, it is trimmed back, but if it's already equal or smaller, there is no 'padding-out' effect.
- The default value for the parameter is -1 which means don't resize the buffer.
- Clearing the buffer (newsize=0) is most useful when an error occurs, to wipe whatever you were building and send some kind of formatted error page.
- It could also be useful when you belatedly decide to send no content at all, perhaps in a redirect message like 301 (Redirect).
- Resizing the buffer to a non-zero length might in theory let you "undo" a certain amount of work and proceed differently (although I wish you luck if your code is that complicated!).
Control and miscellaneous $functions
$WEB_FINISH
- $WEB_FINISH([flag])
- Terminates the handling of the current dynamic HTTP request and initiates the sending of the response data built so far, or an error page as appropriate.
- The optional parameter is a number which defaults to zero. Any non-zero value means that if the current script is running in an SSI situation the output from the script is in turn parsed for SSI directives. Yes, recursion.
- Note that if this is issued in a User Language request which is followed in its procedure by one or more commands and further UL requests, those following operations will *NOT* get performed. In other words it is somewhat more "potent" than the UL STOP statement which is effectively just a jump to the END statement. Put another way, this function triggers an immediate return to command level, which is what normally signals the end of the handler procedure.
- After calling $WEB_FINISH the automatic error handling is invoked if appropriate. To disable it use $WEB_AUTO_ERROR(-1) - see below. To force the error handling to kick in, call $THREADCODE before $WEB_FINISH.
- The return value from this function is unspecified. No further code gets run after calling it, so there would be no way to test it anyway.
$WEB_USE
- $WEB_USE( 'ON' | 'OFF' | 'BOTH' | 'HISTORY' | 'HISTSTART' | 'HISTSTOP')
- The parameter and return value are string values.
- The return value is the pre-call setting. If you call the function with a missing or invalid parameter it just queries the existing setting - no new value is set. Hence there is no default option.
- 'ON' means that all the things that would go to the normal output destination go instead to the HTTP buffer. You can use this to get existing code which uses the PRINT statement to generate 'text/plain' format content without having to change that code at all. It is also the simplest way to set up a "remote command line" style web page.
- 'OFF' means normal terminal output goes to the daemon's normal output destination and WEBPRINT/$WEB_PUT to the HTTP buffer. This is the setting that is in effect at the start of every script.
- 'BOTH' means normal terminal output goes both to the HTTP buffer and to the normal destination (the daemon's output file). Possibly not very useful.
- The destination of USE output routed by the M204 USE command is not affected by calls to this function. In other words even if $WEB_USE is ON, PRINT output will go to a USE DD in preference if one is open.
- Note that, unlike the USE command, once $WEB_USE is turned on it does not automatically turn off again after the next command etc., but remains on until an explicit $WEB_USE('OFF') or until the HTTP response is finally sent (e.g. at $WEB_FINISH).
- The 'HISTxxx' calls do not affect the ON/OFF output routing status:
- 'HISTORY' does a one-off restropective dump to the HTTP buffer of a copy of the script output history, producing an effect the same as if $WEB_USE('ON') had been called right at the start. Under certain circumstances could be handy if there was a really serious error, or if your application has a "send detailed diagnostics" switch somewhere.
- To enable the HISTORY call, the daemon must have been maintaining an output history for the current script, which by default is not the case, since this would usually be a waste of processing and memory. The HISTSTART and HISTSTOP calls turn on and off the recording of terminal output history, so if you want to reserve the option of calling HISTORY at some point during the script, just call HISTSTART at the top of the script. It's a minor overhead in the grand scheme of things.
- Each HISTSTART call clears any previous output history log.
- At the start of each script the output history log is empty.
$WEB_AUTO_ERROR
Enables or disables the automatic error processing.
- $WEB_AUTO_ERROR(trigger_retcode)
- The parameter and return code are numeric values.
- The value for 'trigger-retcode' is the MSGCTL retcode high water mark that has to be *exceeded* to trigger the auto error processing.
- The return value is the pre-call setting. If you call the function with a missing parameter it just queries the existing setting - no new value is set.
- A setting of -1 turns automatic error processing off (although values over a hundred or so would usually effectively turn it off since by default there are few messages with codes higher than that).
- If you never call this function the default value is 0, meaning any non-zero message retcode will trigger the auto-error processing. "Perfect" application code would in theory issue no error messages, but in some cases you may be happy with the occasional retcode=4 message. In such cases consider whether it would be better to MSGCTL the individual message(s) down rather than upping the $WEB_AUTO_ERROR threshold.
Part 3: Sirius Features
Firstly a brief note about what is meant here be "Sirius Features". In the same way as DPT is code written to work like Model 204 rather than copying or stealing the Model 204 code, these "Sirius" features are not based on actual Sirius code. And none of them have, at the time of writing, been contributed to the DPT open-source project by anybody related to Sirius Software, so fans of their stuff will unfortunately have to make do with these imitations!
All these features are considered part of the core product for emulation purposes, so are not disablable with the CSTFLAGS parameter.
Sirius $functions
General notes
> > > > List of emulated functions > > > >
The Sirius functions are considered part of the core M204 emulation, and are not disablable with the CSTFLAGS parameter like all the other custom functions.
They are all, like all $functions in general on DPT, callable.
$Lists
DPT will by default allow list items of any length, but for Sirius compatibility it can artificially impose a limit of 6124 bytes if you set the CSTFLAGS X'04' bit. The $LIST_MAXIL function, as on M204, tells you which maximum is currently in effect. Some functions test or otherwise use the current maximum, for example many return code -7 to indicate an invalid length parameter. In all cases the processing that takes place corresponds to the maximum currently in effect.
A $list on DPT is implemented by mapping more or less directly to the C++ standard library type "vector<string>", which in classical data structure terms means a simple array of pointers to strings. An alternative might have been to use a linked list (as suggested by the name "$list"), but since the most common type of accesses are to append items to the end and to read them by index number, an array is much more suitable. Only the less-common operations of inserting and removing items would be helped by using a linked list. On Sirius a $list is a custom data structure, and something (judging by the documentation) of a hybrid between array and linked list. Without more information it's hard to say more about its performance characteristics, except that there will be differences from DPT. Where the Sirius manuals discuss CCATEMP page fragmentation after using some $functions, the same issues will not apply on DPT.
"sdaemons"
One or two of the Sirius $functions call out to special background users which sit there permanently waiting to service such requests. On Model 204 they are called "sdaemons" ("s" presumably for "Sirius"). On DPT they are IODev 99 daemon threads very similar to those used for e.g. checkpointing and the web server.
These daemons were introduced for the same reasons as the web server daemons, namely that in most cases each item of work they are asked to do is small in comparison to the overheads of setting up a fresh user thread. This should be kept in mind when submitting a task via e.g. $COMMBG: if you expect the task to be long, using $SPAWN instead to create your own dedicated thread would often be more considerate to other users, and better for the throughput of the system in general.
The number of these daemons running at any time is governed by the FUNNDAEM parameter. The control loop is run by the =PSTFUN command.
Long strings
There is no need for an explicit "longstring" type on DPT, since any string variable can have any length so long as you set the STRINGMAX parameter high enough.
This factor is also relevant when it comes to determining "valid" results in some of the $functions below. For example $SUBREP is officially documented to fail if the result would exceed 255 characters long, but on DPT the failure point is the current STRINGMAX value.
Emulated Sirius $function differences/comments
The following comments about a few of the functions may be of interest. You should consult your Sirius documentation for details of standard operation.
Return codes
In most cases DPT creates the same return codes as the Sirius equivalent functions. One difference is that DPT never gives a specific code for "CCATEMP full", since CCATEMP is not being used.
Sirius wildcard parameters
Whenever a function takes a Sirius-style pattern (e.g. $LIST_GLOBAL_DEL), DPT allows standard M204-style patterns, and in fact converts any Sirius-style to M204-style before processing. Basically this means query = plus and doublequote = pling, so for example 'J*".SM?TH' is processed as 'J*!.SM+TH'.
Character set (ASCII/EBCDIC)
Unless a function explicitly relates to ASCII and/or EBCDIC (like say $E2A where the meaning is clear), the characters are assumed to be ASCII in the Sirius functions. For example $HEXA converts a hex string (X'616263') to ASCII ('abc'). See also general UL notes on this issue.
$COMMxx
The output of an asynchronous call goes not only to the audit trail but to the daemon's output file (see general daemon notes), which can save unpicking it from other system activity if you want a copy for some reason later.
$IMGOVL (and $LISTFINDI/$LISTOVLI when a value is given)
See CSTFLAGS fudge switch X'8000' for how errors during the assignment are handled.
$LIST_CAPTURE
OUTMRL *is* respected, but because of the difference in meaning on DPT, this should not cause problems.
For a note on pagination behaviour see CSTFLAGS fudge switch X'2000'.
Output capture is cancelled if the list is global and is deleted via $LIST_GLOBAL_DEL.
$LISTADDI, $LISTINFI and similar
These functions directly access data in the image buffer, rather than applying all the complexities you get during READ and WRITE IMAGE, such as the population of default item values and the hokey behaviour of DEPENDING arrays. The use of $lists with images is mainly intended to provide an easy way to work with columnar data, rather than to simulate file I/O to $lists as an alternative to files.
$LISTINF
To help cater for the ambiguous error-reporting convention of this function (list items containing e.g. the string "-9" might be confusing) $STATUS2 is also set to the same negative number whenever there is an error, or zero otherwise. This is a DPT custom extension.
$LISTSORT
FLOAT 16 keys are treated as FLOAT 8. This agrees with image behaviour where the upper 8 bytes of a FLOAT 16 item are undefined.
The FI key type is assumed to mean SIGNED BINARY data, but will work for UNSIGNED so long as the most significant bit is not used. E.g. LEN 4 only values up to 2^31.
The PD and ZD key types will work with either SIGNED or UNSIGNED data or any mixture.
List items containing invalid data in the positions for sort keys of numeric formats like packed decimal always sort "high" regardless of the specified direction. This also applies whenever a sort key lies entirely outside the list item data. This behaviour was chosen to be like the SORT RECORDS statement.
$PROCGET
On DPT this function also sets $STATUS2 to get round the ambiguity between documented meanings for the null string result (is it EOF or just a blank line in the proc?) $STATUS2 is set to 0 (OK), 1 (EOF) or 2 (error).
$RANDOM
The result is by default an integer, but see CSTFLAGS fudge switch X'4000'.
See also custom variations DPT_RANDOMXXX later.
$RANDOM_SEED
The variable need not be 140 bytes long. Any numeric variable, or string capable of holding a 10-digit integer is OK.
Sirius compiler directives
Sirius directive
The only option currently supported is the case option for temporarily overriding the active mixedcase UL mode until the end of the current procedure.
Sirius Case {ToUpper|Leave}
This directive is entirely case-insensitive. Note also that although the DPT editor will recognize these keywords, they do not affect colour-coding or uppercasing of code.
Precompiler macro language (!...)
Pling directives are not currently supported on DPT.
Mixedcase User Language
The following Sirius-style features are supported on DPT to enable mixedcase User Language programming.
- BEGIN command in mixedcase starts a mixedcase request.
- COMPOPT parameter (only X'01' and X'02' bits - the X'04' bit effect is not supported on DPT).
- Sirius case compiler directive.
Editing mixedcase code on DPT
When writing mixedcase User Language in the DPT client editor, you will almost certainly want to turn off the option which makes it uppercase whole lines as they're entered (as you would enter "*LOWER" mode) on M204.
On the other hand, if you're sticking with the convention of uppercase language keywords (OCC, SCREEN, $function names etc.), leave that option active, since having the editor auto-uppercase and colour them is a good way to encourage adherence to that coding standard. Remember this is just a graphical feature of the editor, and having it active does not affect how the compiler on the host might treat mixedcase keywords at any given moment. BEGIN/B is recognized in mixedcase regardless of editor settings.
See also: full details of case-handling options in the DPT editor.