A Proposed Convention for the Extended use of the Name Attribute

Tags: #<Tag:0x00007f31d0952208>

A Proposed Convention for the Extended use of the Name Attribute

By Bruce Johnson
Version 1.01 : 4 September 2019

1 Synopsis

By extending the use of the NAME attribute on fields, we can make reflectors more powerful.

2 Acknowledgements

This proposal is based on an idea proposed by Eric Lankreijer, a software engineer at RADventure (www.radventure.com).

Thanks to Eric Lankreijer , Mike Hanson, Andy Wilton, John Hickey, Mike Duglas, Gus Creces and Arnor Baldvinsson for reviewing drafts of this document and suggesting improvements.

3 Background

Clarion includes a number of commands which are useful in reflecting structures for use in generic code. WHO, WHAT, WHERE, ISSTRING, ISGROUP and HOWMANY all play a part in understanding a structure. By understanding the structure it's possible to perform generic code on that structure.

4 Limitations

While the above commands are useful, they are limited. It is not possible to determine a field type (beyond it being a STRING / GROUP / something else) and complex types (pointers to Queues, Strings, Objects whatever) are completely unsupported.

In addition, other attributes, defined in the dictionary, but not used in the data declaration are not visible. For example a LONG field would be noted as being “not a STRING or GROUP”
but it would not be clear that the field contained a date, or what picture format that date typically takes, or even what range of numeric values are acceptable.

ISNUMERIC is not useful in reflection as it only determines if the contents of the field are numeric or not, and not the type of the field.

5 Tufo structure

An undocumented structure is declared in tufo.int and can be used to help with reflecting. An example of tufo.int can be found at
https://github.com/MarkGoldberg/ClarionCommunity/blob/master/CW/Shared/Src/TUFO.INT

While this can help extend the usefulness of WHAT, using Tufo is complicated and remains undocumented. What is known about it has been pieced together unofficially over the years. As an undocumented structure it is also subject to change (although it does not appear to be changing.)

Even with TUFO though it's not possible to glean other information, declared in the dictionary at runtime.

6 Field Name Attribute

In the Clarion language each field declaration can contain a NAME() attribute. This is usually used in TABLES to indicate an external name for a field, one which determines the field name in the external database.

The NAME attribute is also legal in any other field declarations, including groups, local and global variables, and object properties. It should be noted that it is not used in VIEW structures, on PROJECT statements as these are not field declarations.

The use of the field name has already been extended in Clarion itself. Attributes to a field can be added to the external name of a field in the dictionary. These attributes are separated from the actual external name with a space-then-a-pipe symbol. For example
field    long, name('field |READONLY')
Currently code which makes use of the WHO command, to get the NAME attribute for a field, needs to remove anything after the first pipe command when parsing the actual field name. For example an xml export routine, which uses the NAME attribute to identify the field in the output, would need to remove this extra information.

In current Clarion if there are multiple attributes, they need to be space-pipe separated. For example;
field    long, name('field |READONLY |NOWHERE |BINARY')
Other code can use this information to change behaviour. For example the Clarion CREATE command treats fields with |READONLY differently to fields which do not have that modifier.

6.1 Currently known Clarion Attributes

The known, documented attributes are
READONLY, NOWHERE, BINARY, CHECKFORNULL, 
SELECTNAME=Value, UPDATENAME=VALUE, INSERTNAME=value
They are case sensitive. Other, undocumented, attributes may exist.

7 Proposal

The NAME attribute for all fields would become a pipe-separated-list that can be used to store multiple attributes, which in turn are all available to reflecting code using the WHO command. The use of this technique would extend beyond simple table declarations. It could also be used on any structure that can be reflected, including table records, queues, objects and groups.

7.1 Scope of this proposal

This proposal does not address how the NAME attribute may be populated by templates. Ultimately templates may populate the name attribute in the future, but that is outside the scope of this document.

7.2 Changes required in Clarion

No changes are required in Clarion for this proposal to work. As mentioned earlier Clarion already supports a pipe separated list for the Name attribute. File drivers already ignore attributes they do not support.

7.3 Prop:Name

The piped attributes only appear in the result returned from the WHO function. They do not appear when using
{prop:name}
By implication this technique can not be used on individual local or global variables.

8 By Convention not a Requirement

This proposal does not require a program, or accessory to use this syntax. The NAME attribute could be extended in any way the program desires, as long as the attributes are recognised by the code that is using them.

This proposal is merely a suggested convention which, if generally adopted, would allow programmers consistency in the way they code, and tool writers consistency in the way the name attribute is parsed.

It should be noted that tools are under no obligation to change behaviour based on any attribute.

9 Attributes

Information included in the NAME would be optional but could include the following;

9.1 The first attribute is always the field name.

It is a condition of using WHO that the first attribute in the list should always be the actual external name. This attribute is not optional if any other attribute exists.

9.2 Optional attributes

All other attributes, other than the name should be considered optional whenever possible. Some specific parsers may require that some attributes are required.

9.3 Order not important

The order of the attributes should be considered as unimportant. With the exception of the actual external name which has to be present, and has to appear first.

9.4 Case Insensitive

The case of all attributes should be case insensitive, excluding

9.4.1

the actual field name in the first position in the list and

9.4.2

any key(value) attributes where the value is a string.
Note that some tools may require Case Sensitive attributes. For example the native Clarion SQL drivers make use of the attributes READONLY, BINARY and NOWHERE and these are Case Sensitive.

9.5 Ignore whitespace

Whitespace before and after the pipe character should be ignored. Thus
| readonly
and
|readonly
and
|readonly |binary
should be interpreted in the same way.

10 Defined Attributes

All of the attributes defined below may optionally be implemented by a parser. Unsupported attributes must be ignored. This section is not to determine what MUST be implemented. Tools should however implement these attributes as described below. If alternate functionality is required, use an attribute other than the list below.

10.1 Clarion File Driver Attributes

The following attributes are used by the Clarion File Drivers.
BINARY, READONLY, NOWHERE

10.1.1 Known data types

Known simple data types should be considered as reserved words, and must not be used for product-specific custom attributes.

The known data types are

ANY, MEMO, BLOB, GROUP, BYTE, BOOL, BFLOAT4, BFLOAT8,
DATE, TIME, SHORT, USHORT, LONG, ULONG, SIGNED, UNSIGNED, 
REAL, SREAL, DECIMAL, PDECIMAL, STRING, PSTRING, ASTRING,
BSTRING, CSTRING, USTRING, WORD, DWORD, BOOLEAN. 

10.2 Type from the field declaration.

The data type could be duplicated. This serves to give parsers more information than that provided by the reflecting commands (ISSTRING etc).
For example;
SomeQueue   Queue
startDate     Long, name('StartDate | Long')
notes         &String, name('Notes | &String')
total         Decimal(12.2), name('Total | Decimal(12.2)')
            End

10.3 References

All attributes starting with a & character should be considered as a reference to a data type. Product specific custom attributes (other than as references to product-specific data structures) must not begin with the & character.

In addition to all the simple data types other known types are &QUEUE, &FILE, &VIEW, &KEY, &OBJECT, &CLASS, &WINDOW and &REPORT. For example;
notes         &String, name('Notes | &String')
As a reference may be to a specific class, queue, or other type some references may be unrecognized by the parser. For example;
notes         &SystemString, name('Notes | &SystemString')

10.4 Name(Value) attributes

Some attributes are recognizable by being a single word (see list of data types above). Some are recognizable by starting with a known character, or pair of characters (see the use of & to indicate a reference above).

Some attributes are best expressed as a name-value pair. This could be anything – the concept is simply to name the parameter, and pass any data to the parameter in brackets. For an example of this see SQL Data Types below.

10.5 SQL Data Types

A number of tools assist with the creation of SQL tables based on dictionary file definitions. An attribute SQLTYPE is suggested as an attribute such a tool would use. For example;
paid         Byte,name('paid | sqltype(tinyint)')
If the type was specific to a particular instance of SQL then the attribute name could be extended;
paid         Byte,name('paid | mssqltype(tinyint)')

10.6 AS Data Types

It’s somewhat common to use one (simple) data type to store data of a particular kind. For example a Byte can be used to hold a Boolean value, or a Long may be used to store a Date. It would not be appropriate to use the “as data type” in the name, as a type. For example
StartDate     Long, name('StartDate | Date')
In the above the field is clearly declared as a LONG but the DATE in the NAME attribute would suggest a DATE type, not a LONG time. This could lead to problems for parsers that expected StartDate to be an actual Date.
To avoid this issue the AS attribute is suggested. For example
StartDate     Long, name('StartDate | As(Date)')
This serves to differentiate between the actual data type, and the nature of the data being stored in the datatype. Here’s another example
Location     String, name('Location | As(GPS)')
As always the actual text inside the attribute will depend completely on the values that the reflection code (ie the code calling WHO) supports.

10.7 Multiple Declarations

Multiple attributes may be used to indicate both a generic and specific type. For example;
notes      &SystemString, name('Notes | &Object | &SystemString')
This would indicate that the item is both an Object reference, and more specifically a reference to a specific class (SystemString). The parser could choose to make use of the generic, or specific form, depending on which is appropriate.

10.8 Picture Formats

Picture formats can be identified as an attribute that starts with the @ character. Product specific custom attributes must not begin with the @ character. For example;
StartDate     Long, name('StartDate | Long | @d3')
StartDate     Long, name('StartDate | @d3')

10.9 Scope attributes

Scope Attributes indicate the scope of the data to the parser. For example exporting tools may choose to not export fields with a scope set to private. The following are a set of suggested attribute names set aside for scope usage;
PRIVATE, PUBLIC, READONLY, WRITEONLY
Example
User  Group
Name     String(100)
Password String(100),name('Password | private')

10.10 Rename, xmlName, jsonName, etc

It may be desired to rename a field, such that a parser makes use of a name different to the external name. It is not usually possible to change the actual external name of a field in the database, however an alternate name may be preferred for export to say XML. In this situation an attribute called RENAME may be used. For example;
Invoices   File
             Record
Id             Long,name('Id | Rename(InvoiceNumber)')
This approach could be extended for even more specific export names. XMLNAME and JSONNAME are obvious candidates. For example;
Id             Long,name('Id | xmlname(Invoice) | jsonname(Inv Num)')
Exports to other named formats should follow this convention.

Note that this attribute would not be useful to actually name an external field in a database – the first item in the Name attribute is used for that so it serves no purpose to have a separate attribute for that. This would be used for exports (and imports) – not the actual database itself.

10.11 Validation Attributes

It may be desired to include attributes that determine validation rules for the field.

10.11.1 Comparisons

Attributes starting with the > character indicate the value must be greater than some value. For example
Amount Long, name('Amount | > 0')
Attributes starting with the < character indicate the value must be less than some value. For example
Score Long, name('Score | < 100')
Attributes starting with the >= characters indicate the value must be greater than or equal to some value. For example
Amount Long, name('Amount | >= 0')
Attributes starting with the <= characters indicate the value must be less than or equal to some value. For example
Score Long, name('Score | <= 100')
Attributes starting with the <> characters, or ~= characters, indicate the value must be not equal to. For example
Amount Long, name('Amount | <> 0')
Or
Amount Long, name('Amount | ~= 0')
Note in Clarion the < character in a string often needs to be encoded as <<. In the case of the above operators a single <, or a double << can be used. In both cases the WHO command returns a single <.

10.10.2 Req

The REQ attribute indicates the field cannot be zero or null.
Amount Long, name('Amount | Req')

10.10.3 Inlist

The INLIST attribute requires the value to be within a specific list.
Status Long, name('Amount | Inlist(Paid,Unpaid,Pending)')
Note that the list is a comma separated list, not a pipe separated list.

10.10.4 Validate

More complex validation could be placed inside a VALIDATE attribute. This can then be parsed, and evaluated, by the tool as required.
Status Long, name('Amount | Validate(value %2 = 1))')

11 Custom Attributes

The goal of this proposal is not to limit any tool into a specific set of attributes. It is expected that different tools may require, or desire, additional attributes.

In additional individual programmers may wish to make use of custom attributes which are unique to a specific dictionary or program.

11.1 Tool Providers

It is desired that a list of used attributes is maintained by tool makers in a suitable forum. In this way usage of common attributes can be standard-by-convention, and tools will not conflict with each other.

The location of this list should be in a publicly available space, and should be updated as required on a consensus basis.

11.2 Programmer Attributes.

The use of the exclamation point (!) as the first character in an attribute is considered as reserved, and should not be used by any tool maker. This allows individual programmers to use this as an identifier which is unique to their program. For example;
Id      Long,name('Id | !restricted ')
Total   Long,name('Total | !pretty ')

12 Document Revision History

Version 1: 23 Aug 2019
Version 1.01 : 4 September 2019
Updated paragraph 6.1,
Added paragraph 10.1.1 (which went missing when posting here).
3 Likes

Have you gotten a nod from SV that they don’t consider this to be a bug in WHO()?

See https://www.clarionlive.com/BrowseEpisodes/ww!528 for proposal and discussion.

IF Who() don’t work, have you considered the “new” Prop:Whatever ?

no.

WHO does work.

cheers
Bruce

Is there a character limit? Is it consistent against things like structure size, label length? whatever clarion version is intended support?

Hey Brahn - take both hands off the beer when typing mate :slight_smile:

I presume you are asking, which version of Clarion does this work in? My guess is pretty much “all of them”. I need to test backwards, but I’d expect it to work in at least clarion 8, and probably Clarion 5.5 and later.

I did an informal test for character limit, and it seems reasonable. 300+ I think it was. (I’ll dig out the exact number, and run some more tests on structures of various sizes…)

Cheers
Bruce

Just curious what limits you had tested or were expecting.

While the grounds of the proposal seem clear and I doubt anyone can argue against potential benefits vs cost/downside, use of the name attribute feels to me like a kludge. The obstacle, of course, is a parting of the seas before SV would gain awareness and provide accommodation through an additional attribute. Nevertheless, if anyone was to have an idea implemented by SV prior to release of Clarion.net, this proposal seems likely to be one that could be heard.

I think there is another default property (not documented as far as I know):

| CHECKFORNULL

This can be used (for SQL drivers) to make sure the view engine works well when there are null-values in the columns that are involved in the sort order of that view. When the view engine is used to locate a certain value (say for example “somefield <= ‘abc’”), sql engines by default “omit” null values. Now with this property present, the view engine generate a where clause like this “(somefield <= ‘abc’ or somefield is null)” and now the null values are in the resultset.

Actually … I couldn’t live without it …

In addition, from the docs:

The NAME attribute of a field or a blob in an SQL table can contain any of the following:

| SELECTNAME=value

| UPDATENAME=value

| INSERTNAME=value

CHECKFORNULL is also in the docs.

document updated, thanks chaps.

Added INT64, UINT64, and VARIANT to your list and sorted it.

IMO, CLASS, and the “Entities” such as FILE, INDEX, KEY, QUEUE, REPORT, VIEW, and WINDOW might be good to add too.

ANY, ASTRING, BFLOAT4, BFLOAT8, BLOB, BOOL, BOOLEAN, BSTRING, BYTE, CSTRING, DATE, DECIMAL, DWORD, GROUP, INT64, LONG, MEMO, PDECIMAL, PSTRING, REAL, SHORT, SIGNED, SREAL, STRING, TIME, UINT64, ULONG, UNSIGNED, USHORT, USTRING, VARIANT, WORD