Clarion Unicode blog post v2 - Report EMF Page Files

The new blog post is an update and links to the ‘Ustring testers guide’ with some more detailed Unicode explainers, and details on new work including the new
TopspeedW driver with 512G max file size.

the post also links to a new doc

and while it does have a lot of details for report tool authors there is also useful data for every Clarion dev

EMF Page Files — Transition Note for Report Tool Authors
Tech brief for developers who ship report-related products for Clarion: preview
replacements, export/document generators, report add-on templates, or anything
that reads report page files or implements the report generator interface. What
changed, when it appears, and the transition path for each integration
style.

Robert

  • Does FORMAT return (pushes to the string stack) an Unicode string always?
  • Do properties return (push to the string stack) Unicode strings or it depends?
  • Is USTRING a new base type now? If yes, how conversion rules for base types changed?
  • Is short conversion between base types implemented? For example, (U)STRING->LONG without intermediate DECIMAL/REAL values.

Robert,

I can assure you that I’ll have questions but I have too damn much on my mind at the moment to even think about it.

Stay tuned!

I can confirm a Report with UNICODE will create EMF files in the Preview Queue and show Unicode:

This is my own Report Test simple preview that shows the Preview Queue in a drop list. The EMF works fine in an IMAGE control.

Notice the temp file name is now CLA plus 16 digits and ends with .EMF for Unicode. WMF files get the same file name, possibly to fix the Report Pages Lost/Dup problem (I’ll have to test).

The format seems to be CLA0000 + HHHH + PPPPPPPP i.e.

  • Always starts with CLA0000 in my tests
  • + HHHH a unique per Report instance ID 4 Hex Digits
  • + PPPPPPPP 8 Hex Digits for page number, so 2 billion pages

It looks like they took the old 7.3 temp name CLAhhhh.WMF and inserted ‘0000’ then appended an 8 hex Page Number.


Below an example without UNICODE on the Report and shows ? ? ? for the Chess and .WMF for the file name. WMF cannot show Unicode.


Ctrl+D to open the Designer will reject Report .... UNICODE so add it after. It did not work to add the new Report{PROP:Unicode}=True which says its for BLOB/MEMO. I’ll hunt for a PrintProp:.

UnicodeRptTest1-v2.zip (2.5 KB)

I found undocumented Property to turn ON Unicode for a Report like this

Report REPORT,AT(),PRE(RPT),THOUS  !!! not UNICODE here
       END
Prop:UnicodeReport EQUATE(7A82h)      !Undocumented Prop does REPORT,UNICODE
    CODE  
    OPEN(Report) 
    Report{Prop:UnicodeReport}=TRUE   !Make EMF Page Files 

That number 7A82h comes after the last Property.clw PROP:OriginalHeight EQUATE(7A81h)

@RZaunere does this Property do something other than Reports?

I tried the PROP: with SYSTEM but it did not change the Report to Unicode. It seems it would be useful to set it once so All Reports output EMFs. For Lee that would help him test his libraries with every report outputting EMFs.

Carl,

I’m unable to test anything at the moment.

I was wondering, what happens to this report if you forget to indicate UNICODE at the report level?

Off the collar, do templates support UNICODE entries?

If so, what’s needed and how do you include UNICODE in the templates?

Small example. RPM introduced page of pages and my template allows the developer to enter the placekeeper which, now, could need to be UNICODE.

The Unicode will show as Question marks ? ? ?

If you look up at my post the 1st screen shot shows Unicode Chess Pieces and the file name is an .EMF

The 2nd screen shot shows the Chess as Hello World ? ? ? and a .WMF file name. This is documented in the Test Guide.

Thanks for finding that REPORT..UNICODE, it was a pre-parse validator that didn’t get touched (fixed now).

The report page naming now generate unique names (process id + per-report sequence), and yes it is the fix for the [quote=“CarlBarnes, post:4, topic:9287”]
Report Pages Lost/Dup problem
[/quote]

Nice find, Carl — that’s a real switch, and your read of it is essentially correct, with a couple of caveats worth knowing before anyone leans on using it.

What it is. 7A82h is the internal runtime face of the REPORT ,UNICODE attribute (internally it’s called the “meta kind” — which page format the report records, WMF or EMF)?. It was the in-development force switch, and we deliberately left it in place when the UNICODE attribute became the public switch. It has no equate in Property.clw because it’s an unsupported property — the attribute is the supported way to get EMF pages.

Exact work seq. since you’ll want them:

  • It answers on the report object only. SYSTEM, PRINTER, and TARGET have no touch to it — that’s why your SYSTEM attempt was a silent no-op.
  • It’s honored after OPEN(report) and before the first PRINT. Once the first page starts, the WMF/EMF format is locked and any later writes are silently ignored — a report never changes format mid-run.
  • It’s symmetric: setting it to FALSE before the first PRINT forces a REPORT,UNICODE report back to classic WMF pages.
  • Reading it back returns TRUE when the report is on the EMF format.

The caveat that makes it second-class to using the UNICODE attribute: TEXT controls choose their window class when the report structure opens, and the class choice is what carries UTF-16 through to printing. The PROP fires after that choice. So a report flipped to EMF via the PROP produces EMF pages, _but a TEXT control bound to wide data (USTRING, Unicode BLOB) will still narrow its content — and because the “Unicode text in a non-UNICODE report” diagnostic keys off the same latch you won’t get the error 546 warning either. In other words: for narrow reports where you just want EMF page files — exactly Lee’s library-testing scenario — the PROP works fine. For reports that actually carry Unicode content, use the attribute; it’s the only switch that gets everything.

On a SYSTEM-wide “all reports output EMF” switch: we’ve thought about it, and for testing it could be useful. If we add it, it would be checked at report OPEN (the same point the attribute is), which would also close the TEXT control gap above — so it would be a better global switch than looping the PROP. The reason we’re cautious is that it’s global: the report-to-file export parser is compiled into each application, so any app built before the EMF-aware parser, or using a third-party WMF-only export/preview tool, would fail on every report the moment that switch is set. That’s the same reason WMF stayed as the default and the UNICODE attribute is opt-in per report. If SYSTEM level ever ships, expect it as a test-only switch in the same undocumented spirit as 7A82h, likely not as a supported way to flip production apps wholesale. OTOH I’m listening to any voices that would prefer the opposite; i.e SYSTEM level switch, or potentially even defaulting to EMF for all Reports (I *think Lee may lean that way?)

Short version: your PROP is fine as a test harness tool on narrow (no UNICODE attrib) reports if used within the timing window I described, and understanding that it’s unsupported and the UNICODE attribute remains the real thing. I’ll wait to here from Lee on whether the SYSTEM global helps him.

  • No — FORMAT is width-preserving, not always-wide. It’s compiler-lowered (no builtins.clw entry): the value is pushed, transforms the top-of-stack entry in place.
  • Control text-content props are width-aware: {PROP:ScreenText} on ENTRY (flags.wide), TEXT, STRING controls and the caption push wide when the control actually holds wide data, and take the legacy narrow route otherwise
  • Yes — first-class base type. TYP_USTRING = 51, it’s the wide CSTRING: raw UTF-16LE, in-band wide NUL, data capacity n−1 units, SIZE() = 2n bytes
  • all conversions are implemented, some involve the DECIMAL path but avoid the REAL

Conversion to REAL or DECIMAL as an intermediate value must be controlled by the define(bcd_arithmetic) pragma.

DECIMALs are more slow than REAL. Also, instructions using for implementation of BCD are absent in the 64 bit mode of Intel/AMD processors. Emulation of these instruction will made DECIMALs even more slower in future 64 bits.

For the most common case, nothing special is needed: a developer can just type “Seitenübersicht” — or any text in their own language — straight into your prompt today. It’s stored as ANSI bytes in the .app, generates as ANSI source, and prints correctly whenever the machine’s codepage covers the characters — most of the time German, French, Spanish, Scandinavian, Eastern-European text on the matching codepage: type it in, done. The one thing the report needs for wide-correct output is the UNICODE attribute, and on a UNICODE report those ANSI entries widen correctly through the active codepage at print time.

The template language needs nothing new either. The whole AppGen chain (template files, the registry, prompt values stored in the .app, and the generated source) is byte/ANSI storage end to end, by design: generated source is always ANSI.

When the typed in text is outside the codepage (Greek on a Western machine, CJK, symbols) Unicode goes through the chain as an ordinary Clarion wide string literal, whose spelling is plain ASCII.

The literal form is U’…'; inside it, the metachars take decimal UTF-16 code point values (with <n{r}> repeating a value, and << for a literal <). So Greek “Σελίδα” is:

U’<931,949,955,943,948,945>’

and a mixed placekeeper like "Page — " is U’Page <8212> '. Characters above FFFFh get written as their surrogate pair, or built at runtime with the two-parameter CHR(codepoint,1). Because the spelling is ASCII, it survives the prompt entry, the .app, TXA export/import, and generation without any encoding concerns.

Two things to handle on the template side:

1. Emit the prompt value as an expression, not inside quotes. The U prefix sits outside the quote marks, so a template that wraps the entry in quotes can never receive a wide literal. So for the page-of-pages placekeeper:

#PROMPT(‘Page-of-pages text (expression):’,@S80),%PageOfPagesText,DEFAULT(‘’‘Page ‘’’)

Note: it’s the expression 'Page ', quotes included (doubled per template-source rules), because the entry is now expression-valued. The generation side emits the symbol raw:

#AT(%AfterOpeningReport)

?PageOfPages{PROP:Text} = %PageOfPagesText & ’ ’ & PAGENO()

#ENDAT

A developer who wants plain English enters 'Page ’ and the generated line is:

?PageOfPages{PROP:Text} = 'Page ’ & ’ ’ & PAGENO()

A developer who wants Greek enters U’<931,949,955,943,948,945> ’ and generation produces:

?PageOfPages{PROP:Text} = U’<931,949,955,943,948,945> ’ & ’ ’ & PAGENO()

2. The target report needs the UNICODE attribute. Wide text only survives onto the page on a REPORT,UNICODE report (EMF pages). On a report without the UNICODE attr the runtime posts error 546 once and then prints narrowed text— deliberate, so a long job isn’t killed by a diagnostic/error.

If your template ships to mixed compiler versions: the new compiler defines the flag _USTRING_, and the older ones don’t. Generate wide-Ustring code inside a COMPILE(‘…’, _USTRING_) block (older compilers OMIT it) and your template stays valid across both.

I was able to confirm that the WMF/EMF name is “process id + page#”.

This means only 1 Report can be running at a time … and that is NOT going to work.

CPCS Reports has a feature I use that lets you string together multiple Reports into a single Preview Q and Print all. All of the Reports are open at the same time. The final Report adds the pages from the other Reports. E.g. I have a Detail Report, then a separate Summary report after. I think @lodestar Lee White can confirm this also.

I created a test that Opens Report 1 and Prints 12 pages, it does not close Report 1. Then Opens Report 2 and Prints 12 pages. The 2nd Report gets the same EMF file names as shown in this window:

Then the test adds the 2 reports into 1 PreviewQ and shows it (like CPCS). What you see is only Report 2 pages because they overwrote Report 1.

You can have Reports running in 2 different Threads. I did not test if that is handled. Should I?

TwoRptsRun_20260815_184333.ZIP (2.8 KB)

Thanks for the test program. The problem was from a ctor that zeroed out the tmpnameid. Its fixed now and is changed to use InterlockedIncrement so it can survive multiple threads running reports simultaneously.

Robert,

I’ll dig through this after I get my energy back. Just got home from the hospital where I ended up Sunday afternoon bent over in pain. Seems my right kidney, which has always caused me problems, decided to get stopped up with large stones - one the size of your thumb!

Hospitals are not the place to rest but I’m home now where I can.

They will be going back in 2 weeks from now to remove the stent and reduce the stones so they can pass.

Getting older is NOT always a lot of fun! At this moment my bed is calling my name and I’m going to answer - I’m tired!

Appreciate the work you’re doing - it will help once I get back on my feet.

And for the record RPM introduced page of pages AND appended reports, Larry just stepped on my toes! But I survived!!!

Hey Lee,

Rest up for now, and when you do get back at it make sure you get in touch with me for anything you need from SV, whether its answers/explanations, sample code, or library-side exports/functions.