Report Pages Lost or Duplicated in 11.1

ademirpiva · August 25, 2022, 2:06pm

When you print a report, it prints one page instead of the other, for example:

Prints page 246 twice, in place of page 78 and in place of itself, page 78 does not print in the report, in the same report it loses one or two pages.

I did 3 tests with Clarion 10, all ok, no errors.
I did 4 tests with Clarion 11, in 3 reports gave this error, each time the error occurs on different pages, it is not always on the same page, one report lost only 1 pages, in 2 reports lost 2 pages, always exchanged with another page of the end.

CarlBarnes · August 25, 2022, 7:51pm

What release of 11? Does this use Report Targets at all e.g. you output to PDF?

Do you see this in Preview, or just when flushed to Printer on paper, or just in PDF ?

The Report pages are WMF files in your Temp Folder. You can put %TEMP% in the address bar or at a C:> DOS prompt CD %TEMP%. The files will be named CLAxxxx.TMP. If you Rename or Copy them with a .WMF extension they should open in Paint so you can see if the WMFs are also messed up, I would think not.
Edit: In 11.1 the files have a .WMF extension i.e. CLAxxxx.WMF

You should check your TEMP folder for left over (orphaned) CLAxxxx.tmp files and delete them. Do a DIR CLA*.tmp and check the count. If that folder has 1000s of them it will cause problems. It is safe to delete all the CLA temp files: DEL CLA????.tmp when a Report is not in preview.

I wrote the below Temp cleaner I run once per month for all users. Crashed reports, bad code and bad templates can leave orphan TMP files.

I also have this Report Test project that would make it easy to generate simple Reports for the exact 100s of pages you need. Since every page could have the same text and a page number you could write code to open all the Temp/Wmf files and find problems.

gist.github.com

https://gist.github.com/CarlTBarnes/29f78a2ca43813e83489dc62a4d5e0c0

AbcPreview_RptTest_FontProp111.clw

!Report Test Framework by Carl Barnes from https://gist.github.com/CarlTBarnes
!Using ABC Report Previewer instead of my simple previewer
!
!This tested Report${PROP:FontXxx} broken by 11.1
!
!-----------------------------------------------------------------------------
!Try various REPORT specs and PROP Settings quicky in this little Win32 project that has its own simple Previewer.
!If you want to report a bug this can be used to create a small example
!
!1. Create a new Win32 Project and paste this code.

This file has been truncated. show original

ReportTest.clw

!Report Test Framework by Carl Barnes from https://gist.github.com/CarlTBarnes  https://git.io/JtP6r
!-----------------------------------------------------------------------------
!Try various REPORT specs and PROP Settings quicky in this little Win32 project that has its own simple Previewer.
!If you want to report a bug this can be used to create a small example
!
!1. Create a new Win32 Project and paste this code.
!2. Add the Variables used on the REPORT  - at #2 below
!3. Add Your REPORT and DETAIL's          - at #3 below
!4. Add your PRINT() code and PROP code   - at #4 below
!5. Run and Test, adjust code, repeat

This file has been truncated. show original

WidthCalc_ReportTest.clw

!Test created for https://clarionhub.com/t/report-item-width-and-height/3629     
!   I have a report band with 2 variable string controls named AgentName (font 16) and LicenseNo (font 12) 
!   in which I would like to print the LicenseNo right after AgentName. How can I get the width of 
!   AgentName when value is assigned so that I can adjust the XPos of LicenseNo?
!
!How to measure STRING #1 on Report to position STRING #2 right after it
!Answer: Must not use a Picture STRING(@s20).
!        Must use STRING('xxx') and PROP:Text to assign value then can get PROP:Width
!
!Report Test Framework by Carl Barnes from https://gist.github.com/CarlTBarnes

This file has been truncated. show original

There are more than three files. show original

Rick_UpperPark · August 25, 2022, 8:40pm

Too many files in the temp folder can definitely cause a problem. The API clarion uses to generate the temp file names is old and won’t work once there are 65,536 files in the folder, even though newer versions of windows do not have that limitation.

CarlBarnes · August 25, 2022, 9:28pm

Are you are doing large 1000+ page Reports they can close very slowly due to a change in Remove. Some template changes noted in this post close reports much faster.

The test program generates simple 1000+ page reports. You could modify it to scan the WMFs to see they have all pages without duplicates,

vitesse · August 26, 2022, 12:08am

Hi Ademir

I wonder if this is related to the newsgroup post from 2022-04-07 by Andrej ?

Newsgroups: softvelocity.clarion.reports
From: andrej *****@gmail.com
Subject: wmf files getting overwritten for large number of pages
Date: 7 Apr 2022 08:49:44 -0400

Hello

I found an issue in report generator. The generator uses win function
getTempFileName from win.h to generate unique file names. This is all
great, but clarion apparently skips by way more than 1. If I use this
function to generate filenames, I get

claCF00
claCF01
claCF02
…

However, when I check temp wmf files generate by clarion report
generator, I see for example (this is from actual preview queue, not on
disk):

CLACB74.wmf
CLACF9B.wmf
CLACF9C.wmf
CLAD26C.wmf
…

This way for ~300 pages printed it will often happen that a file
already created will be overwritten, which leads to missing or wrong
pages on the report preview.
Does anyone have the same issue and knows what might be the cause?

Regards,
Andrej

===========

I can only see one response

From: Carl W. Sumner *********@compuserve.com
Newsgroups: softvelocity.clarion.reports
Subject: Re: wmf files getting overwritten for large number of pages
Date: 8 Apr 2022 13:07:55 -0400

I think you have found a bug, but I am not sure if it is in the report
generator or in the Windows API.

Some people think that a random number is always unique, but it is not
true…

You should report it on the bug reporting site, anyway.

http://problemtracker.softvelocity.com/

Carl Sumner

CarlBarnes · August 26, 2022, 2:13am

If the WMF file names are being duplicated that seems like it can be determined by scanning the Report Preview Queue?

Maybe could use the PROP:TempNameFunc function to assign WMF file names in a way you’re are sure they are unique. It requires you to delete the files. I was doing that anyway because the RTL Close(Report) was so slow.

https://clarion.help/doku.php?id=prop_tempnamefunc.htm

There’s a hook PROP:OpenReportHook for Open(Report) where you could set PROP:TempNameFunc for every Report, and a hook for Close(Report) where all the files could be deleted.

For a Page File Name I would suggest CLApppp-tt-ssss.wmf e.g. CLA1691-2-123.wmf

pppp = GetCurrentProcessID() which is unique for all open programs
tt = THREAD() in case multiple threads are running reports
ssss = a sequence page number that would be a SeqNo LONG,THREAD

The LONG,THREAD variable would be zero for each new thread so the file seq page numbers would start over. Or just make SeqNo STATIC without THREAD so the numbers go up until the program closes. Then no need for Thread() but the file names would not be sequential if 2 reports were running (which is unlikely as reports are kid of modal).

I would put the files in the Windows Temp folder. I would wrap the code in a Critical Section.

ademirpiva · August 26, 2022, 2:44pm

Thanks for replying.
I tested on Clarion 11.1.13810 and 11.1.13815.

The error happens when viewing and printing on paper, in this report I don’t have a direct option for PDF.
The %TEMP% folder on my PC was low on files, but on the client PC it had 3902 files in total and 50 Cla*.WMF.

It makes sense that the problem is related to the amount of files in the %TEMP% folder because this error only happens using the Client Database that prints with the most pages, this report has 546 pages (WMF), 268 JPG, 1 PNG and 1 GIF.

Is this “related to the 2022-04-07 newsgroup post by Andrej”?
I don’t think so, because my report loses a page and duplicates another, if that were the case it would only lose one or more pages, but it wouldn’t duplicate another.

It’s really too slow to close the reports, this report takes several minutes.

FedericoNavarro · August 26, 2022, 2:46pm

Also reported on PTSS 43308 by Martin, Donald on 2022-05-12 Reports skipping and duplicating pages, on Additional Info he states:
"Problem did not exist in Clarion 11.0.13630.
I recompiled program in the version 11 and report is correct. I noticed the page files are created as .tmp files in v.11.0 where in v.11.1 they are .wmf types. "

CarlBarnes · August 26, 2022, 4:26pm

I was able to reproduce duplicate WMFs in 11.1.13815 using a modified version of my Report Close Slow Test I’ll attach at the end.

After the Report is generated I call this function to check the Preview Queue for duplicate file names and WMFs missing i.e. ~Exists(). You could add this to your program to test real reports and know how often it is happening.

CheckForPagesLOSTorDUP      PROCEDURE(QUEUE PreviewQ)!,BOOL,PROC
PrevQ:FileName     PSTRING(256) 
DupCheckQ QUEUE,PRE(DupChkQ)
PageNo      LONG            !DupChkQ:PageNo
FileName    STRING(260)     !DupChkQ:FileName
    END
PgX     LONG
BadMsg  ANY
BadCnt  LONG
    CODE
    LOOP PgX=1 TO RECORDS(PreviewQ)
         GET(PreviewQ,PgX)
         PrevQ:FileName = CLIP(PreviewQ)
         IF ~EXISTS(PrevQ:FileName)       !Test OR PgX=33   !Problem: File not on disk?
            BadMsg=BadMsg &'<13,10>Page '& PgX &' file not found: '& PrevQ:FileName
            BadCnt += 1
         END

         DupChkQ:FileName=UPPER(PrevQ:FileName)
         GET(DupCheckQ,DupChkQ:FileName)                !Check if WMF Name for already in DupQ i.e. Duplicate
         IF ~ERRORCODE()                  !Test OR PgX % 22 = 0
            BadMsg=BadMsg &'<13,10>Pages '& PgX &' and '& DupChkQ:PageNo &' same: '& PrevQ:FileName
            BadCnt += 1
         ELSE
            DupChkQ:PageNo = PgX
            ADD(DupCheckQ,DupChkQ:FileName)
         END
    END
    IF ~BadCnt THEN RETURN False.
    
    BadMsg=BadCnt &' problems found in '& RECORDS(PreviewQ) &' page report:<13,10>'& BadMsg
    CASE Message(BadMsg,'Report Pages Lost or Duplicated - Process ' & Glo:ProcessID,ICON:NoPrint,'Close|Copy Text',,MSGMODE:CANCOPY)
    OF 2 ; SETCLIPBOARD(BadMsg)
    END
    RETURN True

One test showed 4 pages duplicated and displayed this:

The Previewer in this test lets you see the Queue and screen shot below shows those files appear twice. Note the Report pages have the “Process ID” so it would be possible to tell if the WMF file was from a different process. I did not check for that… yet.

To create more contention you can run multiple at the same time with the /GO common switch. I include a Run3.BAT that does START ReportPagesLostTest.exe /GO 3 times.

The other thing you can do is press the HALT button in preview to leave orphaned WMF files in the Temp folder. The main window has a Purge Temp button to clean that up.

Here’s my test project. Note the Build Clarion library version 11.1.13815 (from PROP:LibVersion) shows on Windows and Reports so you can try in 11.0 or 10.0 and be sure what version was tested.

ReportLostPagesTest.zip (10.5 KB)

ReportLostPagesTest-V2.zip (10.6 KB)

CarlBarnes · August 26, 2022, 5:03pm

I added the Process ID on most Windows and the Report pages (new V2 in above post).

I am seeing pages from a different process and plenty of duplicates testing 11.1:

When I look in the Temp folder at the next logical file name CLA9A07.wmf that should have been page 9 it is from a different process, the same 4912:

ademirpiva · August 26, 2022, 5:55pm

Very good Carl.

I think this is the error that occurs here in my program.
I will test this function early next week.
Then I’ll come back with the results.

Thanks.

anon23294430 · August 26, 2022, 11:50pm

I’d get it sending msgs to debugview to see if you can hunt down the bug. I wouldnt rule out hardware problems though.

CarlBarnes · August 27, 2022, 2:34am

I ran my test compiled with 11.0.13505 and did not see any of the problems.

I ran 3 simultaneous reports of 500 pages. I repeated that 3 times. I’m testing on a Surface with an SSD.

I upped it to 5 simultaneous reports of 1000 pages. I repeated that 5 times. I left 10,000 orphaned .tmp files in the Temp folder for more stress. Another test was fine.

No problems at all in 11.0.13505 tests while 11.1 failed probably 1/3 or 1/2 the time.

ademirpiva · September 14, 2022, 5:00pm

Carl Barnes’ solution solved my problem.
After several days of testing the report on the client, there was no more error.
Many thanks to the colleagues who were willing to help me.