This will show you the order of the API calls and data being passed to the Win32 API calls.
Choose all or some of the win32 API calls to log, save to disk and let the app run.
Quite possibly - maybe a debugging exercise accidentally left in. Whatever the case it needs fixing and the original problem tracker ticket has now been updated with info from this thread.
To recover every 1-8 hours you could Shutdown your EXE and Run a new one. Not a bad idea for long running processes.
My test below made a few changes:
- Test code in a Procedure so not Global / Static
- Test Procedure is Start() to be in new Thread
- Open Window shows Message(), then starts Prop:Timer=1
- At end Close that Thread to check if memory recovered
PROGRAM
MAP
TestTimer PROCEDURE()
NewThreadMsg PROCEDURE(STRING TestWndCap,STRING TestThreadNo)
END
CODE
START(TestTimer)
TestTimer PROCEDURE() !Put test in Procedure so Not Global
W WINDOW('Timer Leak?'),AT(,,200,50),CENTER,SYSTEM !Find Timer(1) in OpenWindow
END
StartTime LONG
EndTime LONG
TimeTaken LONG
EventCnt LONG
CODE
OPEN(W)
ACCEPT
CASE EVENT()
OF EVENT:OpenWindow
Message('Ready to Start Timer||VMMap "Select Process" TimerLeak1.EXE|After OK here then OK VMMap')
StartTime = CLOCK()
0{PROP:Text}=0{PROP:Text} &' - Started '& FORMAT(StartTime,@t4)
0{PROP:Timer}=1
OF EVENT:CloseWindow
EndTime = CLOCK()
TimeTaken = EndTime - StartTime + 1
CASE Message('End Time <9>'& FORMAT(EndTime,@t4) & |
'|Time Taken <9>'& FORMAT(TimeTaken,@t4) & |
'|Event Cnt <9>'& EventCnt & |
'',0{PROP:Text},ICON:Clarion,'End Test|Continue')
OF 1 ; BREAK
OF 2 ; CYCLE
END
! OF EVENT:Timer !nothing to do
ELSE
EventCnt += 1
END
END
START(NewThreadMsg,, 0{PROP:Text}, THREAD())
CLOSE(W)
RETURN
NewThreadMsg PROCEDURE(STRING TestWndCap,STRING TestThreadNo)
CODE
YIELD() ; YIELD()
Message('Time Leak Test completed in Thread '& TestThreadNo & |
'||This message is in Thread '& THREAD() & |
'||Go check VM Map and F5 Snapshot|to see if Memory Released?', |
TestWndCap, ICON:Question)
A way to track Process Memory is using VMMap from SysInternals. This lets you dig into what kiods of memory are being used and growing. This is external to your program so requires no changes. You can take Snapshots (F5) and compare them using the Timeline button or Ctrl+D to Show Difference.
The test starts with a Message() to let you get memory monitoring setup.
After I click OK in VMMap I “Select Process” and below it initial Snapshot shows Total Working Set of 11.472 Mb. The Image takes 9.520 Mb, that is the EXE and DLL memory.
After almost 2 hours and 451,000 events I press Close and a Message opens. In VMMap press F5 to take a Snapshot
Using the Timeline button you can drag across Snapshots (there are just 2) and the upper display shows the change. Total up 30 Mb and Total WS went up 6.180 Mb
Pressing “End Task” on that Message ends Accept and the Thread, but first does a START(NewThreadMsg) to show the below message. This was to see if closing the Thread freed memory:
Below shows Timeline comparing Start to after Test Thread close. Total memory shows as down 4 Mb but Total WS is still up 6.472 Mb. Not sure what to make of this. There are a lot of features. There are videos on VMMap if you search like this video.
Download of older 32-bit VMMap v3.26 that included the CHM Help.
VMMap_3.26.zip (679.1 KB)
When virtual allocation memory is leaked, it’s represented in VMMap as Private Data:
Hi Carl,
The app is running as a service. If I were to force it to close periodically, I would have to set the service ‘recovery’ to restart on all ‘failures’. I don’t like doing this because (a) it can hide real crashes and (b) it may impact the SLAs we have with some clients. Fortunately, this issue was found before we released any C12 code (the C9 version runs 24/7, only restarting once/month for Windows updates).
In terms of what’s going on with your example: we know that the leak is 12 bytes per ACCEPT iteration (it’s not just timer events, all leak). With 451k events, that’s 5.4MB. You see 6.4MB because you started a new thread. By default, Clarion threads have a stack size of 1MB (and the working set includes heap and stack space). In other words, you are seeing the leak but it’s obfuscated slightly by the new threads.
Your example does prove something new: the RTL is not freeing this memory when the thread exits. So it’s more serious than I thought (that could have been the one saving grace), especially for long running apps.
Jon.
You could try a Windows API SetTimer() that’s setup to callback to a TimerProc to see if that also leaks. That will require some code refactoring as you will not be in the same procedure. You can see Module or Global variables in the callback.
Timers - Win32 apps | Microsoft Learn.
You could also try without the Window and Accept loop using a Waitable Timer Object. That seems like a good way for a Service that cannot show a window.
Much easier to stay on c9 until the fix in C12 is released. With something as serious as this i suspect a fix will be forthcoming.
Using an unaffected build of c11.1 would be a useful interim step, that should accept Jon’s apps (perhaps with the C9 dict). That would at least get most of the bugfixes in the IDE and runtime since C9.
And the wide template screens, which (for me) was the overriding killer feature in C11.
Hi Carl,
The service is based on internal class which expects a window. This class is used by all our client-side applications. It checks the server-side license and state (licensed/unlicensed/number of queued/processed jobs, etc) - information usually shown in the status bar (where there is a window). Via some virtual methods the service sends emails instead (license expiry warnings/job result summaries and the like).
Bruce is right here, it’s not that easy to change - possible but hard to justify (the boss wants new features, not refactoring!). We might as well stick with C9 until a fix is available. Indeed, all the apps are hand-coded so there’s no real benefit from the IDE improvements. The only reason for making the change now is to see how the Unicode stuff is shaping up (when it’s eventually released!).
Jon.
Check back, last I heard it would be by the end of the week. 
I would tell you to go to 11.0.13505 which is what I use. There are some problems after that, mostly due to Any Screen, that I would not use it for production. Like Reports lose pages or include pages from other reports.
Hi Carl,
Is that reporting issue in C12? Clearly they are generating temp file names without ensuring the file doesn’t already exist (so they are getting reused). I don’t see anything in the problem tracker for this. My impression is that SV are looking at the tickets, even if they don’t actually get opened.
Jon.
Yes
It did with the first release 13941. I just tested 14000 running 5 reports and 4 of them had page problems. You see messages like this.
ReportPagesLostTestProject.zip (10.9 KB)
Probably worth adding your test app to the problem tracker. My guess is they are using GetTempFileNameA internally but they changed what they pass in the ‘uUnique’ parameter from the affected version of C11. If they previously passed zero, the function would check the file didn’t already exist. But as there can only be 65535 unique names this will become really slow as the number of temp files increases. The ‘fix’ for performance would be to pass a non-zero value, but in that case the function doesn’t verify that the file doesn’t already exist. Whoops!
My systems run a Temp Folder Clean during shutdown every few weeks. I’m often amazed the first time I add this how many 1000+ orphan CLA*.TMP or CLA*.WMF files there are. If you use big Clarion Reports I think you must do this Temp Clean else long time users eventually have trouble.
Temp files are supposed to be cleaned up when the report is closed (explicitly or implicitly). So this shouldn’t be happening unless there’s another bug here or the files are in use by something else (possibly a ‘previewer’ on a different thread to the REPORT itself - don’t do that). In any case, deleting these files yourself doesn’t help if the underlying issue is with the temp file name generation. Once a file name gets reused it could be for the current report or even one being generated by a different thread or process (pages from completely different reports would randomly appear). To avoid known performance issues with GetTempFileNameA, SV should create a function which mangles in a GUID to ensure a unique temp name. Although it shouldn’t be necessary, I’d be inclined to go ‘belt and braces’ - wrap the function in a global mutex and loop (generating a new GUID) until you can create a file that doesn’t already exist.
Jon.
The original point of the temp file deletion wasn’t to avoid the collision, but because SV changed the DELETE in the RTL to use a different API, making it much slower to close a report. Way way way slower for large reports.
Not sure what you mean. If there was proper temp file creation there can be no collision on removing files. If they changed to deletes from a basic RemoveFile to SHFileOperation (presumably with FOF_ALLOWUNDO) that would be silly (at least for temp files). I don’t use REPORTs myself, but it sounds like it’s worth a problem tracker ticket up to urge them to change it back? A simple example is usually effective.
The Report Closes Slow is not the issue I was referring to here.
I used that test project to create a stress test to prove Report Pages were being lost in 11.1+. So the windows look very very similar and could be confused. One option in the test was to Halt in the Preview to leave orphan CLA Temp files and create more stress.
There is no underlying issue or bug in Temp file cleanup.
CLOSE(Report) must happen for the Temp files to be cleaned up. Any GPF, Hang, Shutdown, Network Disconnect, etc, will prevent CLOSE(Report) and leave Orphan Temp files.
Check your own Temp folder, or if you are remoted into a customer with:
DIR %TEMP%\CLA.*
You can find 1000’s of files. My experience is if you have serious reporting you should run a Temp folder cleanup like mine.
Also if you use CPCS Report templates it creates ########.WMF files, but does not delete them. I see Tracker PDF*.TMP files also being left in the temp folder. My tool cleans those up. I know many developers assume the Temp folder gets purged by Windows, but it does not.