Memory leak with TIMER

I’ve been advised (via the C12 newsgroup) to repost this here…

It would be appreciated if someone from SV could look at problem tracker
#43507. This will prevent our C12 adoption and it’s very likely other Clarion
users will be impacted. The code to reproduce (see below) couldn’t really be
more trivial. It leaks memory on each timer event (with this example, approx.
4kb every 5 seconds). The higher the numeric TIMER attribute value the less
obvious, but it still leaks.

  PROGRAM

  MAP
  END

W WINDOW('Test'),SYSTEM,TIMER(1).

  CODE
  OPEN(W)
  ACCEPT.
  CLOSE(W)

Same happens with Clarion 11.1.13855

It could have been introduced any time since C9 - I jumped straight to C12.

I’ve tested in 12, 11, 10 and 6.3 and all of them show the same issue. I made the test easier so you can see the results.

                    PROGRAM

                    MAP
                        MODULE('kernel32.dll')
                            GetCurrentProcess(),ULONG,PASCAL,RAW,NAME('GetCurrentProcess')
                            LoadLibrary(*CSTRING lpLibFileName),ULONG,PASCAL,RAW,NAME('LoadLibraryA')
                            GetProcAddress(ULONG hModule, *CSTRING lpProcName),ULONG,PASCAL,RAW,NAME('GetProcAddress')
                        END
                        MODULE('psapi dynamic')
                            ! No static import and no psapi.lib: DLL(1) makes this an INDIRECT
                            ! call through an import slot. The global GetProcessMemoryInfo_fp
                            ! variable (NAME matches this prototype) IS that slot, and we fill
                            ! it at runtime with GetProcAddress (see DO BindPsapi below).
                            GetProcessMemoryInfo(ULONG hProcess, *PROCESS_MEMORY_COUNTERS pmc, ULONG cb),BOOL,PASCAL,RAW,DLL(1)
                        END
                        ShowMemory()
                    END

PROCESS_MEMORY_COUNTERS     GROUP,TYPE
cb                              ULONG
PageFaultCount                  ULONG
PeakWorkingSetSize              ULONG
WorkingSetSize                  ULONG
QuotaPeakPagedPoolUsage         ULONG
QuotaPagedPoolUsage             ULONG
QuotaPeakNonPagedPoolUsage      ULONG
QuotaNonPagedPoolUsage          ULONG
PagefileUsage                   ULONG
PeakPagefileUsage               ULONG
PrivateUsage                    ULONG    ! PROCESS_MEMORY_COUNTERS_EX - true "Private Bytes"
                            END

! Dynamic-loading plumbing for GetProcessMemoryInfo (psapi.dll loaded at runtime).
! NAME('GetProcessMemoryInfo') aliases this variable onto the import slot the
! DLL(1) prototype calls through, so writing it = setting the function pointer.
GetProcessMemoryInfo_fp UNSIGNED,NAME('GetProcessMemoryInfo')
hPsapi              UNSIGNED
PsapiName           CSTRING('psapi.dll')
GpmiName            CSTRING('GetProcessMemoryInfo')

! Leak-tracking counters
BaseBytes           ULONG            ! Private Bytes captured at the very first sample
LastBytes           ULONG            ! previous sample, for change-per-tick
CurBytes            ULONG            ! current sample
DeltaBytes          LONG             ! CurBytes - LastBytes  (signed)
GrowthBytes         LONG             ! CurBytes - BaseBytes  (signed)
AvgPerTick          LONG             ! GrowthBytes / Ticks   (smoothed leak rate)

W                   WINDOW('Test'),SYSTEM,TIMER(1),AT(,,260,100)
                        STRING('Private bytes:'),AT(10,10,80,10)
                        STRING(''),AT(95,10,150,10),USE(?MemText)
                        STRING('Change/tick:'),AT(10,25,80,10)
                        STRING(''),AT(95,25,150,10),USE(?ChangeText)
                        STRING('Total growth:'),AT(10,40,80,10)
                        STRING(''),AT(95,40,150,10),USE(?GrowthText)
                        STRING('Avg/tick:'),AT(10,55,80,10)
                        STRING(''),AT(95,55,150,10),USE(?AvgText)
                        STRING('Timer ticks:'),AT(10,70,80,10)
                        STRING(''),AT(95,70,150,10),USE(?TickText)
                    END

Ticks               LONG

    CODE
        DO BindPsapi
        OPEN(W)
        ShowMemory()
        ACCEPT
            CASE EVENT()
            OF EVENT:Timer
                Ticks += 1
                ?TickText{PROP:Text} = Ticks
                ShowMemory()
            END
        END
        CLOSE(W)
        RETURN

BindPsapi   ROUTINE
        ! Load psapi.dll and resolve GetProcessMemoryInfo at runtime - no import lib.
        hPsapi = LoadLibrary(PsapiName)
        IF hPsapi
            GetProcessMemoryInfo_fp = GetProcAddress(hPsapi, GpmiName)
        END

ShowMemory          PROCEDURE()
pmc                     LIKE(PROCESS_MEMORY_COUNTERS)

    CODE
        IF ~GetProcessMemoryInfo_fp            ! not resolved -> don't call through a null pointer
            ?MemText{PROP:Text} = 'psapi unavailable'
            RETURN
        END

        CLEAR(pmc)
        pmc.cb = SIZE(pmc)
        IF ~GetProcessMemoryInfo(GetCurrentProcess(), pmc, SIZE(pmc))
            ?MemText{PROP:Text} = 'Unable to read'
            RETURN
        END

        CurBytes = pmc.PrivateUsage            ! true Private Bytes (matches Task Manager)
        IF ~BaseBytes                          ! first sample -> establish baseline
            BaseBytes = CurBytes
            LastBytes = CurBytes
        END

        DeltaBytes  = CurBytes - LastBytes     ! change since the previous tick
        GrowthBytes = CurBytes - BaseBytes     ! total growth since startup
        IF Ticks                               ! steady leak rate; smooths chunky OS commits
            AvgPerTick = GrowthBytes / Ticks
        ELSE
            AvgPerTick = 0
        END

        ?MemText{PROP:Text}    = FORMAT(CurBytes / 1024,@n_10) & ' KB'
        ?ChangeText{PROP:Text} = FORMAT(DeltaBytes,@n_11) & ' bytes'
        ?GrowthText{PROP:Text} = FORMAT(GrowthBytes / 1024,@n_8) & ' KB (' & FORMAT(GrowthBytes,@n_11) & ' bytes)'
        ?AvgText{PROP:Text}    = FORMAT(AvgPerTick,@n_11) & ' bytes'

        LastBytes = CurBytes

Mark, without making me dig through documentation on API calls :slight_smile: what are you expecting this app to do/show?
I now have builds in 6.0 and 5b …

Hi Paul

The program tracks the process’s Private Bytes (same figure Task Manager shows). It reports two things: Total growth — how much that has risen since the app started — and Avg/tick — the average number of bytes leaked per timer event. The per‑event average is the key number, because raw Private Bytes rises in chunky steps (Windows commits memory in blocks), so the average is what reveals the true, steady leak rate underneath.

Mark

If it has been around since C6 days, then I’m thinking we would have noticed by now.

Which makes me think perhaps the example is too simple. Ie something else (which is now missing) causes the leak not to happen. I’m going to test a bit more.

OK, so a ever-increasing total growth is bad?
In 6.0.915 and 5b I’m seeing avg/tick tend towards 0, and total growth/private byte going up and down.

Hi Mark,

I think this makes it a bit harder to see - the extra work you’re doing on each tick means fewer timer events can be processed. It would probably be better to only update the screen every 10 ticks or so. In addition, I would expect some growth initially, but that should settle down. This is what happens when I run your version in C9 (total growth eventually steady at 60KB). In C12 total growth is now 1852KB. I’d be very surprised if this was in 6.3 but I don’t have that to test with. It’s definitely not in C9.

Incidentally, the memory is not freed when the window is closed. But it is just before the process exits (it might be freed on a per-thread basis, I haven’t tried - but that could explain why this seems to have gone unnoticed). If you leave the app running for long enough and then close it, you’ll see the process hangs around for quite a while churning CPU as it frees all the thousands of small allocations it’s made.

Jon.

In Clarion11.1.13855 I’m not seeing any growth either.
There’s some on the first tick, then nothing. So not seeing the problem there. Will try now on C12.

update: I’m also not seeing any problem on Clarion 12.0.0.14000.
So either the issue is specific to some build of Clarion 12, or there’s something else in play. I’m at 15000 ticks, and everything is stable. No memory growth.

15000 ticks is not really enough to see the issue (especially with Mark’s version). I think it’s easier to see with my original app. Just keep an eye on the ‘Working Set’ in Task Manager. There will be a little fluctuation from time-to-time but it will leak about 4KB every 5 seconds or so. Note the memory before going for lunch - by the time you get back it will probably have increased by 1-2MB.

I’m using 12.0.0.14000. I haven’t tried any other version except C9 and the problem is not there.

Jon.

ok, I’ll try a long test.. and keep you posted.

Follow-up — I need to correct this.

I built a small instrumented version of the reproducer to measure what’s actually happening per timer event. It dynamically loads GetProcessMemoryInfo from psapi.dll (no import lib needed) and reads PrivateUsage — the same “Private Bytes” Task Manager shows — then reports total growth, a cumulative bytes/tick average, and net growth over a rolling 1000‑tick window.

On C12 (build 14000):

  • Private Bytes rises ~270 KB in the first few seconds, then plateaus.
  • Net growth over each rolling 1000‑tick window settles to 0 — it stops climbing.
  • The cumulative bytes/tick figure keeps decaying toward 0, which is the signature of a one‑time warm‑up cost (heap reserves, window/GDI resources, lazy RTL init), not an unbounded per‑event leak.
  • Total growth varies run to run (~270–490 KB) and doesn’t track the number of timer events — again inconsistent with a steady per‑tick leak.

I also drove the same work two other ways to separate “the timer” from “Clarion’s event loop” — a Win32 SetTimer callback that posts a Clarion event, and a raw TimerProc that does the work directly — and saw the same flat result.

So this is probably not a timer bug, and on my C12 build I can’t reproduce an unbounded leak at all — the early growth is bounded warm‑up. Apologies for the earlier misdiagnosis.

If anyone can get sustained growth (window growth staying positive across tens of thousands of ticks), please post your exact build and OS — that’s the data that would confirm a real defect.

Latest code and project on GitHub.

Mark

For me, It doesn’t grow on C9, but it does grow on C11 and 12.

test

Hi Mark,

The leak is tiny; around 12 bytes/tick. Your version of the app has a lot of ‘noise’ which makes it harder to see the problem (you have to keep it running much longer before it starts to drown out the ‘warm up cost’). I changed it to only update the screen every 100 ticks. After running for around 2 hours (give or take, not sure exactly when I started it) it’s done 350k ticks and ‘growth’ reports as 4276kb. Task Manager shows a working set of 18,980kb.

I have spent hours investigating this, going from a complex app that runs as a service and boiling it down to a trivial example (honestly, TIMER was the last place I thought to look!). But there is a bug here, you’ve just got to let the app run for longer to see it.

Jon.

So Boy George was right? Now the only problem is getting that song out of your head :slight_smile:

Thanks Daniel,

It should be noted that C9 starts off with a higher memory footprint. Running my original version, it takes around 10 minutes for the C12 exe to start using more memory than C9. There will be some fluctuation but after 20 minutes or so the leak will be clear with C12 using around 700-800kb more. I can’t post a screenshot, but after running both side-by-side for 40 minutes the working set for C9 is 14,204kb and for C12 it’s now up to 16,044kb.

Jon.

yes, I’ve run it for several hours now an it has stabilised on leaking 12 bytes per event.
Clarion 12.0.14000

Update: It leaks 12 bytes per EVENT. All events contribute to this, not just event:Timer.
I used Event:User instead of Event:Timer and got the same effect.

Thanks Bruce, nice find. I concur - it is indeed any event. There’s no waiting around with this example. The leak is obvious from the get-go:

PROGRAM

MAP
END

W WINDOW(‘Test’),SYSTEM.

CODE
OPEN(W)
ACCEPT
POST(EVENT:User)
END
CLOSE(W)