How can I access the raw stack data? MS callback proc prototype changes when asm starts popping the stack

What are your prototypes?

So I’ve actually been changing the prototypes around for the callback proc to see what would work

                 HMODULE hModule,
                 LPSTR lpType,
  [in]           LONG_PTR lParam

I’ve tried many combinations as a prototype
clarioncallback(long,long,long),bool
clarioncallback(long,long,long),bool,Pascal
clarioncallback(long,long,long),bool,C
clarioncallback(long,long,long),bool,Raw,Pascal !Cant use Raw with a clarion proc
clarioncallback(long,long,long),bool,Raw,C !Cant use Raw with a clarion proc
clarioncallback(long,ulong,long),bool !ulong works with other api’s using makeintresource
clarioncallback(long,long),bool !not passing the hmodule

So the problem I keep seeing here is the address in lptype (2nd param) is moved to the hmodule param and the address in lParam (3rd param) is moved to the 2nd param.
I’ve tried sending null aka zero as the hmodule to enumerate the program calling these api’s, I’ve tried loadlibrary the same running program, and loadlibrary a different exe thats not running, and in those cases ie where hmodule is not null, the address in hmodule appears at the procedure call and then start the stack popping bit in assembler and the hmodule address dissappears and lptype becomes hmodule and lparam becomes lptype.

This is weird, I’ve never seen this happen like this before, but where I have the ulong in the prototype, ulong works with other api’s that take a long pointer to a cstring or const cstring and also takes a ulong.
Now for that trick to work, I have two api protptypes in the module section.
eg
LoadIconA function (winuser.h) - Win32 apps | Microsoft Docs
LoadIconA uses lpcstr and MakeIntResource so I’ve defined to prototypes in the module(‘win32’) section

 IS_LoadIconA(long,*cstring),long,raw,pascal,name('LoadIconA')
 IS_LoadIconA(long,ulong),long,raw,pascal,name('LoadIconA')

so the code uses procedure overloading for the windows api’s and this works well.

Same for LoadImageA as it also uses MakeIntResource
LoadImageA function (winuser.h) - Win32 apps | Microsoft Docs

Now whilst its not a prototype being used as a callback proc, I do have other callbacks working.

For example I have the DLLGetVersion callback working just fine eg.
Module(‘kernel32.lib’)
IS_DllGetVersionProc(long),long,pascal,name(‘DllGetVersion’)
End
Pragma(‘link(ISKernel32.lib’)

NotifyIcon.GetDllVersion   Procedure
Loc:lpProcName Cstring(100)
Loc:DLLGetVersionAddress Long
Loc:DllVersionInfo2Address Long
Loc:hResult Long
     Code
     Loc:lpPRocName = 'DllGetVersion'
    Loc:DllGetVersionAddress = IS_GetProcAddress(self.vLibraryAddressShell32,Lov:lpProcName)
    IF Loc:DllGetVersionAddress
       self.vDllVersionInfo2.DllVersionInfo.cbSize = Size(self.vDllVersionInfo2)
       Loc:DllVersionInfo2Address = Address(self.vDllVersionInfo2)
        Loc:hResult = self.DllGetVersionProc(Loc:DllVersionInfo2Address)
        IF Loc:hResult = 0 !S_OK
           self.vDllVersionMajor = self.vDllVersionInfo2.DllVersionInfo.dwMajorVersion
        End
   End

NotifyIcon.DllGetVersionProc   Procedure(long pDllVersionInfo2Address)
    Code
     Return(IS_DllGetVersionProc(pDllVersionInfo2Address))

but the parameters returned in resourcetype shunt left once the assembler for the callback starts.
So with a clarion app, the first resource type returned is 1492344d/16c578h which leads to the David A Bayliss dump aka DABDUMP. (Now thats interesting seeing what identifier appears online! in much the same way searching for anyscreen on shodan.io throws up lots of links :wink:), but when looking at the mappings of the stack registers to the parameter prototypes what happens is:
EAX=resource address eg Address to DABDUMP
EBX=lpType address
ECX=7h (first 64k addresses in windows are unusable)
EDX=0
ESI=Address(EBX)
EDI=64d/40h
EBP=address(self.enumerateTypesA) parent class method calling this callback procedure
ESP=? Its before the warning This program cannot be run in DOS mode.
EIP=address(self.EnumerateResourceTypeProcA) ie the address of the callback

Then when I step into the call back procedure ie first line of any local vars
EAX resource address changes to zero, losing the address in effect.
EBX becomes address stored in the phmodule (1st param in callback) when it should be lpType
ECX=7h
EDX=0
ESI=pointer to phmodule (1st param)
EDI=64d/40h
EBP= address(self.enumerateTypesA)
ESP=same address
EIP=havent established.

So the address I would expect to see in lpType (2nd param) shifts to the hmodule (1stparm) and lParam (3rd param) shifts to the lpType 2nd param and hmodule (1st param) if populated shifts to the lParam (3rd param).

Its really weird, but another oddity which I dont know if its connected, is I have separate method in this class to get the hmodule, When it does, the handle is stored in a classwide (global var) which resets to zero when I call the method that calls the callback. So now using loadlibrary to look up the hmodule in the method that calls the callback as the filename also stored in a claswide variable is not reset.

Its weird, but maybe I spent too much time on this yesterday?

I should add the same happens with the parameters for enumResourceNamesA api which has 4 parameters in the callback.

So been going through the stack registers.
EnumResouceType
1st Iteration or 1st Resource Type
When I land on the procedure call in the debugger
eax points to the BINRES or DABDUMP resource
ebx points to another location when BINRES, but doesnt point to anything if DABDUMP
ecx 6h
edx 0h
esi might be the 4 byte pointer and 2byte INT, its 6bytes before esi
edi 40h
ebp points to 4bytes before esp, and appears to be the procedure/class method address.
esp points to the same location, not yet determined what the data is at the address though.
eip points to the same location regardless of exe.

When I step onto the first local var in the procedure/class method
eax changes to 0
ebx stays the same
ecx stays the same
edx stays the same
esi stays the same
edi stays the same
ebp changes to an address 48bytes less than the first. Consistently. so procedure is getting a new address?
esp changes to an address 48bytes less than the first. Consistently.
EIP changes to an address 48bytes more than the first consistently.

2Iteration or 2nd enumerated Type
Landing on the procedure name
eax same, points to BINRES or DABDUMP
ebx same as 1st iteration
ecx 6h
edx 0h
esi points to the same location as seen in the 1st iteration or 1st enumerated type.
edi same as first iteration.
ebp same as first iteration
esp same as first iteration

Step onto the first local variable
and it all behaves as the 1st iteration stepping onto the first local variable definition in the debugger, but this time eax shows 1 instead of 0, and then increments to 2 but this appears to be the RT types as documented here Resource Types (Winuser.h) - Win32 apps | Microsoft Docs

Thing is I cant get to the stack register, and what gets sent to the declared parameters changes between landing on the procedure name and then stepping onto the first local variable in the debugger.

However whilst I write this, I’ve changed the prototype to 14 bytes allowing for the makeintresource short, and pretty much as documented above, land on the procedure you get one set of values in the stack register and byte parameters, step onto the first defined local var in the debugger and the byte parameters all change.

So far the only pattern I can see with 14 bytes, is byte parameters 11, 12 & 13 shift to 12,13 & 14.

I’m beginning to think its going to be quicker to load the exe or dll using the dos driver and then overlay the various group structures for a pe file, starting here.
IMAGE_DATA_DIRECTORY (winnt.h) - Win32 apps | Microsoft Docs and
PE Format - Win32 apps | The .rsrc Section

I cant for the life of me figure out why the parameters are changing in the callback procedure. Its weird.

TL;DR, but if you’re using a Clarion class method as the callback procedure, that ain’t gonna work. Try using a helper function that isn’t inside a Clarion class object. It won’t have that implicit first parameter that Clarion methods have.

1 Like

Such as?

I think I’ve solved it with 6 longs as the prototype for the class method.
so 1-3 are garbage, 4-6 are the parameters in reverse order, ie
long param 4 = lParam
long param 5 = type
long param 6 = module handle.

But I’d still be interested in what helper functions you suggest. One thing I did notice is if I define a procedure prototype in the class clw module, like a standard clarion procedure as a “helper function”, I cant access anything from the class, ie I cant read or write to referenced q’s or any of the properties (vars).

I see what happens with enumresourcenames as this has 4 params in the callback.

This reminds me of a comment Friedrich made in an email about 8-10years, the stack corrupts itself and I didnt believe him, but there is the [self] reference seen in the debugger and it did make me wonder why is this being passed as a param.
Anyone know?

Edit.
Just added a prototype with 10 longs, last 3 longs ie 8,9 & 10 in reverse order have the params from the enumresourcetype callback procedure. And its reporting the correct types found, including the manifest makeintresource(24) if one is embedded into the resource file.

It needs the minimum of those first 3 longs in this instance, so there is 6 longs as a minimum total, in order to get the data back, albeit in reverse order from these callbacks. I’ll have to go through the compiler switches and see if there is anything I’m missing to get it to work properly though.

If I add the pascal attribute to the class method, hmodule is not in any of the 6 longs, long param 1 shows the resource type which is the 2nd param in the windows callback prototype, and long param2 shows the lparam value from the 3rd param in the windows callback prototype. But the other problem with the pascal attribute is it removes the class method from the class, it stands isolated as an independent procedure in the debugger and I then cant access any of the class methods or properties which kind of defeats the object of using a class. So the 6 long params is a compromise and workaround I can live with in order to access the rest of the class.

fwiw. :roll_eyes:

I meant just use a procedure that is not a method of a class.

Clarion methods have an implicit first parameter which represents the class object itself (e.g. SELF).

MyObject.MyProcedure   PROCEDURE(STRING pMyString) is the same as this:

   MAP
      MyProcedure(MyObject pSelf,STRING pMyString)
   END

You can call it with either kind of syntax. e.g.

  MyProcedure(MyObject,'Test')
 or
 MyObject.MyProcedure('Test')

both ways are legal.

So that’s why a method is not really suitable for a callback. So create a procedure outside of the class that you can call.

1 Like

Ok I see what you mean by a helper function, I thought you meant some sort of Windows API helper function I didnt know about.

Now whilst your example works, for most/some situations, with these enumresource[type]/[name]/[Language] callback apis it still doesnt work, the parameters get garbled again but this time I cant use 6 longs because the compiler throws an illegal return type or attribute error, I can only use the 3 longs it expects.

But I’ll bear this technique in mind if I find a use for it.

Edit.
I wonder if this is why noone has been able to get COM to work in Clarion written in Clarion, ie because of the prototype handling like in this situation in a class? I’m going to have to find out… :wink:

I wasn’t trying to show you a technique. Just demonstrating that class methods are not suitable to use how you’re trying to use them, but I’m not sure you got that message.

If you want your (NON-CLASS-RELATED) callback procedure to “know” which instance of your class it’s dealing with, you might need to maintain a module level queue of instances or handles (or whatever you use) that it can look up and respond to. But whatever you do with that procedure, it can’t be a class method.

The 3rd parameter of EnumResourceTypesA( , , lParm) allows passing a LONG that could be the Address of the Class or a Group. The EnumProc callback function is passed this Long so can MyClass &= (lParm). This extra Long is very common in API callbacks.

[in] lParam

Type: LONG_PTR

An application-defined value passed to the callback function.

1 Like

I’ve replied here but Address(Procedure Overloading) - Address Parameter ambiguous - How can I specify which procedure? - #6 by anon23294430

but I’ll post here.

Thats not necessary as the above explains. :grinning:

I have to admit, I’m quite pleased I’ve cracked this problem! :crazy_face:

There are no examples in any of the class source code files I’ve seen, partly because of the problem with the prototyping of some callbacks in the first place.

Edit. I havent tested Queues referenced in the class yet, but everything else works just fine and I can see all the data in the class and change it just fine from the callback method. No need for any special messaging handling or any other communication methods, like synching classes. It just works.

So this problem with the parameters not matching up with the MS docs has been bugging me especially looking at this list here x86 calling conventions - Wikipedia

So @jslarve suggested having the callback procedure unconnected to the class, ie standalone, which I did but I have to wrap it in pragma statements. Adding C to the procedure prototype is not enough, the pragma statements have to wrap the procedure prototype otherwise they still come to you in the reverse order.

   Module('Classfile.clw')
   Pragma ('call (c_conv => on)')
   CallBackProc(long phModule, long pType, long plParam), Bool, C
   Pragma ('call (c_conv => off)') 
   End

The CallBackProc then receives the parameters in the correct order, no implied class parameter as the 1st parameter because its not part of the class and if you try to add the class as the 1st parameter the compiler wont let you.

It seems this callback procedure has to be external from the class, or if you want to make it part of the class, so far with testing this cludge of accepting the parameters from the callback procedure in reverse order to the MS documentation starting after the minimum of the 3 long parameters at the beginning seems to be the only way.

Well at least I’ve learnt the pragma statements can wrap procedure prototypes.

fwiw.

Then use the Pascal calling convention, which is the same as C except in reverse. Then you don’t need the Pragma() statements at all.

CallBackProc(long phModule, long pType, long plParam), Bool, Pascal

All Windows API prototypes are this way, just because you’re converting from a C declaration doesn’t mean you actually use the C calling convention
http://attryde.com/clarion/cw_prototypes.htm

Because it doesnt work, at least on my machine.

The help page titled C, Pascal (parameter passing conventions)
states

The C & Pascal attributes of a procedure prototype specifies that parameters are always passed on the stack. In 32-bit programs, both C & Pascal conventions pass the paameters to the stack from Right to Left. The difference is who (callee(C) or caller(Pascal)) is cleaning the stack on return.

In other words the C & Pascal attributes force parameters are passed on the stack instead of using the Topspeed register based passing convention, and control who cleans the stack, but they both pass parameters to the stack from Right to Left.

The pragma statement is what changes the parameter passing from Right to Left to Left to Right.

Likewise the C & Pascal attributes also “detach” the class method from the class, so it appears separate from the class in the debugger.

I think when these C & Pascal attributes where written, it was well before OOP in Clarion, so I dont know if the attributes have had any work done on them when OOP was introduced to Clarion.

Edit.

We arent supposed to use memcpy() any more
memcpy, wmemcpy | Microsoft Docs

On the point of using pointers in Clarion it says use Ulong_Ptr or Int_ptr should be used today
Rules for Using Pointers - Win32 apps | Microsoft Docs

So that would mean a Type’d group thinking ahead for 64bit Clarion.

Its interesting that Paul says use a Ulong and yet I got told to use a long, by countless other people in person and on the newsgroups and stated in the help docs, so when trying to decipher what the underlying data type is using this link
Windows Data Types (BaseTsd.h) - Win32 apps | Microsoft Docs
we can see that some datatypes describe the numerical range, like in the link “Word” is the range 0 through 65535 so that would be a Clarion Unsigned short aka a ushort.

Voids in clarion seem to be something to disable linker warnings in the compiler.

Now who wrote the help docs in the early versions of Clarion? Wasnt it Paul? I know Bob Foreman did some help doc updates when working for Softvelocity, but I see on youtube he’s now working for HPCC systems now.

But I do wonder when looking at these docs and the limited examples, if the subject was fully understood at the time it was documented?

Hi Richard,

I think you’ve over-thought this Richard. Because of the extra Class parameter you went down a road, but alas you’ve now gone round in circles. It’s time to back right up and start again.

a) I’m assuming a Windows prototype here - in which case use ,Pascal not ,C.
Windows uses pascal parameter passing, not C. so the API CALLS should be prototyped as ,Pascal,Raw

b) Add ,Raw to the prototype. It’s not really needed in this case, but won’t hurt, and is a good habit for all your prototypes.

c) the callback procedure must not be a method in the class. It can be a procedure in the class CLW.
the prototype here should be ,PASCAL as well.

d) most callbacks have a param that you set when setting up the callback, that they then pass back to you in the callback. This is most likely a long. In that case in the setup (presumably inside a class method), set this long to
address(self)
Then inside the callback procedure have something like

CallBackProc Procedure(long phModule, long pType, long plParam)
object &RichardClass
code
object &= (plParam2)
object.callback(…)

Using this approach your small helper redirects back into your own object, and so you can process the callback there if you wish.

HOWEVER be very careful with this. It’ll ONLY work if the callback is happening on the same Thread as the original object / setup. If windows starts a new thread this approach will fail. In most cases Windows will not start a new thread, but read the docs for the API carefully. Also, if you like, use THREAD() to make sure.

cheers
Bruce

I believe it is perfectly safe to use providing you know what you are doing and are careful.

Lots of programmers make “off by one” mistakes. Writing outside the intended memory will always cause potential problems. In fact putting any bugs in your code will cause potential problems.

But we should be striving for bug free code unless they are stealth revenue streams, but that would depend on how our contracts are worded which provide our remuneration packages.

For these apis and their Ex versions
EnumResourceTypesA function (winbase.h) - Win32 apps | Microsoft Docs
EnumResourceNamesA - Win32 apps | Microsoft Docs
EnumResourceLanguagesA function (winbase.h) - Win32 apps | Microsoft Docs

with these callback procedures
ENUMRESTYPEPROCA (libloaderapi.h) - Win32 apps | Microsoft Docs
ENUMRESNAMEPROCA (libloaderapi.h) - Win32 apps | Microsoft Docs
EnumResLangProc callback function (Windows) | Microsoft Docs

there is no combination of pragma statement with or without the C or Pascal attribute that works with full access to the Class methods and properties that called it unless you do whats described below!

So by having the first 3 parameters as Longs which do nothing, and then adding the parameters defined in the callback procedures above in reverse order to how MS have documented, you get the correct parameters back and it all works very nicely. :smiley:

I’ll be using this on other callback procedures of this type now that I’ve discovered it because I have the added luxury of being able to access the class in its entirety when I change the lParam datatype from a long to my BaseClassName. This is afterall what I wanted to achieve anyway, I have it working and I’m running with it.

You aint changing my mind on this one! :face_with_raised_eyebrow:

How about posting a basic example project of calling Enum Res to help others see how you made this work and try it. Post working Code like I did in CopyFileEx.

I will and it might have a price tag with it but all the information is online here but first I’m still trying to secure my now offline dev machine because some criminals keep hacking it and windows out the box is insecure, and there are a lot of active service US military personnel and British state employees both active and ex-employees involved with this hacking of my computer where I live. I dont like that!

Well the named Enumerated Types like DabDump and BinRes are interesting. I didnt quite know what to expect.

DabDump shows the ABC classes
ABEIP_RSC
ABERROR_RSC
ABPOPUP_RSC
ABQUERY_RSC
ABUTIL_RSC
and then procedures with windows (and possibly source code files havent tested) in either 8.3 or long filename form, ie the _RSC = the clw filename.

BinRes Debugview shows
RCDBGSVC
RCDBGSYS
RCDBGSYS64

Edit.
Its surprising what can be stuffed inside resource files. Here’s one example “encrypted” with the XOR97 function.
Mikko Hypponen: Fighting viruses, defending the net - YouTube

I’m getting the impression, looking at a variety of exe’s and dll’s that nobody bothers with language files.
Some resource files dont even have a language ID, but most are the system defaults.