Compiler hangs in Debug mode

KevinErskine · November 4, 2023, 11:39pm

This is where it hangs - all my other DLLs and EXE compile w/ debug on, just not the Data DLL

Making cspwobject.obj : pragma changed
made cspwobject.obj
Making cspwmaplobj.obj : pragma changed
made cspwmaplobj.obj

4:37:01.9907576 PM	Clarion.exe	25660	Process Profiling		SUCCESS	User Time: 25.7968750 seconds, Kernel Time: 2.0000000 seconds, Private Bytes: 442,605,568, Working Set: 501,067,776	"C:\Clarion\v11\bin\Clarion.exe" 
4:37:01.9908375 PM	PrjServer.exe	16036	Process Profiling		SUCCESS	User Time: 10.0312500 seconds, Kernel Time: 2.1093750 seconds, Private Bytes: 79,503,360, Working Set: 130,801,664	"C:\Clarion\v11\bin\PrjServer.exe"  Project "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\AWGbl.cwproj" Configuration "Release" Platform "Win32" Target "Build" SolutionDir "C:\Ragazzi\Clients\Cornerstone\AFW.v40q" SolutionFileName "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\Alarmwin.sln" Verbosity "Normal" BinPath "C:\Windows\Microsoft.NET\Framework\v4.0.30319" AdditionalProperties 2 "ConfigDir" "C:\Users\kevin\AppData\Roaming\SoftVelocity\Clarion\11.0" "NoDependency" "true" MSBuildProperties 4 "BuildingInsideVisualStudio" "true" "ClarionBinPath" "C:\Clarion\v11\bin" "SharpDevelopBinPath" "C:\Clarion\v11\bin" "Signal" "ClarionSemca9588d1-501a-49b1-819d-baa9e2ef10e3"
4:37:02.9859880 PM	Clarion.exe	25660	Process Profiling		SUCCESS	User Time: 26.3906250 seconds, Kernel Time: 2.0000000 seconds, Private Bytes: 442,605,568, Working Set: 501,067,776	"C:\Clarion\v11\bin\Clarion.exe" 
4:37:02.9860061 PM	PrjServer.exe	16036	Process Profiling		SUCCESS	User Time: 10.5468750 seconds, Kernel Time: 2.1093750 seconds, Private Bytes: 79,503,360, Working Set: 130,801,664	"C:\Clarion\v11\bin\PrjServer.exe"  Project "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\AWGbl.cwproj" Configuration "Release" Platform "Win32" Target "Build" SolutionDir "C:\Ragazzi\Clients\Cornerstone\AFW.v40q" SolutionFileName "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\Alarmwin.sln" Verbosity "Normal" BinPath "C:\Windows\Microsoft.NET\Framework\v4.0.30319" AdditionalProperties 2 "ConfigDir" "C:\Users\kevin\AppData\Roaming\SoftVelocity\Clarion\11.0" "NoDependency" "true" MSBuildProperties 4 "BuildingInsideVisualStudio" "true" "ClarionBinPath" "C:\Clarion\v11\bin" "SharpDevelopBinPath" "C:\Clarion\v11\bin" "Signal" "ClarionSemca9588d1-501a-49b1-819d-baa9e2ef10e3"
4:37:03.9919760 PM	Clarion.exe	25660	Process Profiling		SUCCESS	User Time: 26.6406250 seconds, Kernel Time: 2.0000000 seconds, Private Bytes: 442,605,568, Working Set: 501,067,776	"C:\Clarion\v11\bin\Clarion.exe" 
4:37:03.9920061 PM	PrjServer.exe	16036	Process Profiling		SUCCESS	User Time: 10.7343750 seconds, Kernel Time: 2.1093750 seconds, Private Bytes: 79,503,360, Working Set: 130,801,664	"C:\Clarion\v11\bin\PrjServer.exe"  Project "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\AWGbl.cwproj" Configuration "Release" Platform "Win32" Target "Build" SolutionDir "C:\Ragazzi\Clients\Cornerstone\AFW.v40q" SolutionFileName "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\Alarmwin.sln" Verbosity "Normal" BinPath "C:\Windows\Microsoft.NET\Framework\v4.0.30319" AdditionalProperties 2 "ConfigDir" "C:\Users\kevin\AppData\Roaming\SoftVelocity\Clarion\11.0" "NoDependency" "true" MSBuildProperties 4 "BuildingInsideVisualStudio" "true" "ClarionBinPath" "C:\Clarion\v11\bin" "SharpDevelopBinPath" "C:\Clarion\v11\bin" "Signal" "ClarionSemca9588d1-501a-49b1-819d-baa9e2ef10e3"
4:37:04.9864955 PM	Clarion.exe	25660	Process Profiling		SUCCESS	User Time: 26.7812500 seconds, Kernel Time: 2.0000000 seconds, Private Bytes: 442,605,568, Working Set: 501,067,776	"C:\Clarion\v11\bin\Clarion.exe" 
4:37:04.9865159 PM	PrjServer.exe	16036	Process Profiling		SUCCESS	User Time: 11.0000000 seconds, Kernel Time: 2.1093750 seconds, Private Bytes: 79,503,360, Working Set: 130,801,664	"C:\Clarion\v11\bin\PrjServer.exe"  Project "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\AWGbl.cwproj" Configuration "Release" Platform "Win32" Target "Build" SolutionDir "C:\Ragazzi\Clients\Cornerstone\AFW.v40q" SolutionFileName "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\Alarmwin.sln" Verbosity "Normal" BinPath "C:\Windows\Microsoft.NET\Framework\v4.0.30319" AdditionalProperties 2 "ConfigDir" "C:\Users\kevin\AppData\Roaming\SoftVelocity\Clarion\11.0" "NoDependency" "true" MSBuildProperties 4 "BuildingInsideVisualStudio" "true" "ClarionBinPath" "C:\Clarion\v11\bin" "SharpDevelopBinPath" "C:\Clarion\v11\bin" "Signal" "ClarionSemca9588d1-501a-49b1-819d-baa9e2ef10e3"
4:37:05.9816247 PM	Clarion.exe	25660	Process Profiling		SUCCESS	User Time: 27.0781250 seconds, Kernel Time: 2.0000000 seconds, Private Bytes: 442,605,568, Working Set: 501,067,776	"C:\Clarion\v11\bin\Clarion.exe" 
4:37:05.9816467 PM	PrjServer.exe	16036	Process Profiling		SUCCESS	User Time: 11.1562500 seconds, Kernel Time: 2.1093750 seconds, Private Bytes: 79,503,360, Working Set: 130,801,664	"C:\Clarion\v11\bin\PrjServer.exe"  Project "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\AWGbl.cwproj" Configuration "Release" Platform "Win32" Target "Build" SolutionDir "C:\Ragazzi\Clients\Cornerstone\AFW.v40q" SolutionFileName "C:\Ragazzi\Clients\Cornerstone\AFW.v40q\Alarmwin.sln" Verbosity "Normal" BinPath "C:\Windows\Microsoft.NET\Framework\v4.0.30319" AdditionalProperties 2 "ConfigDir" "C:\Users\kevin\AppData\Roaming\SoftVelocity\Clarion\11.0" "NoDependency" "true" MSBuildProperties 4 "BuildingInsideVisualStudio" "true" "ClarionBinPath" "C:\Clarion\v11\bin" "SharpDevelopBinPath" "C:\Clarion\v11\bin" "Signal" "ClarionSemca9588d1-501a-49b1-819d-baa9e2ef10e3"
4:37:06.9806434 PM	Clarion.exe	25660	Process Profiling		SUCCESS	User Time: 27.9062500 seconds, Kernel Time: 2.0000000 seconds, Private Bytes: 442,605,568, Working Set: 501,067,776	"C:\Clarion\v11\bin\Clarion.exe"

mriffey · November 5, 2023, 2:51am

For me, it’s an appgen single exe. Looking at the task mgr, no IO bytes (read,write,other) changing on clarion.exe or prjserver.exe for minutes. Hang looks like this every time (pic attached)

also · November 5, 2023, 2:29pm

Is there some sufficient (about 100% / number of processor’s cores) CPU usage? If yes, you can use the Process Explorer to determine the thread that eats the CPU: right click on the line with Clarion.exe or PrjServer.exe depending from which is showing CPU usage and select the Properties… item. In the process properties window choose the Threads tab. Select the thread with
instant CPU usage and then press the Stack button below the thread list. The topmost entry corresponds to code that run to the infinite or very-very long loop.

If there is no sufficient CPU usage by PrjServer.exe, the problem can’t be in the linker (CLALPE.DLL).

KevinErskine · November 5, 2023, 5:22pm

Weill, i did what you said, but got error. It still must be something w/ CW or builder etc since if i turn debug off it compiles.

also · November 5, 2023, 6:31pm

2.12% of CPU usage mean that there is no synchronization conflict and execution flow is not within an infinite loop. Such percents tell that the process takes and releases CPU quantums in normal mode. Next step to try is to attach the debugger running PrjServer.exe process:

Find the PID of PrjServer.exe in the Process Explorer
Find the TID of the thread that eats the CPU time
Run

cladb -p <pid>
In the debugger open the Thread List window (main menu → Window → Thread List)
Highlight the line for the thread with found TID
Press the Ctrl-Space key to suspend selected thread and the press the Space key to show that thread’s in the Stack Trace window
Right click on the most bottom upper-level node in the Stack Trace window and choose the Locate In Assembler item in the popup menu
Make the height of the disassembler window as much as possible and provide the screenshot.

If execution flow is withing .Net managed code or thread structures are damaged (0-address as the thread start address can tell about this) failing is possible on one of steps.

All the Win32 part of CW is building using internal command-line utility which does not use .Net, and there are no any problems with making projects in any mode.

Multiple reasons are possible why results of making are different in release and debug modes can be. For example, MSBuild can fail on parsing some debug-related parameters in the CWPROJ project.

mriffey · November 6, 2023, 12:03am

Alexey - thanks for your replies on this thread. This one seemed like an easy one to test.

I set my hanging compile app to debug, tried to build. It hung. I killed the ide and prjserver and reopened the app. Debug was still on. I copied ener.cwproj to enerdebug.cwproj.

Next, I switched the project to release and compiled successfully. I then closed the app and IDE and copied ener.cwproj to enerelease.cwproj. When I opened them in Beyond Compare - they were identical.

I then went through this exercise with the SLN. Once again, there were no differences between the debug SLN and the release SLN.

I think I am missing something here. Perhaps this is related to the debug options on the IDE’s command line call to MSDebug (which I dont have).

also · November 6, 2023, 12:46am

CWPROJ files have separate section of settings applied to build the project in release and debug modes.

If you can build CW projects from IDE, you have MSBuild.

There are several reasons for me why the problem is not within compilers/linker:

If they would run in some internal infinite loop (e.g. during building of the debug info), the CPU usage would be about equal to
100%/<number of CPU cores>
Everybody can look the list of functions imported by compilers/linker DLLs. These functions can’t be a reason of locks themselves.
As follows from reports, the problem exists under Win11 only.

But to be sure, the fragment of code where the PrjServer.exe process hangs is required. If it corresponds to any Win32 part of Clarion, I hope I can identify it.

mriffey · November 6, 2023, 1:08am

Sorry - I didnt mean to imply that I dont have MSBuild. I know I have that

What I mean was that I dont have the command line parameters that the IDE uses when the IDE calls MSBuild from the command line.

However, I do have my own command line script to call MSBuild and compile a Clarion app. In order to eliminate the IDE and anything the IDE does at build time, I ran the following command which Ive successfully used many times in the past:

“C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe” c:\photoweb\enercalc.com\crm11\ener.cwproj /p:Configuration=Debug;WarningLevel=1;ClarionBinPath=c:\Clarion11\bin;PlatformTarget=x86

This command also appeared to hang, so I used the instructions you provided previously and found this result in procexp:

mriffey · November 6, 2023, 1:09am

Here’s a better view of the cycles delta column, which is still changing rapidly.

also · November 6, 2023, 2:20am

The offset 27E4h from CLALPE.DLL:SETCALLBACK corresponds to a function that processes debug records in the particular OBJ file. I can’t be sure about possible changes made in C11.13815 and later updates, but there were no changes in that code since C60.
The only reason why that code can fall into infinite loop is if the debug info in an OBJ file is damaged.

Could you look in the Process Explorer which OBJ file is opened in the PrjServer. process when it hung? If possible, send that OBJ file to me.

mriffey · November 6, 2023, 3:30am

Hello,

I am using 11.0.13244, if that helps.

I re-ran my command line MSBuild and this time, the MSBuild process seems to be jumping about between clalpe.dll!COMPILE 0x3878, and clalpe.dll!COMPILE 0x387F and the previously discussed CALLBACK offset.

All comments above relate to running MSBuild from the command
My testing now returns to the IDE for the rest of these comments:

I dont see a way to tell which OBJ is being processed. The IDE just says this: Making C:\photoweb\enercalc.com\CRM11\ener.exe

Mark_Sarson · November 6, 2023, 2:10pm

Hi

Although I do not have anymore to add than what others have reported. I will add my observations.

This issue also occurs in Clarion 10.

I use VM’s for my clients, and this occurred last November (2022) to me in a C10 data.dll running on Windows 11 pro. I ended up created a new Windows 10 VM and moved the development over to that thinking that some update must have caused the original problem.

All went well until recently (still on windows 10) when the issue started again.

I followed a tip from @ljwilson in this thread that fixed the issue, however, I believe the KB update referred to in that thread came out around August this year, so quite clearly after this happened to me the first time.

In a detailed build output on both occasions, the line where everything got stuck was “Building Debug Tables”. I’m unsure if this is everyone’s issue though.

Obviously that fix is not ideal, but I thought I would add what I know.

Mark

also · November 6, 2023, 11:46pm

I have no this version on my computer.

This code corresponds to a module with functions to handle strings. This module is from TS Modula-2 times…

In the Process Explorer:

Select the “Handles” item in the main-menu ->View->Lower Pane View submenu
Find a file with the OBJ extension in the lower pane (press the Type column header for faster search)

ChristianB · January 4, 2024, 11:20am

Hello all, I’ve been reading through this thread as I’m experiencing the same problem with some .dlls and .exe when compiling in debug mode. We are using Clarion 10.0.12799. My tests have revealed that, as for now, the only way of making this work is by using msbuild 4.7.x and p.t in a win7(!) vm. Win10 (22H2) and 11 both have msbuild 4.8.x (or newer) and compiling fails with the symptom of “compiler hang” building the debug tables. I agree it’s probably not a real hang as cpu-usage is very low. So to summarize: msbuild 4.8.x and newer has this problem. Extremely frustrating as compiling in debug mode is crucial to have available. If anyone has anything new to report…shout out!

also · January 9, 2024, 11:37pm

If someone still has this problem in C10/C11 and wants to help to find its roots, please contact me.

KevinErskine · January 10, 2024, 8:18pm

I still am unable to get a DLL app to compile with debug on v11

Bruce · January 12, 2024, 1:15pm

Thanks to work done by Alexy, With Mark Sarson providing examples, an issue in the linker has been identified. The root of the problem are procedure names being exported, with really long auto-generated (“mangled”) names.

Typically most parameter types are known by the mangler, and generated names are short. However if you are passing Objects as parameters, and passing a LOT of them, then you can exceed the linker limits, and that’s not good.

One source of this issue is CSPWCOM.INC. This file is used in Office Inside and File Explorer.

A new build of Office Inside (4.73) (CapeSoft Office Inside) has been uploaded with specific, unique, external names for these methods now set. This bypasses the mangler, and hence the length is no longer a limit.

OfficeInside 3 (and earlier) and File Explorer 5 (and earlier) also make use of this file. Since those are all obsolete now, no updates have been made to those (and none are expected to be made.) However if you are using one of these old products, and you are getting this issue, then please contact me and I should be able to supply you with an updated file. (And reasons not to be using old obsolete versions of things.)

Thanks again to Alexy for tracking down this is, and for Mark following through with examples to him.

also · January 12, 2024, 1:44pm

+building debug info records for such too long names incorrectly by compilers.

KevinErskine · January 12, 2024, 2:14pm

That makes sense to me and the DLL i have the issue with uses Office 4.71 - i will see about updating

also · January 15, 2024, 2:21pm

RZ gave permission to share fixed compilers and linker to users who can test them with projects causing the problem and who can provide the feedback. So, everybody who had the hang problem can ask me via PM.