Msbuild example for Clarion

[brahn]

I originally posted this as a gist but never wrote a blog or otherwise published it. Maybe someone here will find it helpful!

msbuild example for Clarion

This is an example of how to build a Clarion multi dll project from the command line.
This was useful to be able to correctly build a multi-dll project containing circular references.

---

The BAT file

There are two calls so everything gets compiled.
Output is piped to log files for later reference

%windir%\Microsoft.NET\Framework\v2.0.50727\MSBuild master.msbuild /p:ClarionBinPath="c:\Clarion8\bin" /p:NoDependency="true" /p:Configuration="Release" /p:CopyCoreFiles="true" /target:First > build_first.log
%windir%\Microsoft.NET\Framework\v2.0.50727\MSBuild master.msbuild /p:ClarionBinPath="c:\Clarion8\bin" /p:NoDependency="true" /p:Configuration="Release" /p:CopyCoreFiles="true" /target:Second > build_second.log

contents of master.msbuild:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Circular"  
  xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <ItemGroup>
        <FirstPass Include="global.cwproj;
  	manager.cwproj;
		sec.cwproj;
		integration.cwproj;
		utils.cwproj;
		scheduler.cwproj;
		reports.cwproj;
		settings.cwproj;
		import.cwproj;
		orders.cwproj;
		quotes.cwproj;
		products.cwproj;
		servicemgr.cwproj;
		knowledgetree.cwproj;
		crm.cwproj;
		"/>
    </ItemGroup>
    <ItemGroup>
        <SecondPass Include="integration.cwproj;
        	quotes.cwproj;
		orders.cwproj;
		servicemgr.cwproj;
		master.cwproj;
		"/>
    </ItemGroup>
    <Target Name="First">
        <MSBuild Projects="@(FirstPass)"  
          Targets="Build" ContinueOnError="true" />  
    </Target>
    <Target Name="Second">
        <MSBuild Projects="@(SecondPass)"  
          Targets="Build" ContinueOnError="true" />  
    </Target>
</Project>

Original Gist:

https://gist.github.com/fushnisoft/5772134

[StuAndrews]

Hey Brahn, what’s the reasoning for two parts, why couldn’t you put all your apps into the first list?

[brahn]

Circular dependencies in the case of this solution. Clarion PE does not include the smarts to deal with the multiple compiles needed to resolve the problem so I solved it this way instead

[brahn]

Instead we have a hand coded project that includes a .prj with a bunch of #implib statements that creates all of the libs from all of the EXPs. That way we can do a single pass compile.
– c/o @d.harms

Neat idea Dave, thanks!

---

Excerpt From “Basic Compiling and Linking”

#implib

#implib <libfilename>

The #implib command is used to create (if necessary) a dynamic link library file. There are two forms of this command, which operate in slightly different ways. If a single filename is specified, this names an import library file, which is created (if not up-to-date) from the object files in the link list. The object files are scanned and each public procedure or variable is exported. For example:

#pragma link( fred.obj, joe.obj )
#implib mylib.lib

In the second form of the #implib command, an import library filename and a module definition file (.exp—see Module Definition File below) are both specified, and the library file is created (if not up-to-date) from the symbols named in the definition file. This form of the command is equivalent to using the tsimplib utility that comes with SoftVelocity C, C++, and Modula-2.

#implib <expfilename> <libfilename>

Using #implib in the second form requires you to create and maintain the list of exports ‘by hand’, whereas the first form exports all public names automatically. The use of a module definition file is an advantage if you need to maintain compatibility with previous versions of an interface, and it also allows you to export only the procedures which need to be exported.

When #implib is used with a module definition file, the link list is cleared.

---

[StuAndrews]

Good stuff Brahn, thanks. And thanks DH!

[MarkGoldberg]

On a somewhat related subject.
Here is a batch file (.cmd is the same as .bat), That I use to call MsBuild.
It’s part of the Clarion package for use in Sublime Text, and VS Code
It’s designed to take arguments, because I call it via the Sublime Text build project system
A given project is configured for what to compile, the configuration (build or release) and version of clarion - given by a folder

fushnisoft/SublimeClarion/blob/master/bin/CompileCW.cmd

Note in ST3, packages are distributed in a zipped form
so you’ll need to extract the .cmd from inside of the package to be able to call it.

In your Sublime Text Project settings:
“build_systems”:
[
// see http://docs.sublimetext.info/en/latest/reference/build_systems.html
{
“name”: “Clarion”,
“cmd”: [
“${packages}\…\Installed Packages\CompileCW.cmd”,

             // What to build, convention is the same folder and basename as this sublime-project file, but with a .sln instead
             "${project/\\.sublime-project/\\.sln/}",

             // Build Configuration typcially Debug|Release  
             "Debug",
             // "Release",

             "C:\\SV\\Clarion10.0.12567",

             //(Optional) CW_Ver, such as "8.0.9579" will default to current
             ""                   
    ],
    //uses perl style regex: (maybe this) http://www.cs.tut.fi/~jkorpela/perl/regexp.html
    "file_regex": "^  \\s*[2>]*(.*)\\(([0-9]*),([0-9]*)" 
    // hard code in the 2 spaces at the start of the line, to JUST use the final summary error lines
    // CON: this means I can't click on rows as the compile occurs
    // PRO: In line error messages are no longer duplicated

    //"working_dir": "${project_path}:${folder}}",
    //"shell" : true,           
}

[CarlBarnes]

Handling circular references with #implib
https://clarionmag.jira.com/wiki/spaces/clarion/pages/399918/Handling+circular+references+with+implib

I found Dave Harms had written the above a CMag article on the topic of using #ImpLib with nice screen captures. Be aware the article example uses the wrong syntax that flips the file names as LIB EXP. In his post here he shows it right as #ImpLib DllA.EXP DllA.LIB

I used the Article to make an example using DLL Tutor that I attached.

1. Make a Win32 Project with a CLW that has nothing but MAP. CODE
2. Make a LibCreate.PR file like below and add it to the SLN under Projects to Include
   ---- LibCreate.PR ----
   #implib ALLFILES.EXP ALLFILES.LIB
   #implib REPORTS.EXP REPORTS.LIB
   #implib UPDATES.EXP UPDATES.LIB

I don’t use this, I do a 3 pass MSBuild of DLLs, DLLs with Mutual Imports, EXEs. I’m not entirely sure how ImpLib can work with source control and multiple developers. It seems like a “chicken and egg problem.” The EXP file needs to be in Source control for use by the build server to perform #ImpLib. That file is generated and I’m not sure how stable it is so will it merge well?

ImpLibExampleForDllTutor.zip (2.5 KB)

[CarlBarnes]

Dave’s article had the correct syntax of #implib LIB EXP, it was wrong as #implib EXP LIB in the excerpt from page 130 of AdvancedTopicsReferenceGuide.pdf, the example projects in that PDF do show correctly e.g. #implib %%drvname%%.LIB %%drvname%%.EXP. The wrong syntax will trash your EXP and not give you a LIB.

I attached a corrected DLL Tutor example project to generate LIBs with the bwlow syntax. There’s not need to do AllFiles since if it is the first one built.
#implib ALLFILES.LIB ALLFILES.EXP
#implib REPORTS.LIB REPORTS.EXP
#implib UPDATES.LIB UPDATES.EXP

I also changed its target to a LIB so there’s never a pointless EXE, and then in the Post Build Cmd is DEL LibCreate.lib

My tests show this is optimized to only make the new LIB if it changes, i.e. if you run it twice in a row the LIB is not created again. I would guess it makes it, checks its the same, then does not replace it, but I did not test this. The project language does allow “#if reports.lib #older reports.exp #then #implib …”

ImpLibExampleForDllTutor_Corrected.zip (12.4 KB)

[CarlBarnes]

@CarlBarnes

I’m not entirely sure how ImpLib can work with source control and multiple developers. It seems like a “chicken and egg problem.” The EXP file needs to be in Source control for use by the build server to perform #ImpLib. That file is generated and I’m not sure how stable it is so will it merge well?

I realized this is not a check-n-egg problem and will work perfectly. Unlike interactive building using the IDE when you build in a Batch mode:

1. First you Generate CLWs for every APP and that makes the EXP files for every DLL
2. Next you run MSBuild on all the CwProj’s to compile the CLWs that were generated

The first CwProj simply needs to be the Lib Creator. I like this idea and will try it next week so I can take out step 2 of the MSBUILD that re-compiles the DLLs with mutual imports.

The one possible issue is since Clarion 7 you can skip the EXP file and specify EXPORT on a procedure or data. This is similar to C++ declspec(dllexport). The Linker will add it to the LIB file and not the EXP so #implib would make a bad file. I almost never do this, and only for quick and dirty tests or prototypes. I tried it but decided I liked the EXP because I use it for other things.

You can use LibMaker to go get the mangled name, then edit the EXP to add it. You must remove EXPORT from the procedure or it will be a duplicate. (Or easier, prototype in the importing project and the linker errors will have the mangled names it wants.)

7.1 preliminary release notes and a Happy Thanksgiving! - Clarion

FEATURE: New attribute for procedure prototypes and static variables declarations: EXPORT. The EXPORT attribute forces the variable or procedure to be added to the export list of the DLL even if it is not listed in the EXPORTS section of the EXP file.

There is also this feature. I forgot the reasons, I guess its like Name(‘xxx’). It can be found in other build systems like Visual Studio.

FEATURE: Ability to use substitution of exported name in the EXPORTS section of the EXP file. The entry in the EXPORTS section can have format [=] @?

[CarlBarnes]

The below DOS command will create the #implib lines in file ImpLib.TXT to paste into the .Pr file. This assumes the EXP files are present. It checks IF Exist .EXE and does not generate those.

DEL ImpLib.TXT & FOR %f IN (*.exp) DO IF not EXIST %~nf.EXE ECHO #implib %~nf.LIB %f >> ImpLib.TXT

In a BAT file you need % doubled to %%

@ECHO OFF
ECHO This will make a ImpLib.TXT file with all the lines for the PR 
pause
del implib.txt
FOR %%f IN (*.exp) DO IF not EXIST %%~nf.EXE ECHO #implib %%~nf.LIB %%f >> ImpLib.TXT
Notepad ImpLib.TXT

In one small project (8 DLL / 5 EXE) the second pass to compile the 4 DLL APPs with mutual imports only took 9 seconds. So skipping pass 2 by changing to use #implib will not see time any significant time savings. It will eliminate import errors on the first pass, and the build order will no longer matter (but I would try to do them in order). its the way to go.

[Rick_UpperPark]

This is very nice @CarlBarnes. Just used this technique on a project with some circular references where I was building into a new project folder that had never been built in.
I added the .PR with the implib statements to my data DLL since it always gets compiled first, rather than a separate project.

[CarlBarnes]

Thank you for the reminder on this! I have a C5 project where a Single DLL compiles to export 24 Classes. I need to split that into 24 DLLs of one class each. I first wrote a program to split the 3000 line EXP into 24 files. Them I used this to make 24 LIBs.

I’ll list the steps I took to make it easier to do this next time:

1. Create the ImpLib2Txt.BAT file in post #10 in the Project folder
2. Run ImpLib2Txt.BAT to create ImpLib.TXT with #ImpLib lines
3. Download ImpLibExampleForDllTutor_Corrected.zip attached in post 8
4. Unzip in your project folder
5. Edit LibCreate_ImpLib_prj.PR to add the #ImpLib lines from ImpLib.Txt in step 2
6. In Clarion Open LibCreate_ImpLib.cwproj
7. Build and it should make all your LIBs

---

Screen captures from steps 1 2 5 7

---

The reason I’m doing this is the Classes have Topspeed File declarations in the Class CLW. The C5 builds fine with those files are encapsulated as local. In C8 - C11.1 build throws duplicate symbol errors on the Keys. I’d say the compiler is making them public for the linker. So I am trying putting each Class in its own DLL since those KEY symbols are not DLL public.

[anon23294430]

Has anyone created a template for the MS Build system, ie parsing and reading it in into a template which can be used to make settings changes before exporting it back? I cant find anything suggesting a template exists searching this forum and relevant posts eg
Get Compiler Version & Build Config for use in a PostCompile Build action - code / Snippets - ClarionHub

msbuild example for Clarion · GitHub
Too much hand code for my liking, I’d rather modify it from within a template that guides the decision making process.

And is this factually correct, you cant alter the project settings when using the appgen in C6 or early?

TIA

[jslarve]

@anon23294430

And is this factually correct, you cant alter the project settings when using the appgen in C6 or early?

You might be able to. And if you’re not, there’s nothing stopping you from generating an external .prj (a la Capesoft multiprj)

[anon23294430]

I’m currently generating my own prj file from my templates, I didnt know about capesofts multiprj. Now I know.

[Rick_UpperPark]

Here are the basics for building a call to msbuild to compile your .cwproj files.

"<msbuildPath>\msbuild.exe" msbuild.exe /property:NoDependency=true /property:ClarionBinPath="<Path to Clarion.exe>" /l:FileLogger,Microsoft.Build.Engine;logfile="<Path to log file>";Append=true "<Path to .cwproj>"

You can include optional parameters after the ClarionBinPath like:

/property:Configuration=[Debug|Release]
/property:clarion_version=“clarionversion”

The clarion_version property is used when you are using the IDE’s “version” feature. In other words, you are using one IDE to compile with multiple different Clarion installation you have on disk. See the help topic “Versions - adding to the IDE”

The installed version of msBuild can be found in the registry Local Machine: HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\MSBuild\ToolsVersions