A Clarion TPS Parser Class Born from a Corrupted File, a Java Utility, and an AI Coding Agent

1

A few days ago, a user reported that a Clarion application was hanging when opening a window that reads a TPS file containing data and thumbnail images stored as BLOBs.

As expected, the TPS file was corrupted. I tried every recovery method I knew, including TPSFIX plus a example file, but the application still kept hanging.

I then wrote a small utility to try to extract the data by reading the file in physical order, or by record number. In every case, though, reading certain records caused FILEERROR() to return what looked like GPF text, and after that the program either became unstable or locked up completely.

At that point I had run out of ideas, but then I remembered the article Liberating Clarion TPS files, which includes a Java utility for reading TPS files. I had nothing to lose, so I gave it a try. To my surprise, it recovered almost all of the records and BLOBs, skipping over pages that contained invalid data.

I immediately wrote a small Clarion program to read the generated CSV and .BIN files and write the data into a new TPS file. That solved the user’s problem.

Afterwards, I decided to turn the experience into a proper tool in case I ever ran into a similar problem again. Since I do not really work with Java, I asked my favorite AI Coding Agent to port the Java utility to C# so I could make changes more comfortably. I expected that to be a fairly involved process, but the agent finished it in a little over five minutes. It then ran comparison tests against the Java version and passed them all.

Seeing that it had no trouble translating the logic to C#, I decided to try taking it all the way to Clarion. A few minutes later, I had Clarion code passing the same tests. From there, I iterated with the agent to build a reusable class with a simple public API.

The result is now available in the tpsparser repository on GitHub. Here is a basic example of how it is used:

Parser  TpsParserType
  CODE

Result = Parser.Init('C:\data\MyFile.tps')
IF Result <> 0
  MESSAGE('Could not open/parse TPS, error:' & Parser.GetError())
  RETURN
END

Parser.Set()
LOOP UNTIL Parser.Next() 
  pre:EmployeeName = Parser.GetField('EmployeeName')
  pre:DateHired    = Parser.GetDateField('DateHired')
  ...
END

What I found most interesting is that all of the code and documentation in the repo were written by the agent. The only part I hand-coded was the demo in Test.clw.

My takeaway is that, at least for CLW projects with little UI, coding agents are ready to solve complex Clarion problems.

I plan to use this class in the future. It will come in handy for reading data from other Clarion systems or older versions, avoiding Error 47. It could also be used as a fallback in a future version of FM3.

For reference, I used Pi Coding Agent with a ChatGPT subscription and the GPT-5.5 model.

tpsparser-test
tpsparser-test888×548 15.4 KB

2

That is a very interesting example, and I think it shows one of the places where AI coding tools can be genuinely useful.

What stands out to me is that this was not just “AI, go write me a TPS parser”.

There was an existing Java utility, a real-world corrupted file problem, a clear recovery goal, and a way to compare the generated results against known working behavior. That gives the AI rails to run on.

That is where I think AI does very well. Take known logic, port it, wrap it, test it, and iterate with a developer who understands the problem domain.

I agree that one should still treat something like this as a recovery/import/inspection tool rather than a normal TPS access replacement, especially since reverse-engineered TPS parsing is always going to need careful verification. But as a practical tool for rescuing data or avoiding Error 47 during conversions, this could be extremely valuable.

It also makes another useful point.

You do not necessarily have to be using a big agentic coding setup or a particular subscription plan to produce valuable work with AI.

Those tools may be useful, but the real leverage comes from having a clear problem, good source material, tight scope, and a developer who stays in control of the process.

To me, that is the bigger lesson. AI works best when the human owns the architecture, the goal, and the validation.

3

Thanks for the idea, but please tell me how you think your PI is better than using an agent directly (like Claude, ChatGPT, etc.)

4

Well done Carlos.

By chance, just a few weeks ago I wrote something similar also based on the reverse engineering from the same Liberating Clarion TPS files article.

Mine was a bit different and simpler using StringTheory and just spitting out a CSV file.

I used Gemini and Claude to review the code and make suggestions but I was amazed yours took only a few minutes whereas I spent a few days getting it to work.

I am travelling this month without my computer so cannot see my code but when I return home I will compare them just out of interest.

Cheers for now

Geoff R

5

Hi Charles,

Thank you for your comments.

I agree 100%. AI coding agents are tools for developers, not replacements.

6

Hi Geoff,

The difference is probably the model.

I have been experimenting with AI coding for about a year, writing C# code, Clarion code, and C# code that interacts with Clarion. I have tested many combinations, but I have found that GPT models, starting around version 5.2, almost always produce complete and correct code.

7

That is an impressive conversion, good looking code and useful!

I Forked your TpsParsder Repo to try it out. I was curious what data was inside the class so I added a method named DebugClassWindow() that is shown below. A Window with LIST’s to show the 4 queues. If you double click on a line it opens a window to show the Hex of the Payload.

image
image931×966 69.1 KB

I added a Pull Request for you to take this.

I saw the 4 Queues that all had a Payload STRING(32768). I wondered if that was actually used and these browses show the PayLoadLen (aka DataLen) is in the 100’s … it would be the Record size. Being the last STRING it should get clipped, but it would probably be better as an &STRING that is NEW(STRING(PayloadLen)).

The change I would make is PRIVATE to PROTECTED. That would allow this kind of Debug code to be in a separate class file that inherits,

8

Hi Daniel,

If you don’t mind, let me set up a few concepts before answering your question.

First, we have the labs: Anthropic, OpenAI, xAI, Google, and some other smaller labs.

Then we have the models from each lab: Anthropic’s Claude, OpenAI’s GPT models, Google’s Gemini, xAI’s Grok, etc.

As users, we interact with those models through “harnesses”, which are apps that take our prompt, send it to the model along with some tools, and then show us the model’s response. The most famous one is probably OpenAI’s ChatGPT.

One type of harness is the AI coding harness. Some are created by the labs themselves to use their own models, such as Anthropic’s Claude Code and OpenAI’s Codex. There are also third-party coding harnesses that can use models from different labs, and may have different architectures and feature sets, such as GitHub Copilot, OpenCode, and Pi.

Pi is a free, open-source coding harness created by Mario Zechner. You can read about its goals and benefits on its homepage, in its GitHub repository, or in this post by the author: What I learned building an opinionated and minimal coding agent.

Now, to answer your question: for me, the main advantages are that it is very lightweight and token efficient, lets me use various model providers, and can be extended or self-modified to fit particular needs.

For example, at the moment I have two ChatGPT Plus subscriptions, one personal and one for work, which give me access to the GPT models available in ChatGPT. I also have a GitHub Copilot subscription, which gives me access to several models from different providers. I can use all of those accounts and models from Pi.

Another example: Clarion uses ANSI-encoded .CLW files. Most coding agents, including Pi by default, only handle UTF-8 files. I asked Pi to create an extension for itself that adds ANSI-aware editing tools, and I use that to edit Clarion files.

In the end, it is just a matter of choosing the combination of harness and model that you are comfortable with. For now, I like Pi with GPT-5.5.

9

Hi Carl,

Thank you very much for your comments.

I still have not reviewed the code generated by GPT in depth, and I had to leave on a business trip the day after I made the post.

I think your comment about changing STRING(32768) to &STRING is important. As soon as I am back, I’ll review your PR and get back to you.

10

Thanks for the detailed explanation.

11

Thanks Carlos

I have found that GPT models, starting around version 5.2, almost always produce complete and correct code.

That’s interesting and I will have to try ChatGPT again when I get to my computer. I had tried it a while ago and found it hopeless so I gave up pretty quickly, but perhaps it has improved a lot since then.

Even Claude and Gemini code often is more of a rough guide than actual usable Clarion code with function names hallucinated or borrowed from other languages and just really inefficient code with clips everywhere.

Cheers Geoff R

12

Geoff,

I think one of the big differences with Clarion is that the instructions matter a lot more than they might with some of the more commonly used languages.

If I just ask an AI to “write some Clarion code,” I may still get code that looks plausible but has function names that do not exist, techniques borrowed from C#, JavaScript, or Python, or Clarion code that technically resembles Clarion but would not compile or would be a maintenance headache.

But if I give it specific instructions, the quality improves dramatically. Simple rules make a big difference. Some are coding patterns, some are personal preferences. For example:

Do not invent Clarion functions that do not exist.

Do not borrow syntax or coding patterns from other languages.

Do not assume a method exists unless it has been shown in the INC or CLW.

Use standard Clarion syntax.

Use full IF / END blocks.

Do not use one-line IF statements.

Do not use Clarion reserved words as variable names.

Do not scatter CLIP() everywhere unless it is actually needed.

That kind of guidance can be the difference between “interesting rough guide” and “usable code.”

I think this is especially important because Clarion is not represented in the training data at anything like the same level as C#, Python, JavaScript, or Java. So the AI needs a tighter harness. It needs examples, rules, and correction when it drifts.

The mistake, in my opinion, is treating AI as if it already knows your Clarion coding standards. It usually does not. But when you give it those standards clearly, and especially when you give it the existing INC and CLW files to work from, it can become much more useful.

So I would not say “AI writes perfect Clarion code now.” I would say that AI can write much better Clarion code when it is constrained, guided, and reviewed by somebody who actually knows Clarion.

Charles