How to verify if XML structure is correct in code

[Jane]

Cool.

(Which is apparently not a sufficiently verbose reply to satisfy the 20 character minimum.)
(Is that bytes or characters? )

[vitesse]

great thanks Jane and Bruce - I will be interested to see how it is done.

and yes the tree stuff really is cool.

I can see loading a file

xmlt  xFilesTree
  code
  if xmlt.loadFile('test.xml') = xf:error then stop('error on load').

and then accessing the nodes (XmlTree.WalkTree gives good example code) including checking the ObjectName against some required list, but I am still not clear on how to check Johan’s physical formatting in the xml file (which is largely irrelevant as far as XML validity goes) and report any error line numbers back. As mentioned previously I think they would have been abstracted away when the XML is parsed into the tree.

anyway no doubt all will be revealed in the fullness of time

cheers

Geoff R

[Johan_de_Klerk]

Hi Bruce,
I won’t be able to make today’s webinar.

Thank you in advance for your willingness to show us how to use it.
Just an extra to show if you don’t mind, I also need to do this with JSON.

Regards
Johan de Klerk

[Bruce]

If you are unfamiliar with the boundaries of XML and/or JSON then it can be easy to infer things which are not.

For example, XML allows for whitespace or not. Insisting on a specific white space (CR, LF or not) means that your requirement is not “XML”. In other words, by adding requirements which are not XML requirements, you make your format “not XML”. You might think of it as XML, But in truth it is then a new format, not XML.

At this point, clients can no longer use XML tools to create the necessary file. Since you are adding restrictions beyond the XML format, no one can then use their tool to create the format.

In other words, the very goal of making things easy, correct, and valid is negated. Generating useful error messages that help customers create files correctly is helpful. Adding requirements not in the XML spec is not.

So, before getting into the weeds of “how to validate x”, its helpful to understand whether x is inside the XML spec or not.

I will certainly give Jane (and others) an overview of loading the XML, and walking the tree, and from there Johan (and others) can add their own restrictions. But without you bring there to discuss your actual imagined restrictions, and determing if those restrictions are even useful, its hard to “cover everything”.

But dont worry Johan, we meet every Wednesday, so when you are ready yo join us you can.

[vitesse]

Hi Bruce

it seems we are finally in tacit agreement!

much earlier I said:

@vitesse

Whereas line breaks (that are not within an element) are generally not significant in XML - these and indenting are more for human readability - in your case you want each element to be on a separate line.

Whereas elements can usually be in any order, you want your elements to be in a specified order.

and:

@vitesse

You just need to be aware that you might reject perfectly valid XML where it doesn’t conform to your strict layout.

IOW what you are validating is your specific format - not XML in general.

cheers

Geoff R

[LANSRAD]

Going from “working” to “visually useful” in dealing with XML can definitely be a challenge.

You are right that generally the parser could care less about carriage returns and line feeds and most XML won’t have it because it is just extra baggage in the string. Most ignore it if it is in there, but that is not a guarantee so you have to be careful.

In our QuickBooks wrapper we added a simplified high level QBAI class that provides the ability for developers to write less code or use AI to write working Clarion code using our class by just giving it a link to a simple one page primer.

https://clarionproseries.com/ai/qbai-primer.html
https://clarionproseries.com/ai/qbai-primer.txt

Early on in development of that feature we would ask AI to start out by write the QBXML Request envelope so we could set it as a string, compile and run it against the API to see if it returned the correct results.

Then we built the QBAI code examples that recreated the same string in code to train the AI on what Clarion code it would need to create to get the same results.

Of course the XML that the AI would return was just XML with no CRLF so it was difficult to read to say the least.

We have class methods that can display the RAW XML or a Request or a Response, copy it to the clipboard or send it to the browser.

That works semi-well with small strings by displaying them in a message box.

But frankly it is hard to read and diagnose.

Where we really had to do more was when we would get the Response XML back from the QuickBooks API. Often it was a very large string and even in a TEXT control was rather useless to see if you had the results you were looking for.

The AI Code Checker

We have an interactive AI Code Checker procedure that you can call in the app which allows you to test what you send to the QB API and see the results.

It’s a very cool feature as it allows you to interact with your data live without needing to write or compile any code other than a single call to show the AI Code Checker.

We have a class method to parse the Clarion code generated by the AI and literally build/run it inside the current running program using the QBAI classes.

BTW the Clarion code coming from the AI was generally not well formatted so we parse that and do a “pretty” display so that the generated code is easy to read.

The class creates the QBXML wrapper as a string.

To show the user what is being sent, we just format it so it is readable and display it in a TEXT field on a Request XML tab. We can also display the formatted text in a message box.

But again the data coming back from QuickBooks is a single string of characters and not very useful (or readable).

So we call a class method called QBAI.ValidateResponse() that checks to see that the XML is complete, then we go through it and build a new string with indents and CRLFs so that it displays “pretty” (and easily human readable) and display that in a TEXT field on a XML Response tab..

BTW we could have used xFiles (and thought seriously about it) in the AI Code Checker but we did not want to get carried away with a tree or even queues or files because it was just to allow the user to see that they were getting what they wanted and the data was very temporary for us. So creating a new string on the fly with formatting as needed gave us what we wanted for a static display.

We did find that we could write the raw XML out to a temp file and launch that file and Windows would display it in the viewer of the user’s choice (typically their default web browser) and then that program would display it in a tree format AND the user can collapse/expand nodes on the tree.

Here are some screen shots of how it turned out and also of how the data can be shown/manipulated in the browser from the temp file.

In the AI Code Checker, the developer just pastes in the AI generated code (or types their own).

Then when they click Validate we build the actual QBAI code and execute it in real time.
When we get the response back from the API we call QBParser.ValidateResponse() to validate it and reformat the string.

We have a Request XML tab where we display a formatted version of what we sent to the API.

Also a Response XML tab where we display the formatted version of what came back from the API.

As I mentioned this works well enough for most cases as a developer can see what is being sent to verify that the Request was correct, well formatted and in the proper order.

Then seeing the formatted Response allows them to see the data groups (like multiple customer records) or the nested data for a group so they know they are getting what they wanted.

However it is a “dumb” display, not searchable, collapsible, etc.

So we added the ability to send the current tab contents to the browser by having class methods that write it to a file with a high speed API write and launch the file with ShellExecute.

The user can search with CTRL+F to jump to some data they are looking for or click the -/+ signs to expand or contract nodes to isolate what they are seeing.

It’s not as nice as having it embedded in your app, but it is another way to get the job done.

About xFiles and jFiles

Frankly, xFiles by CapeSoft is brilliant code (or jFiles if you are using JSON) and it is definitely simpler than writing a custom parser to reformat XML (even into a fixed string display).

What you use as always will depend on what your needs are, what your pain threshold is and how quickly you want to get the job done and move on to the next task at hand.

As always with Clarion it usually comes down to trying to work smarter, not harder!