Importing XML using XFiles4

John_Singer · March 20, 2025, 11:02pm

I’m not much of an expert on XML and am trying to get XFile4 to do all the hard work for me but am having a bit of a struggle. I’m using the latest versions of XFiles4, StringTheory & Reflection and C10.12799

I’ve tried the XML to Clarion on Capesoft’s site as well as XFiles4CodeWriter.exe on the same XML and gotten 2 different sets of queues to define the XML. Neither would compile. However I’ve modified the suggested queues to what I think best defines the XML.

I’ve got 2 problems.

I can’t get v to populate. The c and n elements get populated but never v.
Currently I’m just defining v as String(255). But in reality, v could be any length from 0 to 10000 bytes. Is there a way to allocate memory for v based on the length of the field?

I’ve attached a sample XML File (you’ll have to rename to r1.xml) and the clw code I’m using to test.

Thanks for any suggestions.

ex.clw (4.9 KB)
r1.txt (27.2 KB)

CarlBarnes · March 20, 2025, 11:26pm

I’ll post an Excerpt of Code and XML here for a peek without downloading to see XFiles in action:

root                     Group,Name('root')
version                    STRING(255),Name('version | attribute')
system                     &pQueueType,Name('system | queue | RowName(p)')
user                       &pQueueType,Name('user | queue | RowName(p)')
                         End
pQueueType               Queue,Type,Name('p | RowName(p)')
n                          STRING(255),Name('n | attribute')
c                          STRING(255),Name('c | attribute')
v                          &vQueueType,Name('v | queue | RowName(v)')
                        End
vQueueType               Queue,Type,Name('v | RowName(v)')
v                          STRING(255),Name('v | attribute')
                         End
Xml                     Class(xFilesTree)
NewPointer                 Procedure(*Cstring pGroupName, *CString pColumnName),Derived
                         End
str  StringTheory
x        long
v        long
   CODE  
   str.Start()
   str.LoadFile('r1.xml')
   str.ReplaceSingleChars('<195><190>','||')
   str.Replace('||', ' ')
   str.Replace('|', ' ')   !Pickup any strays

   if str.Length() = 0 then Message('XML is empty').
   xml.start()
   xml.SetRemovePrefix(0)
   xml.SetTagCase(xf:CaseAsIs)
   xml.Load(root, str,'root') ! Load From a StringTheory object

  loop x = 1 to records(root.system)
    get(root.system,x)
    str.trace('system: c=' & clip(root.system.c) & ' n=' & clip(root.system.n) & ' v=' & clip(root.system.v))
    loop v = 1 to records(root.system.v)
      get(root.system.v, v)
      str.trace('   v[' & v & '] = ' & clip(root.system.v.v))
   end
  end

  loop x = 1 to records(root.user)
    get(root.user,x)
    str.trace('user: c=' & clip(root.user.c) & ' n=' & clip(root.user.n) & ' v=' & clip(root.user.v))
    loop v = 1 to records(root.user.v)
      get(root.user.v, v)
      str.trace('   v[' & v & '] = ' & clip(root.user.v.v))
   end
  end
  
   xml.DisposeGroup(root) !Optional Safe Tidy up of nested pointers


xml.NewPointer          Procedure(*Cstring pGroupName, *CString pColumnName)
  ! note that pColumnName is the xml tag name, not the label. case sensitive.
  Code
  Case pGroupName
  Of 'root'
    Case pColumnName
    Of 'system'
      root.system &= NEW pQueueType  
    Of 'user'
      root.user &= NEW pQueueType  
    Else
      self.trace('ERROR 1 in xml.NewPointer pGroupName=' & pGroupName & ' pColumnName=' & pColumnName)
    End
  Of 'root.system'
    Case pColumnName
    Of 'v'
      root.system.v &= NEW vQueueType  
    Else
      self.trace('ERROR 2 in xml.NewPointer pGroupName=' & pGroupName & ' pColumnName=' & pColumnName)
    End
  Of 'root.user'
    Case pColumnName
    Of 'v'
       root.user.v &= NEW vQueueType
    Else
      self.trace('ERROR 3 in xml.NewPointer pGroupName=' & pGroupName & ' pColumnName=' & pColumnName)
    End
  Else
    self.trace('ERROR 4 in xml.NewPointer pGroupName=' & pGroupName & ' pColumnName=' & pColumnName)
  End

R1.TXT

<?xml version="1.0" encoding="UTF-8"?>
<root version="1.1">
	<system>
		<p n="version" c="s">
			<v>1.6</v>
		</p>
		<p n="lastModifiedUserId" c="s">
			<v>treasurerint</v>
		</p>
		<p n="WorkstationId" c="s">
			<v>tylrEAGLE</v>
		</p>
		<p n="System_ExternalId" c="s">
			<v>R033781</v>
		</p>
	</system>
	<user>
		<p n="TaxRollId" c="s">
			<v>2024.001260</v>
		</p>
		<p n="AccountId" c="s">
			<v>R033781</v>
		</p>
		<p n="Situs" c="s">
			<v>þþþþþþþ</v>
		</p>
		<p n="OwnerId" c="s">
			<v>OWN68S913</v>
			<v>OWN68S914</v>
		</p>
		<p n="OwnerName" c="s">
			<v>MEARS SCOTT ETAL JT</v>
			<v>MEARS JULIETTE ETAL JT</v>
		</p>
		<p n="PlssLegal" c="s">
			<v>5Eþ15þ30þþþþþþþ</v>
		</p>
		<p n="PlattedLegal" c="s">
			<v></v>
		</p>

…

		<p n="Area" c="s">
			<v>15</v>
		</p>
	</user>
</root>

Bruce · March 21, 2025, 3:41am

John, im away on leave so not near a computer. If you dont have this sorted out by monday, feel free to bump it.

Regarding the string length, changing the declaration of the string to

V    &stringtheory, name ('v | stringtheory')

Should sort thst out. The load will NEW the object for you. You should use DISPOSEQUEUE anyway to tidy it all up at the end.

Regarding the NewPointer code, make sure all paths are catered for. In other words use TRACE to see what comes into NewPointer, and make sure those bases are covered.

Just reading the code, i dont think v should be zn attribute. And i thjnk the rowname for the vqueue should be blank.

CarlBarnes · March 21, 2025, 3:01pm

The way I usually attach files is all in one ZIP file.

Then you can name them anything (like .xml) and include any files. Like here it would good to have the .CwProj file because it has all the Defines for the CS Classes else they will GPF.

Excellent example !!! It’s the minimum of what’s needed and a Project not an App.

John_Singer · March 21, 2025, 3:51pm

Good suggestions. I’ve attached a zip file with the cwproj & sln and the updated source. I incorporated the suggestions from Bruce. Sadly it did not improve the results.

ex.zip (5.0 KB)

CarlBarnes · March 21, 2025, 5:49pm

I’ll post the code from your ZIP at the end so Bruce can see it on his phone.

Below in your code it looks like you did not change V to a &StringTheory as suggested, you changed it to remove | RowName(v)'. Then changed vQueueType’s field from STRING(255) to &STRING,Name('v | &String | RowName(v)'). I don’t know much about XFiles, I mainly know how to post here.

root        Group,Name('root')
version       STRING(255),Name('version | attribute')
system        &pQueueType,Name('system | queue | RowName(p)')
user          &pQueueType,Name('user | queue | RowName(p)')
            End
pQueueType  Queue,Type                         !Was: ,Name('p | RowName(p)')
n             STRING(255),Name('n | attribute')
c             STRING(255),Name('c | attribute')
v             &vQueueType,Name('v | queue')    !Was: | RowName(v)')
            End

!was:  vQueueType  Queue,Type,Name('v | RowName(v)')
!was:  v              STRING(255),Name('v | attribute')
!was:              End
vQueueType  Queue,Type
v              &STRING,Name('v | &String | RowName(v)')
            End

Xml         Class(xFilesTree)
NewPointer     Procedure(*Cstring pGroupName, *CString pColumnName),Derived
            End
str  StringTheory
x        long
v        long
   CODE  
   str.Start()
   str.LoadFile('r1.xml')
   str.ReplaceSingleChars('<195><190>','||')
   str.Replace('||', ' ')
   str.Replace('|', ' ')   !Pickup any strays
   loop while str.Replace('  ', ' ').   !New

   if str.Length() = 0 then Message('XML is empty').
   xml.start()
   xml.SetRemovePrefix(0)
   xml.SetTagCase(xf:CaseAsIs)
   xml.Load(root, str,'root') ! Load From a StringTheory object

  loop x = 1 to records(root.system)
    get(root.system,x)
    str.trace('system: c=' & clip(root.system.c) & ' n=' & clip(root.system.n) & ' v=' & clip(root.system.v))
    loop v = 1 to records(root.system.v)
      get(root.system.v, v)
      !str.trace('   v[' & v & '] = ' & root.system.v.v.GetValue())
      str.trace('    v[' & v & '] = ' & clip(root.system.v.v))
   end
  end

  loop x = 1 to records(root.user)
    get(root.user,x)
    str.trace('user: c=' & clip(root.user.c) & ' n=' & clip(root.user.n) & ' v=' & clip(root.user.v))
    loop v = 1 to records(root.user.v)
      get(root.user.v, v)
      !str.trace('   v[' & v & '] = ' & root.user.v.v.GetValue())
      str.trace('    v[' & v & '] = ' & clip(root.user.v.v))
   end
  end

John_Singer · March 21, 2025, 6:07pm

I did try it and it did not help. (There is a remnant in the loop that is exporting the v node.) My thinking was I would like to get it to work with just a basic data type and then I could upgrade to a StringTheory object. In thinking about a variable that is a ST object brings up a lot of very interesting possibilities on what can be done to the object before you commit it to a table.

John_Singer · March 21, 2025, 10:02pm

Progress has been made. After tinkering with the queues and thinking about StringTheory, it dawned on me where the ST declaration should go. By moving it into the pQueueType definition it consolidated and simplified the creation and especially the loading of the queues.

This is what the new struct looks like now:

root                     Group,Name('root')
version                    STRING(255),Name('version | attribute')
system                     &pQueueType,Name('system | queue | RowName(p)')
user                       &pQueueType,Name('user | queue | RowName(p)')
                         End

pQueueType               Queue,Type !,Name('p | RowName(p)')
n                          STRING(255),Name('n | attribute')
c                          STRING(255),Name('c | attribute')
v                          &StringTheory,Name('v | StringTheory')
                        End

And it worked for the most part. I was getting exactly what I expected. The only caveat is if there are multiple v rows in the same p node. I was hoping that the xml.Load would see this and create multiple lines in the ST object using st.AddLine or string them together with a delimiter of some sort to separate them. As of now it only adds the first v into the Value field.

		<p n="OwnerId" c="s">
			<v>OWN68S913</v>
			<v>OWN68S914</v>
		</p>

Once this is issue is solved, I will be able to start on the much larger project that this will be used on!

Attached is latest source.

Thank you.

ex.zip (4.8 KB)

Bruce · March 24, 2025, 4:18am

So first things first, No John, that’s not forwards that’s backwards - and unfortunately that approach will not work.

For reasons I’ll get into below, you’ll want to grab this morning’s builds of xFiles (4.35) and Reflection (1.31).
This answer is in 2 parts; first the simple version, then the longer version where v can be of undetermined length.

your original code would work with these structures;

root                     Group,Name('root')
version                    STRING(255),Name('version | attribute')
system                     &pQueueType,Name('system | queue | RowName(p)')
user                       &pQueueType,Name('user | queue | RowName(p)')
                         End
pQueueType               Queue,Type,Name('p | RowName(p)')
n                          STRING(255),Name('n | attribute')
c                          STRING(255),Name('c | attribute')
v                          &vQueueType,Name('v | queue | RowName(v) | XmlName()')
                        End
vQueueType               Queue,Type,Name('v')
v                          String(255),Name('|')
                         End

This mostly follows Layout 4, but with a wrinkle, so I’ve added Layout 5 to the docs to cover it.

Your next requirement was to make v into a StringTheory object so as to allow for unlimited length. That’s straight-forward enough;

root                     Group,Name('root')
version                    STRING(255),Name('version | attribute')
system                     &pQueueType,Name('system | queue | RowName(p)')
user                       &pQueueType,Name('user | queue | RowName(p)')
                         End
pQueueType               Queue,Type,Name('p | RowName(p)')
n                          STRING(255),Name('n | attribute')
c                          STRING(255),Name('c | attribute')
v                          &vQueueType,Name('v | queue | RowName(v) | XmlName()')
                        End
vQueueType               Queue,Type,Name('v')
v                          &StringTheory,Name('| StringTheory')
                         End

The only change there is in the declaration of the second v.

This though caused a wrinkle, because Reflection optimised blank field names away from some methods (which it mustn’t do) and also xFiles optimised setting fields with blank names (which it mustn’t do). Hence the updates to both xFiles and Reflection to support this case.

Your code to view the data needs to be upated to match, but I think you’ve got a handle on that.

John_Singer · March 24, 2025, 3:00pm

After making the suggested changes and downloading the latest version, it imported perfectly!

Thank you for your help and for making the modification to xFiles4 and Reflections.