I have an interesting issue that i dont understand and was hoping someone could give me some insight.
Basic overview is i have an API call that gets sent data at regular intervals, 9 out of 10 times though the data has not changed or if it has its minor shifts. So i wrote a source procedure that does a comparison of what we have in the DB vs what is sent up. However when i compare the 2 queue records to each other (both having the exact same declaration and the same data within) they fail the comparison and fall into the needs update process despite being exactly the same.
For additional context i have uploaded the same file 5 times in a row , so the 1st iteration would hit add for all records (it does) , iteration 2 - 5 should all be ignored with no changes, instead i get 1500 updated rows and 3 ignored. IE sometime it does match.
If i change the below code to explicitly check each field individually they all match and it marks all the records as Ignore correctly.
Can anyone explain why the comparison below doesnt work?
The only line i change is If OriginalLines = NewLines into a verbose per field comparison.
OriginalLines Queue(CliJLines:record),Pre(OL)
NeedsUpdate Byte(0)
NeedsDeleting Byte(1)
NeedsAdding Byte(0)
ENd
NewLines Queue(CliJLines:record),Pre(NL)
NeedsUpdate Byte(0)
NeedsDeleting Byte(1)
NeedsAdding Byte(0)
END
Clear(CliJLines:Record)
CliJLines:ClientIdentifier = INTK:ClientIdentifier
CliJLines:FK_Quosum = 0
CliJLines:FK_Quolines = 0
Set(CliJLines:Key_Client_Quosum_Quolines ,CliJLines:Key_Client_Quosum_Quolines)
Loop Until Access:ClientJobLines.Next()
If CliJLines:ClientIdentifier <> INTK:ClientIdentifier Then Break.
Clear(OriginalLines)
OriginalLines = CliJLines:Record
OL:NeedsUpdate = 0
OL:NeedsDeleting = 1
OL:NeedsAdding = 0
Add(OriginalLines)
END
Myjson.start()
Myjson.SetTagCase(jF:CaseUpper)
Myjson.Load(NewLines,STR) ! Load From a StringTheory object
Loop
If Records(NewLines) = 0 Then Break.
Get(NewLines,1)
OriginalLines.FK_Quosum = NewLines.FK_Quosum
OriginalLines.FK_Quolines = NewLines.FK_Quolines
Get(OriginalLines, OL:FK_Quosum , OL:FK_Quolines)
If ErrorCode()
Clear(OriginalLines)
OriginalLines = NewLines
OL:NeedsUpdate = False
OL:NeedsAdding = True
OL:NeedsDeleting = False
Add(OriginalLines)
ELSE
!Force The Primed Fields to Match
NewLines.PK_ClientJobLines_GUID = OL:PK_ClientJobLines_GUID
NewLines.CreatedDate = OL:CreatedDate
NewLines.CreatedTime = OL:CreatedTime
NewLines.NeedsUpdate = 0
NewLines.NeedsDeleting = 1
NewLines.NeedsAdding = 0
If OriginalLines = NewLines
OL:NeedsUpdate = False
OL:NeedsAdding = False
OL:NeedsDeleting = False
ELSE
!Lines Dont Match Update
OriginalLines = NewLines
OL:NeedsUpdate = True
OL:NeedsAdding = False
OL:NeedsDeleting = False
END
Put(OriginalLines)
END
Delete(NewLines)
END
Does the file record’s layout include fields of CSTRING or PSTRING type? Values of these types can have random characters at the tail. So, comparison of entire records can return that they are not equal because GROUPs/RECORDs are being considering as STRINGs.
If size of QUEUE record exceeds some predefined limit, the RTL clears tails of such fields for better compression. If conditions for applying compression are not satisfied, no clearing is doing.
!Force The Primed Fields to Match
NewLines.PK_ClientJobLines_GUID = OL:PK_ClientJobLines_GUID
NewLines.CreatedDate = OL:CreatedDate
NewLines.CreatedTime = OL:CreatedTime
If OriginalLines.Rec = NewLines.Rec
Your Queue(CliJLines:record) inherits that Record and gives the Lines Queues all those fields.
To reiterate Also’s post 3 does CliJLines:record have fields of the CString or PString type?
The possible garbage after the Null Terminator (or [0] length) on those string types will not be consistent and break your comparison. It would help if you post the CliJLines:Record fields here.
Another variant why comparison can return “not equal” for two records is presence of fields of DECIMAL or PDECIMAL type. DECIMAL fields can have alternative values for the negative sign nibble. PDECIMAL fields can have alternative values for the positive sign nibble.
Good evenjng
I suspect that the cstring comment here is the answer , in the morning i will manually declare each field from the table as strings and test the outcome and feedback.
As for what debug shows , if i do a comparison on each field and trace the fields that dont match i get no traces at all as the field by field comparison is a 100% match.
I will also post the record declaration if the above string test doesnt work.
It’s the CString’s. Before you assign OriginalLines = CliJLines:Record you could “clean” each CString CliJLines: Field by changing the junk in the bytes after the Chr(0) to all = Chr(0).
CleanCStringAfterNull PROCEDURE(*CSTRING InOutCstr)
X LONG,AUTO
CODE
LOOP X=LEN(InOutCstr)+2 TO SIZE(InOutCstr) !Len+1 is Chr(0)
InOutCstr[X]=CHR(0)
END
RETURN
Then you need code like this run before OriginalLines = CliJLines:Record
Clean_CliJLines:Record_ForCompare ROUTINE
CleanCStringAfterNull (CliJLines:Name )
CleanCStringAfterNull (CliJLines:Address )
CleanCStringAfterNull (CliJLines:City )
!... and the rest of the CString's
Could probably make a generic class that does this, but it would probably require the undocumented TUFO to find only CString’s. A Code template could generate all the lines.
