Here is a reproduction of my old PTSS report. @Mark_Sarson might be able to describe the situation he found today with phantom records due to the same issue?
Problem Reference: 39361 Date Received: 2012-05-30 Status: Opened for Review Fixed in Build Number: Product: Clarion8 Build No: 89285 Operating System: Category ID: ABC Classes
Summary: Variables used with return value from Position() are not large enough
Description:
The ABC Browse template and classes have a String(1024) declared to hold the return value from Position(). There are circumstances where this is not large enough.
When the templates/classes subsequently try to use a truncated saved position in a call to Reset or Reget an Error 78 is thrown.
Steps to Reproduce:
This is occurring on an SQL based view with several joined tables.
It is currently triggered when used in conjunction with the SortHeader extension/class.
Changing the browse template and classes to have a larger string stops the Error 78 from occuring.
I have marked this as reproducible=always but it does depend on view structure and data withing the view so it is also not always easy to reproduce.
Only tested in Clarion6 but Clarion7-8 have the same template/class string sizes.
Reproducible: Always Severity: Major Visible to public: Yes Example App: None submitted
Our issue, which is caused by the same problem (Storage variables defined to store the POSITION(file.key) etc are not large enough @1024 characters.
The problem I had was with Browses that would have a column that is defined fairly large on the backend (In this case VARCHAR(512). Upon clicking the header to sort the browse in the order of that column, when I scrolled the scroll bar, we would see multiple rows with the same data.
This of course is a classic symptom of not using the pk filed in sort orders, however in our case we were using the primary field, so we could rule that out.
After searching the CLW’s, INC’s and TPx files I modified any local variables that were used to store position in the classes to ANY. There are also some fields in queues that needed modifying of their size.
Going through my diff’s these are the changes I made to correct this error.
###ABBrowse.clw
#####BrowseClass.ResetQueue PROCEDURE(BYTE RefreshMode)
HighlightedPosition from STRING(1024) to ANY
#####BrowseClass.UpdateViewRecord PROCEDURE
Pos from STRING(1024) to ANY
###ABFile.CLW
#####FileManager.UpdateServer PROCEDURE(BYTE HandleError)
HOLD from STRING(1024) to ANY
#####RelationManager.Delete PROCEDURE(BYTE Query)
Current:Position from STRING(1024) to ANY
###abvcrfrm.inc
VewPosition from String(1024) to STRING(10000)
###ABVCRFRM.CLW
#####FormVCRClass.CheckBorders
LastRecordPosition from STRING(1024) to ANY
#####FormVCRClass.Fetch
LastRecordPosition from STRING(1024) to ANY
###ABTblSyn.INC
DeletedRecordsQueue QUEUE,TYPE
RecordPosition STRING(10000)
###ABWindow.Inc
LastInsertedPosition from STRING(1024) to ANY
###ABBrowse.TPW
Change
INSERT(%MakeField, 'ViewPosition', '', 'STRING(1024)','Entry''s view position')
To
INSERT(%MakeField, 'ViewPosition', '', 'STRING(10000)','Entry''s view position')
###ABRELTRE.TPW
change occurs in two places
From
%[20]ValueConstruct STRING(1024) #<! Record POSITION in VIEW
To
%[20]ValueConstruct STRING(10000) #<! Record POSITION in VIEW
These are the changes I can find, put please post here if you find any more.
You could edit the template and put in something to log the value, at least while diagnosing it. Unless you can repeat the problem on demand, then step through with the debugger I suppose.
In Clarion 10 in ABBrowse.TPW ViewPosition size is generated from a call to %ConstructQueue 4th parameter which is omitted and defaults to 1024.
Change #INSERT(%ConstructQueue,'Queue declaration for browse/combo box using '&%ListControl)
To #INSERT(%ConstructQueue,'Queue declaration for browse/combo box using '&%ListControl,2048)
Changed all 1024’s to 2048 in ABBrowse.clw, ABFile.clw, ABWindow.inc. I’m not using VCR and RelTree.
Sorry, seems that 2048 was not enough, i should have used 10000.
In ABBrowse.tpw there is also this function which is called without parameters for a filedrop control (and in ABCFree).
Change: #GROUP(%ConstructQueue, %pDescription=’’, %pAddMark = %True, %pPositionSize = 1024)
To #GROUP(%ConstructQueue, %pDescription=’’, %pAddMark = %True, %pPositionSize = 10000)
In ABFile.clw there is another position variable:
SaveQueue QUEUE,TYPE,PRIVATE
Pos STRING(10000)
I use to fix this error by declaring the following variable in the offending Browse procedure:
ViewPosition STRING(2048)
and then adding a hot field to the browse extension:
.
When the procedure is compiled a warning is reported but it’s ok because the browse QUEUE has two ViewPosition fields and we want that the first one (the one we just added) is used:
That was one I had found but I note I had not documented it above.
I also found issues in 3rd party templates, I would have to go back and check which ones, but I remember Mike @BoxSoft uses 1024 in a product of his. Again, I’ll have to dig through my changes.
By all accounts, I don’t think SV are keen to change the size, I use 10000, but I do understand that might be overkill for some.
The problem is the TPS file specs allow keys upto 15,000 bytes in size, so the ABC templates needed to be made to work with the TPS file specifications as a minimum.
Once I’d established the clarion debugger was giving me duff info which was messing me around, that template hack seemed like the best workaround. Any other problems in the classes should get highlighted very quickly and I note there are other places in the ABC classes where local variables could be too small or the wrong type, I havent looked at your ANY data type changes yet in any great detail.
But this template hack to make Q ViewPosition the same size as the key length + 4 bytes output is a minimum to get the ABC classes to work with Position().
I had Mikes tagging templates along time ago, I cant say I remember it using Q ViewPosition, I know it had its own files which had to be added to the dict to store the any tagged browses, but I never saw any code where it used Q ViewPosition, but maybe I wasnt looking hard enough.
One other thing I did learn, is that it wasnt a Brit who wrote the StandardBehavior ABC class because the U isnt used, unlike here in the UK where U is used in the word Behaviour which draws its roots from France and the land owning Normans…
Like you have documented, there are restrictions in the class methods with local variables so I might need to still apply those changes you have documented but I havent come up with any tests which make it fail, unless you’ve got some tests which will make a class method fail?
I’ve been modifying the ABC classes in places to make it less obfuscated as well, it then gets easier to see WT_ is going on.