Using variables inside `VIEW` `JOIN` expression causes weird results

Quick question - is the use of BIND variables supported in the JOIN expression?
Because for some reason, when I do, I only get a single record back while I now it should return multiple - if I invert the order I get everything except one back - if I move the variable based condition to the FILTER I goes through all the expected records, but I am not sure if this is the correct result fully as the filtering is not done on the JOIN side.

To explain, I have two datafiles, one contains all my invoices and has an invoice number, the second is a special datafile which contains references to an external platform, but as I do not want to create a datafile per entity in my application, I create a polymorphic table, which looks as follows:

ExternalReferences FILE,DRIVER('TOPSPEED'),NAME('data\references.dat'),PRE(Ref),CREATE
NummerKey         KEY(Ref:Nummer),PRIMARY,NOCASE
AccountIdPlatformReferenceReferenceTypeKey KEY(Ref:AccountId,Ref:PlatformReference,Ref:ReferenceType),DUP,NOCASE
AccountIdReferenceTypeKey KEY(Ref:AccountId,Ref:ReferenceType),DUP,NOCASE
AccountIdReferenceTypeEnValueKey KEY(Ref:AccountId,Ref:ReferenceType,Ref:ReferenceValue),DUP,NOCASE
AccountIdKey      KEY(Ref:AccountId),DUP,NOCASE
ParentKey KEY(Ref:Parent),DUP,NOCASE
record            RECORD
Nummer              LONG
Parent LONG
AccountId           CSTRING(39)
PlatformReference   STRING(128)
ReferenceType      LONG
ReferenceValue    LONG
                  END
                END

My View is as follows (at least the original version):

OpenInvoices VIEW(Invoices),ORDER('Inv:Data, Inv:Time'),FILTER('Inv:Paid = paidState')
    JOIN(ExternalReferences, 'Ref:AccountId = accountId AND Ref:ReferenceType = referenceType AND Ref:ReferentieWaarde = Inv:Number'),INNER
    END
END

But this results in the above behavior, below is the looping code:

accountId = '{SOME_ID}' ! Passed through as parameter from the procedure this is in
referenceType = ReferenceTypes:Invoice ! ITEMIZE structure
paidState = 'N' ! Constant value

BIND('accountId', accountId);
BIND('referenceType', referenceType);
BIND('paidState', paidState);
BIND('Inv:Paid', Inv:Betaald);
BIND('Inv:Number', Inv:Bezoeknummer);
BIND('Inv:Date', Inv:Datum);
BIND('Inv:Time', Inv:Tijd);
BIND('Ref:AccountId', Ref:AccountId);
BIND('Ref:ReferenceType', Ref:ReferenceType);
BIND('Ref:ReferenceValue', Ref:ReferenceValue);

OPEN(OpenInvoices);
ASSERT(ERRORCODE() = 0, 'Could not open view: ' & ERROR());

SET(OpenInvoices);
ASSERT(ERRORCODE() = 0, 'Could not set view: ' & ERROR());

LOOP
    CLEAR(Invoices);
    CLEAR(ExternalReferences);
    NEXT(OpenInvoices);
    ASSERT(ERRORCODE() = 0 OR ERRORCODE() = 33, 'Something went wrong while looping through the open invoices: ' & ERROR());
    IF ERRORCODE() <> 0
        BREAK;
    END
    
    ASSERT( |
        Ref:AccountId = accountId AND Ref:ReferenceType = referenceType AND Ref:ReferentieWaarde = Inv:Number, |
        'Found a reference which doesnt the join: <13,10>' & ConcatenateFields(Ref:Record, '<13,10>', TRUE, ': ') |
    );
    
    ! DO SOMETHING
END

CLOSE(OpenInvoices);

UNBIND('accountId');
UNBIND('referenceType');
UNBIND('paidState');
UNBIND('Inv:Paid');
UNBIND('Inv:Number');
UNBIND('Inv:Date');
UNBIND('Inv:Time');
UNBIND('Ref:AccountId');
UNBIND('Ref:ReferenceType');
UNBIND('Ref:ReferenceValue');

If I change the view to the following I get all the expected records, but it feels weird and unnatural:

OpenInvoices VIEW(Invoices),ORDER('Inv:Data, Inv:Time'),FILTER('Inv:Paid = paidState AND Ref:AccountId = accountId AND Ref:ReferenceType = referenceType')
    JOIN(ExternalReferences, 'Ref:ReferentieWaarde = Inv:Number'),INNER
    END
END

A tip … You can save the trouble of all the matching UNBIND’s by using POPBIND() as long as you PUSHBIND() first.

PUSHBIND()                !<-- add Push
BIND('accountId', accountId);
BIND('referenceType', referenceType);
BIND('paidState', paidState);

... many lines ...

POPBIND()   !was UNBIND('accountId'); UNBIND('referenceType'); ...
RETURN

When do accountId and referenceType get updated? Is it intended that they should be constants?

accountId is a parameter to the procedure this is in - so it’s constant for the duration of the view being opened - referenceType is constant always - as this is set according to a EQUATE in an ITEMIZE (which contain all types available) - so yeah, they are intended to be constant

Use Prop:JoinExpression to assign the expression at runtime as a constant string. This will eliminate the need for the bound local variable.

OpenInvoices{Prop:JoinExpression,1} = 'INV:Number = ' & Ref:ReferentieWaarde
! or if INV:Number is a string
OpenInvoices{Prop:JoinExpression,1} = 'INV:Number = <39>' & Ref:ReferentieWaarde & '<39>'

Same with the filter, assigned it dynamically building the expression.
Do this after the view is opened.

Thanks for this - it partially helps but resulted into the same issue - but, I figured out the issue itself, but let me ask a question - is there a difference between below two snippets in regards to functionality?

OpenInvoices{PROP:Filter} = 'EsABzn:Betaald = <39>N<39>';
OpenInvoices{PROP:JoinExpression, 1} = 'BoeRef:AccountId = <39>' & CLIP(account.Id) & '<39> AND BoeRef:ReferentieType = ' & EsA:ReferentieTypes:Factuur & ' AND BoeRef:ReferentieWaarde = EsABzn:Bezoeknummer';
OpenInvoices{PROP:Filter} = 'EsABzn:Betaald = <39>N<39>';
OpenInvoices{PROP:JoinExpression, 1} = 'BoeRef:ReferentieWaarde = EsABzn:Bezoeknummer AND BoeRef:AccountId = <39>' & CLIP(account.Id) & '<39> AND BoeRef:ReferentieType = ' & EsA:ReferentieTypes:Factuur;

Apparently there is - at least for Clarion - the first snippet gives above results - but the second one works - only difference, the ordering of the conditions in the AND statement, it seems one needs to start with the FIELD-to-FIELD conditions and end with the constant ones…

I don’t see a significant difference between your join strings.
I’m certain that the order of the clauses should not matter and I’m pretty confident in SQL it doesn’t matter. I guess it’s possible that the TOPSPEED driver might have a flaw where the order matters in the decision making on the join.
Have you turned on dbtracing to see what the trace log says? See “Logging Driver Activity” in the help.
Also, put an IF ERRORCODE() test after you assign the Prop:JoinExpression to see if an error is thrown for the variation that doesn’t work.

@ThaDaSoft You marked your own post as solution.

Just curious, what you figured out to be the issue?

I’d rather write something like this.

OpenInvoices VIEW(Invoices),ORDER('Inv:Data, Inv:Time')|
                           ,FILTER('Inv:Paid = paidState'              &|
                              ' AND Ref:AccountId = accountId'         &|
                              ' AND Ref:ReferenceType = referenceType' &|
                          '')
    JOIN(ExternalReferences,'Ref:ReferentieWaarde = Inv:Number',INNER
    END
END

Does this make sense and difference (for your files and TopSpeed VIEW driver)?

1 Like

I want the actual filtering to be done inside the join (except the PaidState) to prevent getting duplicate rows because the InvoiceNumber just happens to match - also coming from a SQL background it feels more logical to have the filter in the JOIN instead of on the FILTER of the parent - I already solved it (but only for TOPSPEED, IP-Driver doesn’t like the fix…)

This is where your confusion is coming from. For the TOPSPEED driver it feels more logical to have filter in FILTER rather than in JOIN expression. And this is a real solution for the topic.

1 Like

So each driver has it’s own way you need to write such stuff in - feels a bit weird - because I read the SQL driver prefers it in the JOIN - but the IP Driver also behaves like the TOPSPEED driver and prefers it in the FILTER

It is obvious, the filter expression supposed to be in FILTER while JOIN designed to map table relations. This is TRUE for all the Clarion database drivers.

The difference is that SQL drivers are just translating VIEW definition to corresponding SQL SELECT query, and all the grunt work performed on the server side. Any SQL server engine typically supports rather complex JOIN expressions, unlike lightweight TOPSPEED driver performing locally.