StringTheory replaceBetween method problem after I upgraded from 3.54 to 3.80

Turns out the problem is not directly related to StringTheory although something must have changed there to trigger this behaviour. Possibly something related to Unicode support in ST. @vitesse , you mention changes in ReplaceSlice() method so possibly some low level character manipulation in stMemCpyLeft Procedure.

The problem is with some invisible characters stored in TPS table and passed to ST object. Clarion does not support UTF-8, although pretty much everything supports unicode and defaults to it these days. Clarion is an exceptional :wink: toool so it doesn’t. Not sure what happens when you copy UTF-8 encoded text into Clarion entry control, but all national characters with acute accents are correctly mapped to ANSI code page. Although it may be handled by Windows as I noticed I had to change “Language for non-unicode programs” setting in order to display these characters correctly.

My template text file was created using notepad by customer. When I open it in Notepad++ it recognises it as UTP-8 encoded. The output file created by Clarion is recognised as ANSI
Some data in the table (gearbox codes, engine codes, colour codes), is being copied and pasted by end users from various websites and I came across a problem before when exporting data from TPS file to MySQL table using ODBC driver. With some records I was getting “Error: Incorrect string value: ‘\x81 …’ for column”. To solve this issue I created a small function parsing the string and replacing this CHR(129) with CHR(32) - space. Later on I added few other characters to be removed. I was posting about it in this thread MySQL Record Insert Error,: Incorrect string value: '\x81

So before calling SetValue method I clean the string with

    ASC:Tekst = SrcCleanIllegalChars(ASC:Tekst)
    ThisLabelLine.SetValue(ASC:Tekst)

When the string is cleaned from these illegal characters replaceBetween method works as expected.

Here’s the code of my cleaning procedure

SrcCleanIllegalChars     PROCEDURE  (Tekst)                   

Count LONG
  CODE
        Count = 1
        LOOP LEN(CLIP(Tekst)) TIMES     
            IF INRANGE(VAL(Tekst[Count]),0,31) OR INLIST(VAL(Tekst[Count]),'127','129','131','136','144')
                Tekst[Count] = CHR(32)
            END
            Count += 1
        END        
        Tekst = CLIP(LEFT(Tekst))
        RETURN(Tekst)

So basicaly if one of the characters above is passed to ST object the replaceBetween method produces this result. I noticed that usually “illegal” character is at the beginning of the string, as users select and copy text from the websites and I was able to fix the problem manually by editing a faulty record and removing the first invisible character. Not sure if that matters.
Interestingly, when I tried to replace placeholders with values from my table using Clarion string slicing the output was correct, so probably Clarion is ignoring these characters, so they become a problem only if passed to something external like MySQL ODBC driver etc.

@Bruce
You may want to look how your ST code treats these characters above as this may be an issue for other users of ST and also SQL Driver Kit which may throw an error if end user pastes some text with these characters from the web. Most likely it would be CHR(129). If you were not able to reproduce the problem, note that the “Language for non-unicode programs” setting may also play a role there. My setting is “Polish”