Test if Close(Report) is SLOW on your System? – Mine 100x slower than C6

JaredDavis · September 22, 2021, 8:52pm

Wow. Thanks.

I’ve had anti virus scanners block file deletes.
create file -> do something -> delete file
The antivirus was scanning it and blocked the delete.

Have you looked at the remove() operation api calls to see what it is doing?

Bruce · September 23, 2021, 7:31am

there are a number of tools available which will convert a queue of WMF files into a PDF document. The most obvious is the ReportToPDF functionality built into Clarion (EE?) itself - but there’s also wPDF and the now-not-available TrackerPDF. There are others as well.

Cheers
Bruce

Bruce · September 23, 2021, 2:14pm

Thanks for the whole thread Carl - I’ve implemented a PremiereDeleteFilesInPrintPreviewQueue function in Premiere (build 3.36) . The template will automatically (by default, you can override at global or local level) make the call for you for ABC and Legacy reports. You can also call the function in hand-code if you like.

cheers
Bruce

Bostjan_Laba · September 23, 2021, 2:51pm

Now this I call FAST RESPONSE

Thanks Carl and Bruce

CarlBarnes · September 23, 2021, 8:29pm

I stepped it in the Debugger and confirmed REMOVE( String ) calls SHFileOperation . That function has a zillion features that make it slow. For a single file the function DeleteFileA() should be used as it was prior to C8 adding a second parameter to REMOVE(, Options).

JaredDavis · September 23, 2021, 8:49pm

Thanks. Do the deleted files end up in the Recycle bin with SHFileOperationA?

I can’t check right now.

Rick_UpperPark · September 23, 2021, 9:30pm

That’s an option of the API, but I don’t believe Clarion is calling it that way.

CarlBarnes · September 23, 2021, 9:38pm

I checked and Close(Report) does not put files in Recycle bin.

The API has flag FOF_ALLOWUNDO to recycle files but I don’t see that as an option flag for REMOVE():

REMOVE:FILESONLY         EQUATE(01h)   ! Remove normal files only if name has wildcards
REMOVE:RECURSIVE         EQUATE(02h)   ! Process nested directories recursively
REMOVE:PROGRESS          EQUATE(04h)   ! Show progress indicator
REMOVE:CONFIRM           EQUATE(08h)   ! Ask confirmations

The enhanced REMOVE deletes files using wildcards and there is no option to turn it off.

JaredDavis · September 23, 2021, 10:47pm

Thanks. I was fishing to see if this was the source of another issue I have where the exe pauses for a couple of seconds. It freezes long enough for a watchdog to trigger. Then it gets ugly.

Oggy · September 26, 2021, 8:35am

Tested on the M2 “speedy” disk… Differences is massive… testing

CarlBarnes · September 27, 2021, 9:46pm

Tested on an ordinary corporate office network machine and see it taking 37 seconds to close a 1000 pages report. 37 seconds is longgggg time such that the user may think it hung.

It was taking so long I stopped so have to estimate it would take 20 seconds for a 500 page report which is still a long time when waiting.

I have been running with this fix for 1+ years so these sites are not seeing these delays. This is an easy fix to make in Legacy and even easier in ABC. See post #7 above for details.

jslarve · September 27, 2021, 10:23pm

Have you considered all of the people who used to be able to take a potty or coffee break before you came along?

jkuijt · September 28, 2021, 1:53pm

Hi Carl,

I want to THANK YOU!!

I use the C10 Report Writer Engine DLL (ClaPRLB.dll) to print TXR Reports with a hook to my own previewer and I also noticed a huge delay after closing the Report Previewer window with a report of hundreds of pages. So before the Close (which takes place in EndReport()), I use your ReportCloseTempDeleteAPI, which is ultra fast. I am converting from C6.3 to C10 and in C6.3 everything was fast, so I wondered what was going on in C10. And then I read this article! Thanks again Carl!

IF SELF.PagesToPreview <> 0 THEN ReportCloseTempDeleteAPI(PrintPreviewQueue).  !Use API DeleteFile that is fast !!!
SELF.EndReport()                            !must close report before freeing queue
IF SELF.PagesToPreview <> 0 THEN
  FREE(PrintPreviewQueue)                   !free preview queue
END

ACochran · October 6, 2021, 7:06pm

Has anybody tried using the DeleteFile call from within a CPCS report? Probably I am doing something wrong. This is a Multi-DLL ABC project. I went thru the steps to and added into one of my CPCS reports but when it gets to the DeleteFile(cWmfName) line, I get the following message: “This program has performed an illegal operation and will now end”. Below is the what I get once I click the Detail button.

This program has performed an illegal operation and will now end.

Program : C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\UBSSql.exe
Version : 14.02
At : 14:03:36 on 2021/10/06
Workstation: : AlanSurfaceBk2
User Name: : Alan
Reported error : EXCEPTION_ACCESS_VIOLATION - Error reading data at : 6C14B108h
Windows : Win 10 , Tablet - 10.0.19043
Clarion : 0.9
Thread : 2 Field : 0 Event : 513 Keycode : 0
Error at address : 06534871h no debug info, Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\UBSRPTS.dll
Stack Trace
[01] 06409A96h no debug info, Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\UBSRPTS.dll
??? 019B7907h Line ?=19 no proc Src=iqxml017.clw Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\IQXML.dll 4.0.1.0
??? 1307E509h Line ?=246 no proc Src=NetEnc.Clw Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\CLAnet.dll 12.26.0.0
??? 01640000h Line ?=19 no proc Src=iqxml017.clw Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\IQXML.dll 4.0.1.0
[02] 010CF9E7h no line number no proc Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 01148D00h Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 01129CCEh Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
[03] 010CF4D1h no line number no proc Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 01129CE4h Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 01148D00h Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 01148D80h Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 0113D2BCh Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 0113D2D0h Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 0113D2ECh Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 0113D2E0h Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 0113D308h Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
??? 0113D2FCh Line ?=47 no proc Src=wxeh.cpp Module=C:\Users\Alan\OneDrive\Apps\C11\UBSSQL\ClaRUN.dll 11.0.13372
[04] 77B07A9Eh no debug info, Module=C:\WINDOWS\SYSTEM32\ntdll.dll 10.0.19041.1023 (WinBuild.160101.0800)
[05] 77B07A6Eh no debug info, Module=C:\WINDOWS\SYSTEM32\ntdll.dll 10.0.19041.1023 (WinBuild.160101.0800)

Thanks for any insight on this issue.
Alan

CarlBarnes · October 6, 2021, 7:11pm

Show more code around that DeleteFile. Show the Declaration of cWmfName, it should be a CSTRING. Show the prototype for DeleteFile(). It must have RAW and PASCAL.

ACochran · October 6, 2021, 7:20pm

Sorry about that. Like I said, I am likely doing something wrong!!!

In the Declarion Section ! Priority 1300
MAP
ReportCloseTempDeleteAPI PROCEDURE(QUEUE RptPreviewQ)

  MODULE('API')
    DeleteFile(*CSTRING lpFileName ),BOOL,PROC,RAW,PASCAL,DLL(1),NAME('DeleteFileA') 
  END
END

Before calling Close(Report)

ReportCloseTempDeleteAPI(PrintPreviewQueue)
Close(Report)

In the Local Procedure

ReportCloseTempDeleteAPI PROCEDURE(QUEUE RptPreviewQ)
QX LONG,AUTO
cName CSTRING(261),AUTO
CODE
LOOP QX=1 TO RECORDS(RptPreviewQ)
GET(RptPreviewQ,QX)
cName=CLIP(RptPreviewQ)
DeleteFile(cName)
END
RETURN
!------------------------------------------------------------------------------------

CarlBarnes · October 6, 2021, 8:14pm

Jeff posted but I think deleted this… The DeleteFile declaration must be in the Global Map i.e. Global embeds

ACochran · October 6, 2021, 8:20pm

Got it… Thanks… Will report back if I still am having issues.

e_mano_e · October 12, 2021, 8:18pm

Carl,

have you reported this to PTSS?

If the Clarion REMOVE() function is used without the second parameter SV could call API DeleteFileA() instead of the slower API SHFileOperationA().

Jens

CarlBarnes · December 22, 2021, 5:34pm

I created a GIST with my FileApiRemove() that can be used to replace calls to the RTL REMOVE(String) function. It calls the API DeleteFile(). It sets the RTL ErrorCode() and Error() so no other code changes are required. This is faster and safer.

  MODULE('ClaApi')
        !These Error functions have been in the RTL forever and appear in LibSrc but are not documented
        SetError(LONG ErrNumberToSet),NAME('Cla$SetError'),DLL(DLL_Mode)
        ClearError(),NAME('Cla$ClearError'),DLL(DLL_Mode) 
  END

FileApiRemove   FUNCTION(STRING Fn2Delete),LONG,PROC   !API Replacement 0=OK else GetLastError(). Can ignore sets RTL ErrorCode()

!===================================================================================================
! FileApiRemove() replacement for Clarion REMOVE() in that it sets the ErrorCode() and Error().
!                 This also returns the API GetLastError() or Zero for Success
!===================================================================================================  
FileApiRemove        FUNCTION (STRING Fn2Delete)!,LONG,PROC !0=ok, else returns Error code, also sets RTL ERRORCODE() ERROR()  
cFN        CSTRING(261),AUTO
ApiError   LONG     !GetLastError() when DeleteFile() fails
ClarionErr LONG     !Api Error converted to matching Clarion Error number
    CODE
    cFN=clip(Fn2Delete)
    IF 0=DeleteFile(cFn) THEN           !Return 0 = Failed to DeleteFile() see GetLastError() for why
       ApiError=GetlastError()
       IF ~ApiError THEN ApiError=1.    !Return says Failed, but no error ... so set something, should never happen
       ClarionErr = ApiError            !Most Clarion Error Numbers are the same as API Error Number
       CASE ApiError                    !Translate some unusual GLE to errors to Clarion 
       OF   32 ; ClarionErr = 52        !API ERROR_SHARING_VIOLATION 32 (0x20) The process cannot access the file because it is being used by another process.
                                        !  Clarion Help says Error 52 = File Already Open - An attempt to OPEN a file that has already been opened by this user.
       END
       SetError(ClarionErr)             !Set Error in RTL so caller can use ERRORCODE() just like if used RTL REMOVE()        
    ELSE                                !non-zero return from DeleteFile() means it worked and file was deleted
       ClearError()                     !Delete worked set Error to Zero (close) in RTL so caller can use ERRORCODE()
    END    
    RETURN ApiError                     !Return is Optional (PROC) so caller can check API Error Code if desired
!--------------------------------------------
!Possible enhancement, if fails could Sleep(100) and retry 5 times, file may have 
!just been closed and Virus scanner is looking at it, or OS is catchng up to commit delete
!
!SQLite has notes about exactly this problem. 
!   #define MX_DELETION_ATTEMPTS 5 
!   do{
!      DeleteFileA(zConverted);
!    }while(   (   ((rc = GetFileAttributesA(zConverted)) != INVALID_FILE_ATTRIBUTES)
!               || ((error = GetLastError()) == ERROR_ACCESS_DENIED))
!           && (++cnt < MX_DELETION_ATTEMPTS)
!           && (Sleep(100), 1) );
!---------------------------------------------

gist.github.com

https://gist.github.com/CarlTBarnes/e5d553a8ffe4287b0e19deac9c432202

RemoveFileWithAPI.clw

!There are a few Remove() things here, what you want is FileApiRemove FUNCTION(STRING Fn2Delete),LONG,PROC
!
!In C8 Clarion REMOVE() was enhanced a second parameter with flags to allow wildcards, recursion, etc
!It was changed from calling API DeleteFile() to use the much more complicated ShFileOperation()
!This makes it much slower and more bug prone
!
!My replacement FileApiRemove() uses API DeleteFile() then sets the Claion Error so it can replace 
!an existing REMOVE() in Clarion code. If you are using Remove(FILE) you'll need REMOVE(Name(File))
!or you can create a FileApi2Remove(FILE FileRef) that calls FileApiRemove(FileRef{PROP:Name}) or NAME()
!I named it "FileApiRemove(" so if you are searching "REMOVE(" you'll file it, versus RemoveUsingAPI() would not.

This file has been truncated. show original