Show or attach the source.
What does Logout/Commit have to do with using Directory() ?
1 Like
DeleteFile does not support wildcards in the parameter string. Clarion’s REMOVE - supports. If parameter of REMOVE is a name of directory, some extra checkings are required. In future Clarion REMOVE as well as some other file-oriented functions is also resolves mapped and SUBSTed drives, virtual folders, directory injections, etc. because this is related to access rights.
Obviously
LOGOUT(1, dirName)
DIRECTORY(dirQ, '*.*', ff_:DIRECTORY)
COMMIT
sorry, but I don’t understand the use of LOGOUT/COMMIT in that case.
Can you explaint it. thanks
1 Like
This is a topic starter’s lifehack.
Using logout/Commit to dump the Directory file queue into a tps file, there are lots of ways I’ve been trying this, like on a 100 second logout, and then if the commit code is encountered within 95 to 100 seconds, it automatically commits and things like that.
ie
Directory(File:Queue,Path,etc)
Counter = 0
Logout(100,TPSFile)
Loop Records(File:Queue) times
Counter +=1
Get(File:Queue,Counter)
TPSFile.Name = File:Queue.Name
Access:TPSFile.Insert
Commit
One thing I think is an overriding factor, is on XP theres a few thousand directories, on Win10 I have 160,000 subfolders, so I’m trying the
FindFirstFileA function (fileapi.h) - Win32 apps | Microsoft Learn
FindNextFileA function (fileapi.h) - Win32 apps | Microsoft Learn
api’s to see which is quicker, I dont know if the transacted api might be faster than the above or not, but I need to log all the files on various media, and hash their contents to spot when any changes occur.
I basically need to log every change that occurs on my PC’s to spot when stuff is being tampered with, like templates changed back to earlier versions and embed clw code changed, which is a form of resource burning me, a security services/military technique but also a criminal one. This is one of the reasons why I have been writing so many of my own templates, because a change in clw code can be explained as human error, templates that work in multiple apps, are harder to explain as human error where the template is used in multiple apps, and the generated clw code is different, highlighting the template may have been tampered with or the clw tampered with between the small window of opportunity that exists between generation and compilation.
A classic example of tampering with code which can alter the behaviour is this post
so if everything is being logged and hashed like using techniques similar to that seen in cryptocurrency to authenticate & log changes, it gets harder to insert backdoors as bugs. I know subversion is supposed to partly address this problem but its got its problems.
With Directory, I have to run through the directory file queue and strip out the old dos directory single and double dot and also the files because despite only using the attrib flag for directory, its still pulling in files, and there is a chance, havent counted that the avg number of files per folder have also gone up in Win10 compared to XP.
Hi Richard
first thing to determine is whether the time difference is due to the directory() command (which is what I thought from the original post) or if it is due to writing out to the tps file.
so time each part separately and compare
I will know shortly if its the difference between api’s. I suspect the Clarion Directory() is using FindFirstFile and not FindFirstFileExA
Namely
FindFirstFileExA function (fileapi.h) - Win32 apps | Microsoft Learn
has these options
FINDEX_INFO_LEVELS (minwinbase.h) - Win32 apps | Microsoft Learn
This one caught my eye.
FindExInfoBasic
The FindFirstFileEx function does not query the short file name, improving overall enumeration speed. The data is returned in a
WIN32_FIND_DATA structure, and the cAlternateFileName
member is always a NULL string.Windows Server 2008, Windows Vista, Windows Server 2003 and Windows XP: This value is not supported until Windows Server 2008 R2 and Windows 7.
You see even though the clarion Directory() function help doc suggests I should be able to use the ff_:Directory equate to filter the results, that doesnt work. I have to specify a wildcard at the end of the path and include ff_:Directory to get it to include the subfolders. I then have to loop through the queue and ignore all entries ie files which are not subfolders.
Anyway just finishing off a class to use the various apis related to FindFirstFileA and FindFirstFileExA etc as I can get more info from these api’s which will be useful.
RichClaCode,
Maybe accessing the files on NTFS drives via the MFT data structures is what you need here?
From your other posts, I’m assuming you might be able to utilize the following technical article:
Super Fast File Enumeration with NTFS Structures
One of my hard disks is of Ext4 file system. Windows accesses it using the Paragon Linux FS driver. I have 2 letters to mapped drives on the home NAS. How suggested way can help with these drives? Incomplete solution = no solution.
I’ve also been looking at FindFirstFileTransactedA function (winbase.h) - Win32 apps | Microsoft Learn to see what the speed is like. Still testing, but I’ll check out your link thanks it looks interesting.
I had that Paragon Linux Ext Driver, I’m surprised you got it to work on windows, I got a refund because it didnt work properly on my machine, it was easier to just setup a Linux NAS with a variety of file systems and plonk Samba - opening windows to a wider world on the linux box so files could be accessed via SMB.
I understand what you are saying though, if you cant access a drive, how can you audit it. Market share is what I’m going after for now.
According to the linked article the process needs to be “Elevated”. That’s ok for something I run on my PC but not for end users.
To access the MFT structures, you must have elevated privileges – a normal user won’t be able to access it
The help clearly states that is NOT how it works… It ALWAYS has Files…
If you add ff_:DIRECTORY to ff_:NORMAL, you will get files AND sub-directories from the path . Since ff_:NORMAL is an equate for zero (0), you will always get files.
https://clarion.help/doku.php?id=directory_get_file_directory_.htm
The DIRECTORY procedure returns a directory listing of all files in the path with the specified attributes into the specified queue .
The help doc clearly states it returns a directory listing of all files in the path with the specified attributes into the specified queue.
But its not working like the help docs clearly states.
If you copy an app using api’s which need to be elevated onto a windows OS which uses elevation, the app’s application icon is automatically modified to include the elevation shield. This is without having to run the app, which suggests windows is analyzing all files copied onto a running OS.
Makes a mockery of the ransomware sideshow doesnt it!
Edit.
Its interesting the very first folder created during installation of windows on xp is
c:\windows
then
c:\windows\system32
c:\windows\system32\config
c:\windows\system32\ras
c:\windows\system32\spool
c:\windows\system32\spool\drivers
and so on. It gives an insight it how windows installs itself and what its installing first and why.
Or it could be the filename. Certain magic words will get you the shield, such as “Update”, “Setup”, and others.
Its a good point so I renamed the app from PCSetup to PCtest, copied from my XP virtual machine onto Win10 and it still added the shield. Maybe its looking at the embedded manifest file or something. Dont know, but using the API FindFirstFileA has got it down to 3mins something which is better.
C:\Users appears to be the first folder created on Win10, and there are some interesting attributes attached to folders, even in XP.
If you have an embedded manifest, and it says requireAdministrator, then yes. Windows is looking at that. You should be able to confirm by looking at the resources of the exe with a resource viewer. If a manifest is present, there should be a folder called “Manifest”.