Here topspeed_other.htm [Clarion Community Help]
Batch Processing Performance
When writing a large number of records, use STREAM() or open the file in a deny write mode, that is, OPEN(file) rather than SHARE(file). After the records have been written, call FLUSH() to allow other users access.
It is very important to use STREAM() when ADDing/APPENDing/PUTting a large number of records. STREAM() will typically make processing about 20 times faster. For example, adding 1000 records might take nearly 2 minutes without STREAM(), but only 5 seconds with STREAM.
It is not necessary to use STREAM() or FLUSH() on a logged out file (performance on logged out files is always good).
STREAM has the effect of LOCKing the file.
Speedy Logging and Automatic Recovery
TopSpeed transaction logging is very fast (about 100 times faster than the Clarion driver). With LOGOUT, the TopSpeed engine posts all transactions to memory. ROLLBACK simply frees the memory, while COMMIT writes out the database changes in a stream.
If a system crashes during a transaction (LOGOUT–COMMIT), the recovery is automatically handled by the TopSpeed driver the next time the affected file is accessed.
So it looks like Prop:Logout is using Stream(), but I do see performance improvements when specifically using Stream()/Flush() in code. So maybe the ABC Filemanager isnt changing the file access mode from Shared (2h) to Locked (12h). I’ll have to use SysInternals to have a look if Logout() or Stream()/Flush() is changing the access mode, but the way Stream() reads, it looks like it is.
Some file systems flush the operating system’s buffers on each disk write keeping the file “logically closed” (for example, the Clarion and TopSpeed file drivers do this by default). The STREAM statement disables this automatic flushing operation. A STREAM operation is terminated by closing the file, which automatically flushes the buffers, or by issuing a FLUSH statement.
It looks like Stream/Flush is using
with NTFS file streams.
As the RecordID Auto Increment in the ABC filemanager is performed every time a record is Added, this can also be an area which is slow in batch processing, so if its possible to use the MS SQL Auto Inc switches on the TPS RecordID field, then rolling your own code to do the AutoInc may also be a way to speed up the ABC Filemanager for TPS files. I’ve never tried this, but if it works it could speed things up a bit in this area.
The PROP:ServerAutoInc file property is used to specify which column of the table will receive the auto incremented query value.
Those options and properties are used at the driver level. To allow the application templates to use them, two user options were also added.
IsIdentity and ServerAutoIncColumn are used at the FILE user option level in the Dictionary.
For more information regarding any of these options, refer to the Server Side Auto incrementing for Clarion SQL file drivers help topic.
This is one area the ABC Filemanager could be improved, because using a TPS file with batch processing, as its going to be just one workstation adding/changing a lot of records, the AutoInc could definately be handled better.