# Most Efficient File Handling to Empty and Import 2 files **URL:** \ **Category:** questions\ **Tags:** csv, tps\ **Created:** [July 30, 2023, 2:38am UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446 "2023-07-30T02:38:32Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![NeutronBoy](https://clarionhub.com/user_avatar/clarionhub.com/neutronboy/32/4355_2.png) [@NeutronBoy](https://clarionhub.com/u/NeutronBoy)\ **Post date:** [July 30, 2023, 2:38am UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/1 "2023-07-30T02:38:33Z")
I want to update a member data file from a .CSV and rather than reading in and comparing records in 2 .TPS files, I will backup the original files by renaming them to filename.BAK. What's more efficient? 1. copying the 2 original files to the newly created .BAK files and then doing an EMPTY on the originals, OR 2. renaming the 2 originals and opening newly created (empty) original files and then refiling the 2 original files with the data from the CSV file Hope that makes sense. Mark ---
**Author:** ![seanh](https://clarionhub.com/letter_avatar_proxy/v4/letter/s/4af34b/32.png) [@seanh](https://clarionhub.com/u/seanh)\ **Post date:** [July 30, 2023, 4:12am UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/2 "2023-07-30T04:12:28Z")
1. I would think. Note it does require exclusive access. Also make sure you use logout/commit or Stream/flush when writing. The difference that makes to TPS is incredible. ---
**Author:** ![vitesse](https://clarionhub.com/user_avatar/clarionhub.com/vitesse/32/837_2.png) [@vitesse](https://clarionhub.com/u/vitesse)\ **Post date:** [July 30, 2023, 8:06am UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/3 "2023-07-30T08:06:02Z")
Yes I agree with what Sean said Also if you have lots of records and the tps files have keys, it may be faster to APPEND then do a BUILD at the end rather than ADDing records which updates the keys after each record is added to the file. See the help for details. BTW were you using the Basic driver for reading the CSV or using StringTheory instead? ---
**Author:** ![Bostjan\_Laba](https://clarionhub.com/user_avatar/clarionhub.com/bostjan_laba/32/2498_2.png) [@Bostjan\_Laba](https://clarionhub.com/u/Bostjan_Laba)\ **Post date:** [July 31, 2023, 8:45am UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/4 "2023-07-31T08:45:05Z")
Or perhaps 3. No copying, no emptying, just overwriting existing records? True, you’d need to take care of old records not being overwritten due to shorter csv compared to previous one. I don’t know if this would be the fastest or most efficient though, it’s just an idea… ---
**Author:** ![RichCPT](https://clarionhub.com/letter_avatar_proxy/v4/letter/r/e19b73/32.png) [@RichCPT](https://clarionhub.com/u/RichCPT)\ **Post date:** [July 31, 2023, 3:13pm UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/5 "2023-07-31T15:13:37Z")
Also worth noting: If these are large data files you might get a noticeable speed improvement in your reports if the tables are built in the key order that your reports use, assuming there is one particular order used on the reports. Of course, if you are using SSD drives then it probably does not matter. I just know when we have re-created a child table of several hundred thousand records in the order used by reports that customers with traditional hard drives have noticed a big speed improvement for subsequent reports. P.S. We’ve been converting customers from “on-premise” to hosted (cloud) solutions under RDP. Reports that took 40 minutes to run on their LAN take about a minute to run on an Azure server using .TPS. ---
**Author:** ![CarlBarnes](https://clarionhub.com/user_avatar/clarionhub.com/carlbarnes/32/1859_2.png) [@CarlBarnes](https://clarionhub.com/u/CarlBarnes)\ **Post date:** [July 31, 2023, 4:25pm UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/6 "2023-07-31T16:25:30Z")
> [@NeutronBoy](#): > > Renaming the 2 originals and opening newly created (empty) original files I would go with this as fastest and least disk work. Since 2 files are involved I would want to insure they BOTH got done, or neither. AFAIK there is no way to put the Rename in a Transaction. I would use 2 DOS files to Open Both these files Exclusively e.g. `OPEN(file, DenyALL + ReadOnly)`. If that fails then I can show a Message(‘Users are active, you cannot update’) and abort. I use a DOS file as a very simple driver to Lock the file with Deny All. The TPS driver is too complex with too many features, e.g. multi-thread exclusive access. If it works then I can Close the DOS file locks and do the Rename()s. There is a small window of time for a user to open the files and cause failure. If you want to be bullet proof you have to deal with it. If the 1st file Rename() fails that’s easy. I can show a Message and Abort. If the 2nd file Rename() fails that’s messy. Must rename File 1 back then show a Message and Abort. The user’s actions may have created an empty File 1 that will prevent rename. ---
**Author:** ![NeutronBoy](https://clarionhub.com/user_avatar/clarionhub.com/neutronboy/32/4355_2.png) [@NeutronBoy](https://clarionhub.com/u/NeutronBoy)\ **Post date:** [August 1, 2023, 1:53pm UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/7 "2023-08-01T13:53:46Z")
Thanks all!! Carl, I’m not sure what you mean when you say you use a DOS file. I assume this is the same as the CSV file that I am importing info from - CW uses a BASIC file driver for that file. I should have mentioned that these files contain a very small number of records - right now it is at about 130 records (with 10 simple fields) and could grow to about 500 or so after a few years, but that is still small so far as CW is concerned. ---
**Author:** ![CarlBarnes](https://clarionhub.com/user_avatar/clarionhub.com/carlbarnes/32/1859_2.png) [@CarlBarnes](https://clarionhub.com/u/CarlBarnes)\ **Post date:** [August 1, 2023, 3:10pm UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/8 "2023-08-01T15:10:51Z")
> [@NeutronBoy](#): > > I’m not sure what you mean when you say you use a DOS file. I meant to define a FILE using the Clarion DOS File Driver. Use that DOS file to `OPEN(,DenyAll+ReadOnly)` your TPS file by setting the NAME() variable. You will have to have the TPS file closed. ```auto OpenThisFile FILE,DRIVER('DOS'),NAME(OpenThisFileName),PRE(OpenA) Record RECORD,PRE() Block STRING(1024) !OpenA:Block END END ... CLOSE(OpenThisFile) OpenThisFileName = NAME(TpsFileofYours) CLOSE(TpsFileofYours) OPEN(OpenThisFile,ReadOnly + DenyALL) IF ERRORCODE() THEN Message('Cannot import the File XYZ is in use.','import') ... ``` * * * I wrote this little utility that allows opening any File using any Modes so I could test my code error handling of locked files. This uses a DOS file to do the Open. Look at this source. > **[GitHub - CarlTBarnes/OpenFileForTest: Open a File for Testing Access and Share Modes](https://github.com/CarlTBarnes/OpenFileForTest)** > > Open a File for Testing Access and Share Modes ![image](https://clarionhub-uploads.s3.eu-north-1.amazonaws.com/original/2X/3/332653c333f8baaa92c8cde4cc31da60804beac4.png) ---
**Author:** ![NeutronBoy](https://clarionhub.com/user_avatar/clarionhub.com/neutronboy/32/4355_2.png) [@NeutronBoy](https://clarionhub.com/u/NeutronBoy)\ **Post date:** [August 1, 2023, 4:02pm UTC](https://clarionhub.com/t/most-efficient-file-handling-to-empty-and-import-2-files/6446/9 "2023-08-01T16:02:02Z")
Thanks again Carl! I will have to digest this all.