Import 100 text files

If I have, say, 100 text files with the same data format, is it possible to have FileMaker import these in a batch -- no user interaction? I don't see how to select more than one file at a time for an import.
Thanks

You could use e.g. Files.List in MBS Plug-in to get a list of files.
Then loop over them.
Build file path, do import of file.
A function like Container.ReadFile can help.

If you can generate the list of the files, and loop through them, putting them into a variable. The file import step will allow you to use the variable as the source.

1 Like

So "no", there is no automated batch import.

That's what I needed to know.

Thank you!! :slight_smile:

I have HUGE files.

To solve this, I just wrote a program to concatenate all of them into a single file and use that. Resultant text file is over 7 GB.

If you have huge files, pre-processing is highly recommended. Have the files in exactly the right shape for import, including meta data fields, so that you can switch off the "auto-enter" options on the import dialog.

Another thing to consider is that Filemaker would rather do many short bursts of activity than a sustained large activity. Plan for a sprint not a marathon. If you run server side scripts to control the import of many small files you will get better performance.

1 Like

Thank you @Malcolm :slight_smile:

You can put your textfiles into a designated folder and batch-import them from there and you can script that:

Very easy, native FileMaker :slight_smile:

6 Likes

So if I had 100 files like "File1", "File2", .... "File100" would each file go into its own table or could I specify that all 100 files are for the same table? Not clear from the dialog. Thanks.

Well you do need to do a little more work. I've included an example which is very rough, but it demonstrates what you can do
ImportFolderWithTextfiles.zip (148.4 KB)

2 Likes

That's cool thanks.

I already got it all working by concatenating all the files into a single file (>1 Billion text file lines!).

Yes I did read that, but that would not be automation (but you could write an applet that concatenates all files in a folder) :grin:

I didn't combine the text files manually in a text editor. LOL. :slight_smile:

I wrote code to do that combination, just not FileMaker code. Since this is a one-time import, it didn't need to be totally automated (as in recurring imports). There were several things the concatenation code needed to do, like quote the fields, remove blank lines, and filter out some other stuff.
The code was very fast. 30 files each having about 500,000 lines all combined into a single file in about 70 seconds.

There were eight files in all, so over 1 Billion lines concatenated, programmatically..

Thanks again.

Not in it´s own table, but every file is in a container field in a record of its own. I did not really undeerstand what your question means :slight_smile:

Oliver..
no.. each file in its own RECORD.. not TABLE.. it seems to me EXACTELY what you where searching for. Ok.. if you prefer all on one big bloob.. in one Record.. you preprocess and create one file, with applescript, python or whatever tool.. But your question " is it possible.. .100 files.. all text.. batch import".. is answered here.. Yes.. just have them in one folder and import that folder.

Nice implementation, I ve never needed to use the data files commands, finally I see a usability for them