FileMaker Performance Variations - platforms, networking and Apple Silicon Macs

Heads-Up!
1

This has stemmed from our new SaleFaith manufacturing/MRP system due to go live in the first week of January. As part of this we’re migrating the data from an ECI M1 ERP system. The process initially involves some manual data extraction from the Microsoft SQL database using SSMS, import this into a FileMaker ‘Transfer’ file, then import the bulk of the data using an ODBC connection and finally run some data cleaning and a small amount of data restructuring.

After all the data has been imported into the transfer file, we run a master script calling many subscripts in SaleFaith MRP to import from the transfer file and run many routines to reconstruct the data for our new system.

Initially we hosted all files within FileMaker Server 19.4.1 on the client’s LAN based Windows Server (hypervisor virtual machine). FileMaker Pro 19.4.1 was then run locally on this server, connected to the hosted files to run the scripts. The uninterrupted run time for this was consistently between 7 and 8 hours.

I needed to test a run and, as I had my previous 13” M1 MacBook available, ran the same migration locally, no server involved and, lo and behold it completed in just over 40 minutes!

At this point, I couldn’t resist the opportunity to carry out some performance tests, having 3 Apple Silicon MacBooks, our cloud servers, the client’s Windows Server, a 2019 MacBook Air and a 2013 15” MacBook Pro. I couldn’t guarantee that the previous tests were like for like, as this is on-going development. So I created a baseline data set and ran it on these computers. Here are the results:

Platform FileMaker Configuration Duration (hh:mm:ss)
Windows Server 2016 16Gb, 8-Core Xeon Silver 4110 2.1Ghz FileMaker Pro run locally using non-hosted files with FileMaker Server service stopped 02:06:01
Windows Server 2016 16Gb, 8-Core Xeon Silver 4110 2.1Ghz FileMaker Pro and FileMaker Server both running on this server with FileMaker Pro connected to the hosted files 07:12:20
MacBook Pro M1 13 inch 16Gb FileMaker Pro run locally on local (non-hosted) files 00:51:57
MacBook Pro M1 13 inch 16Gb FileMaker Pro run locally on local (non-hosted) files (initial test, double checked due to speed of completion) 00:42:00
MacBook Pro M1 14 inch 16Gb FileMaker Pro run locally on local (non-hosted) files 02:18:01
MacBook Pro M1 14 inch 16Gb FileMaker Pro run locally on local (non-hosted) files (repeat test) 02:03:19
MacBook Pro M1 14 inch 16Gb FileMaker Pro run locally on local (non-hosted) files (initial test, double checked by the above) 00:41:29
MacBook Pro M1 16 inch 16Gb FileMaker Pro run locally on local (non-hosted) files 02:20:34
MacBook Pro M1 16 inch 16Gb FileMaker Pro run locally on local (non-hosted) files (repeat test) 02:20:34
MacBook Air 2019 16Gb 1.6Ghz Dual-Core i5 FileMaker Pro run locally on local (non-hosted) files 05:48:12
MacBook Pro 15 inch 2013 8Gb FileMaker Pro run locally on local (non-hosted) files 10:20:48
MacBook Pro M1 16 inch 16Gb FileMaker Pro run locally on local (non-hosted) files run in Windows 11 for Arm within Parallels VM 04:45:41
MacBook Pro M1 13 inch 16Gb FileMaker Pro run as streamed workspace in Microsoft Remote Desktop connected to our most powerful cloud-hosted Windows FileMaker Server 06:00:06
MacBook Pro M1 13 inch 16Gb FileMaker Pro run locally connected to our most powerful cloud-hosted Windows FileMaker Server 111:04:35 (over 4.5 days)

Up until our very last test, we’d assumed that our initial sub 1-hour completion on the M1 MacBook Pros was down to additional routines having been added and these were not like-for-like comparison. However, having ensured the files were identical within all tests, I realised we didn’t have the MacBook Pro 13” results, so ran that today and it completed in the 42 minutes shown above.

This has added some confusion and is going to require some more retests. However there are some pretty safe conclusions from the above:

Macs have got quicker over the years and the M1 MacBooks are unbelievably fast (unfortunately, we do not have a late Intel based MacBook Pro to test)

There isn't a lot of difference between the original M1 and the newer M1 Pro SoCs for this type of work. The M1 MacBook Airs and 13 inch MacBook Pro are bargains.

For high volumes of data management that includes imports, replace field contents, and many looping scripts through data, always do this locally and not on hosted files

FileMaker Pro 19.4.1 running within Windows 11 for Arm on a Parallels virtual machine on an M1 MacBook Pro is quicker than a 2019 MacBook Air dual-core i5

Our cloud-based centralised delivery of FileMaker Pro, connected to our cloud-based FileMaker Servers, is quicker than FileMaker Pro connecting to FileMaker Server running on the same Windows Server. We also appreciate that this is not an optimal setup for FileMaker and hope to run the same tests on a PC on the LAN connecting to the server at some point in the future

Be very, very careful how you design your databases if you wish to use any cloud-based FileMaker Server whilst running FileMaker Pro locally on your computers. Don’t expect to be able to use all of FileMaker’s features. Otherwise, if you need to do something like we’ve described above, start the process Sunday and then book the rest of the week off as holiday (we started the 111 hour test at 16:40 last Sunday and it finished just before 08:00 this Friday morning)

We work from a village in the middle of a forest in Norfolk, England. We only have fibre to the cabinet at the moment and have an optimised business broadband service offering roughly 12Mbs upstream, 50Mbs downstream. However, this will be much better than many people’s home connections that have much higher contention rates.

I’ll try to carry out some retests due to the M1 variations, we have to try to gain some continuity, but currently have no explanation for this.

I've attached screenshots of each test.

Kindest regards

Andy
Speed Test summary 101221.pdf (3.1 MB)

5 Likes
2

@AndyHibbs, thanks for sharing the results of your extensive test campaign with us!
Two things I can confirm:

  1. cloud connections are a true bottle neck
  2. 13“ M1 MacBook Pro is an incredibly performant machine
3

I believe we may have got to the bottom of the performance differences and it is quite exciting for the future. Repeat tests were carried out and the pattern continued with the 13” M1 MacBook Pro out performing the 14” and 16” M1 Pro MacBooks. A couple of additional Windows tests were carried out as well.

Lots of head scratching, then I realised that the 13” M1 had been erased and setup again, whereas the M1 Pro machines were our working machines. There was an identifiable difference - Rosetta 2!

The Intel processes running on the M1 Pros are included on the attached PDF, which were mostly menu items such as Dropbox. All Intel processes except for CarbonComponentScannerXPC, which was running on the 13" M1, were quit and the test rerun. This time the 14” M1 Pro was within seconds of the 13” M1, more than halving the processing time to under an hour.

We’re quite stunned that Rosetta 2 had such an impact on our shiny new M1 Pro MacBooks and do need to repeat the tests a few more times to fully qualify these results, but all indications are that this is the case.

For the record, the last set of results are both listed below and the screenshots are within the attached PDF.

MacBook Pro M1 13-inch 16Gb FileMaker Pro run locally using non-hosted files 00:51:56
MacBook Pro M1 13-inch 16Gb - repeat test FileMaker Pro run locally using non-hosted files 00:49:59
MacBook Pro M1 Pro 14-inch 16Gb - with background Intel processes running FileMaker Pro run locally using non-hosted files 02:25:57
MacBook Pro M1 Pro 14-inch 16Gb - without background Intel processes FileMaker Pro run locally using non-hosted files 00:50:34
Windows Server 2016 16Gb 8-Core Xeon Silver 4110 2.1Ghz FileMaker Pro run locally using non-hosted files 02:07:29
Windows Server 2019 19Gb 2.7Ghz VM - background Windows processes were running FileMaker Pro run locally using non-hosted files 03:14:24

Additional conclusions to those originally posted:

Apple Silicon laptops smoke Intel Windows servers when running FileMaker Pro 19.4.1 (apologies, but we have no high end Windows 10/11 PCs here, Parallels and online virtual machines fulfil our needs)

We need to dump Rosetta 2 as quickly as possible

The M1 MacBook Pro 13-inch and MacBook Air continue to match the M1 Pro computers for typical FileMaker Pro data processing

We’re now looking forward to repeating the ‘non-Intel’ status running Parallels, Windows 11 for Arm and FileMaker Pro 19.4.1 for windows

Kind regards

Andy
Speed Summary 111221.pdf (2.3 MB)

2 Likes
4

@AndyHibbs - is it possible to do some more comparisons?

  • FM Pro and FMServer running on same machine (FMPro connecting via localhost / 127.0.0.1 )
    vs.
  • FMServer only, scripts running under Perform Script on Server (PSOS)
    vs.
  • FM Pro run locally, no server

My experience has been that PSOS can be quite fast, although I don't think it generally beats running the script under FMPro with a purely local database...

5

Hi @xochi

We’ve taken the opportunity to do these tests due to the nature of the data take on as we approach the end of a 2-year project. We’ve got the direct comparison in the first set of tests using Windows, to compare connecting to the FileMaker Server via localhost/127.0.01 vs running FileMaker Pro on local, non-hosted files. Connecting to the localhost server needed between 7 and 8-hours to complete the processing, whereas running locally sped this up to just over 2-hours (the first 2 results in the first list published above). A big difference.

Unfortunately, we’re not in a position to attempt to convert all the multiple scripts into PSOS, the above has eaten a huge amount of testing time already.

All the best

Andy

6

Scribbles

Final testing has now been completed and, as suspected, the Intel processes and Rosetta 2 have a massive impact on performance. Here are the last test results with FileMaker Pro 19.4.1 processing local (non-hosted) files:
MacBook Pro M1 Pro 14-inch without Intel processes running - 1st test 01:32:12
MacBook Pro M1 Pro 14-inch without Intel processes running - 2nd test 01:34:27
MacBook Pro M1 Pro 14-inch with Intel processes running 02:24:14

The above would appear to confirm that running Intel processes in Rosetta 2 does significantly slow down an M1/M1 Pro MacBook when processing a lot of data within FileMaker.

However, it doesn’t explain why the original M1 MacBook Pro 13” is so consistent in taking about 40 minutes to complete the task, whereas the, supposedly, faster M1 Pro MacBooks’ processing time varies from 40 minutes to roughly 1.5 hours.

I’m convinced, albeit have no proof, that if I had wiped the M1 Pro machines and reinstalled Monterey afresh we’d get the same performance as the M1 MacBook. There has to be something running that is hampering their speed. FileMaker Plugins were involved. The essential plugins were MailIt and Scribe and I’ve just realised that on our production Macs we also have 2empowerFM Desk Assistant and Reactor 6. I guess another test should be in order as and when we have time. Equally, it could be something else. Both M1 Pro MacBooks were setup without any settings or application transfer to avoid bringing old stuff over.

5 Likes