N.B. I'm posting this in the Community too.
Looking for some advice on an issue one of my customers has had lately. They are experiencing significant slowdowns and server not responding warnings at random times - FM is slowing to a crawl. This started in FMS 16, remained in 18. When they still had the issue they upgraded to 19 (in the last few days). Clients are FMP 18/19 and FMGo 18. Initial diagnosis suggests multiple logins for the same client, which intriguingly have the same timestamp to the millisecond. These show in the log as 5 separate events for each client.
The issue manifests similarly to this Community thread: https://community.claris.com/en/s/question/0D50H00006h91bW/fm-server-14-multiple-users-logged-in-multiple-times-on-their-same-device
Server (up to date v19) is hosted on a Mac Pro (Rack, 2019) with 1912 GB memory running Catalina 10.15.7
Approx 40 simultaneous connections, FMP and Go all Mac based. No web direct.
Network is 10gig (copper) connection to Netgear M4300 series virtual stack switch fabric. Port is auto-neg, full-duplex. DNS - derived from local firewall/router - Fortigate 60E. Generally very stable, no observed issues with dropped/malformed packets.
Am I missing anything obvious, or has anyone encountered anything similar? Any help or suggestions would be greatly appreciated.
Can you link the slowdown to particular user actions or scripts executing?
It doesn't seemed to be linked to any particular script or action. This is a highly complex system written by my customer. My role is to provide advice and guidance. I have asked them to enable the top call stats log and await the results of that.
Top call stats, in conjunction with the other logs, should identify the issue, especially if you can log the actions of the primary culprit .
Did you already check network conditions? It’s not only up-/download performance but also latency (Ping) that counts.
Thanks for all the input everyone. It looks like the causes were multiple:
Early on it was found that Spotlight had not been disabled but doing so did not solve the issue completely.
There were two other issues that werE more directly causative. First one table had a large number (approx 5 million) that were being searched as part of a script. Purging these records helped. N.B. To do this it is best to delete in batches of day 5,000 in a loop with a 10-15 minute pause between each (thanks @GaryPalmer for that suggestion). The script has also been rewritten.
Second and more worrying we found a rogue instance of FileMaker Server 18 running at the same time. We have no idea how that happened having run through a set of standard upgrades from 16, to-7 to 18 to 19.
Also confusing is how a copy of Rapport had been installed (albeit not running) at some point on a machine purchased new that had never knowingly had it installed.
Finally, we upgraded the OS to Big Sur and did a fresh install of FMS19. So far so good.
The final possible contributory factor is the inevitable patchiness of Wi-fi in the warehouse. Thanks again to @GaryPalmer for his insights on how to fix this.
Thanks are due too, as well as this forum’s members, particularly @WimDecorte, to Claris tech support who pinpointed the second instance of FMS 18 and Rapport.
@Code-J, thank you for the detailed description of things you investigated and remediations.
Could you please elaborate why purging of records is better done in batches and a minutes-long pause between purge runs?
Of course. The pause of 10-15 minutes allows FMServe time to free up its resources for other activities. Using this strategy, users will barely notice the interruption of purging 5,000 records every 15 minutes or so, whereas purging 5 million records will tie up the server resources whilst it chugs through the lot, which can take a while. Truncate, apparently, is also a big no no in this situation.
Actually a pause/resume of a very short duration, executed more often and on fewer deletes per cycle, might prove to be more effective. Most FM actions are very quick and use little resource; even a 1 second pause in the delete code would provide a lot of processing headroom.
On a related server performance note: seeing significant slowdowns with FMS 19.4 if the client is any earlier version. This is on a similar scale windows server to your MacPro but with 500 users and 30 million records. Updating FMPro to 19.4 has reduced the user complaints over performance differences.
Update: updated to 19.5.1 a couple days before the 2 major issues in 19.5.1 showed up in 19.5.2 release notes. Immediately updated to 19.5.2 and all has been well since.