Filemaker Native CF Performance

Lounge (Discussions)
21

As I'm sure you have read in books like "Code Complete", Benchmarking isn't meant to be fun; rather, it's meant to nail down a key non-functional requirement: Performance.

There's nothing like a real example!

Mr. OP -- Below is a quick benchmark video I did for you so we can get past the performance opinion 'hand-waving' and see something usable ... in action.

In the video link below, you'll see I used two lists -- each with 50,000 rows and a micro-service (not a plug-in). I quickly demonstrate several of the implemented list operations I've had running for .... years. These only took a little while to implement in the first place ... no waiting for a plug-in vendor, or for FMI, etc. These functions are in complied code so they should run faster than a scripted solution with larger data sizes.

(Works with FMP 12+ FMS not required but would work. Also works with any HTTP-enabled application on Mac, Windows, or Linux.)

See what you think.

22

Attached a sample file using javascript and some CFs. Interestingly javascript through a webviewer is way faster than executed through a plugin (BE, MBS), fastest CF I found is Bruce Robertsons AntiFilterValues, the worst is OP's CF ZapValues....

best
Otmar

FilterList1ByList2.fmp12.zip (1.3 MB)

4 Likes
23

How does yours compare with my video results above for 50,000 rows in each list?

Thanks,

24

@anon45965781, give it a try, put your lists in fields UI::List1 and UI::List2
I don't see a benchmark in ms in your video.
Javascript in webviewer takes ~900 ms for ~34000 rows in List1 and ~3400 rows in List2.

1 Like
25

LOL ... I figured the OP could see how long it takes in my video and determine from there.

Also, I don't think what I'm doing in my logic matches what you're doing in your logic so the benchmark comparison may not make any sense anyway.

26

My logic does what the OP was asking for: remove values in List2 from List1.
50000 UUIDs in List1 and List2, 10000 matching --> JS in webviewer takes 8.5 seconds using the code from my file. Using a more modern JS-syntax it comes down to 4.7 seconds:
var list3 = list1.filter( ( el ) => !list2.includes( el ) );

2 Likes
27

It's all good. The syntax between Java and JavaScript really isn't all that different in this case.

For 50,000 elements, based entirely on the video, mine takes about 1 second per operation. I suppose it really comes down to the OP's performance requirement and what language he's most comfortable with. JavaScript is a fine solution.

My existing logic had built-in other list operations I came up with back when I posted that solution on the "FMI" forum. Those other operations may be of no interest to the OP.

Thanks...I appreciate you taking the time to reply. And, like me, you posted an actual solution to remove all the hand-waving so common on posts like this one. :slight_smile:

1 Like
28

@Otmar

Thanks for this contribution to this thread. I appreciate you taking the time and effort as you did.

29

Is that 1 sec to process in the micro-service? Or 1 sec for the entire process including returning and handling the final list in FileMaker?

30

After all the postings above, you didn't watch the video I created with a sample FMP application?

Your answer awaits... :sunglasses:

LOL

31

Oh no. I didn’t. Dealing with helping setup the infrastructure for our many employees to work from home.

I’ll look at it later today.

32

Thanks.

33

Hi fmpdude - back from holiday now (by the skin of my teeth - think I was the last person flying through geneva!) - Trying to look at the video but it's no longer there - any chance of a repost?

1 Like
34

Hey James!

Sorry about that. I wasn't sure anyone was interested in a micro-service approach so I removed the video. After your posting above, I re-posted the video above for you. The video link above shows the micro-service running on the same machine as the FileMaker application.

Note: I updated the video this morning (Monday, March 16, 2020) to also include a larger list test with 100,000 items in each list. And, each list has more realistic differences you'd probably have in production.

THE VIDEO WILL BE AVAILBLE UNTIL 2020-03-19

The first list demo you see in the video assumes "worst case": list1 and list2 are mostly, if not totally, different. In practice, you probably wouldn't have 50,000 items in each list being totally different. So, that's why I included the second list test with 100,000 items in each list.

(I'll be glad to run some sample list values you have if you zip them up and post them.)

Note that the micro-service approach coding is similar to the JavaScript approach. As others have noted, there are pluses and minuses with any approach.

I opt toward micro-services since neither I nor my clients are in a FMP-ONLY environment. Thus, I need any (compiled) code routines I create to be available to these other applications and clients.

Let me know what you think. Setting up a micro-service only takes a couple minutes -- there are two videos I've posted that walk you through the setup and adding new GET methods.

The code is actually pretty easy for this usage. Using built-in list methods (as also in JavaScript), you can easily show just what's in list 1 that's not already in list 2, like this:

image

Lastly, if you have any issues (setup or otherwise), just post back, again, assuming this approach sounds good to you, and I'll try to help!

Good luck. :slight_smile:

1 Like
35

+100

2 Likes
36

Thanks Steve!

Did you watch the video? I really like your YouTube videos, so any suggestions you could offer about how I could make mine better would be appreciated.

Thanks again.

37

Thank you @anon45965781

The questions that I used to try to honestly answer whenever I made a video were:

  1. What would I most like people to learn or take away from watching the video?

  2. What do I think a viewer is hoping to learn, and what level of detail would be most helpful given their imagined objective(s)?

If you can send me answers to those two questions, I'll watch the video you posted and try to give you feedback through the lens of your answers to the above. Perhaps this should be via PM so as not to take this thread off-topic.

Thanks! I look forward to hearing what you have to say.

-steve

3 Likes
38

Hi Steve,

Good questions. Answers below.

  1. With the video above, the goal was to show performance for comparison to other approaches for medium to larger datasets for various list operations.

  2. I assume in this example, that the viewer, seeing a FMP application doing list operations, would be able to determine if it's "fast enough" to justify the approach. I did post a single line of code to more or less match the JavaScript code also posted.

I also try to make videos engaging by using some music, animations, and transitions. I would use a voice approach for a detailed coding video (beyond the "adding a method" video I have here).

Appreciate your kind reply. :sunglasses:

Thanks,