The slowdown also appears without the logging steps. That's how I came to investigate.
All help is appreciated, I want to solve the riddle.
Thanks @jormond.
That's exactly the kind of set up that I was hoping to see.
Just so you know: I really appreciate that you go to the effort to to this.
2 Likes
I ran it two times, first with data logging active and then with data logging disabled. There is no discernible difference. Both rise to 40-45 ms for 'Go To Record (Next)' and remain at 4-6 ms for the write section over 10'000 records.
Haven't tried that yet.
That’s just weird. It’s as if it was reloading table A after writing in table B.
It loops over table 'Source' in layout 'Source' on which the 'Go To Record' step is executed.
After reading data from Source::Record, it switches to layout 'Target' i.o. to write data to table 'Target'.
It then switches to layout 'TimeLog' for writing log data to table 'TimeLog' and then switches back to layout 'Source'. A reload of table 'Source' records in layout 'Source' should not happen.
Can you reproduce this reload?
Here's another approach. added a Pause (0.1) step before the 'Go To Record' step. The latter still takes more and more time to execute. Loop over 1000 records.
This issues seems to be caused by the section that creates a target record. When I disable those lines the go to next record speed does not balloon anymore and GTNR just stays at <1ms each
2 Likes
Another thing that I am noticing is that the "Target "layout is set to a default view of Table.
Among other things, I am going to try making sure that everything is done in a form view, and see what happens.
-steve
1 Like
OK, some VERY interesting results from my last test. If I move the logic from that create Target Record section I disabled into a subscript then the GTNR ballooning still doesn't happen. So I would suspect that FM has some bad memory management going on behind the scenes and it's storing more and more info in memory as it creates those target records. But by moving the record creation to a subscript, it can purge the memory it devoted to that subscript as soon as that script is removed from the stack:
So either it's a memory leak, or it's deliberate memory management but it is not optimized for this task (assuming it's memory-related). But if you just need it to "work" give this a try
5 Likes
My test with blank layouts ( screen shots above ) used form view.
1 Like
Changing default view to Form does not affect the outcome, unfortunately.
That's a really interesting observation @jwilling. I'd been thinking of running a test with chunking to allow FMP to recover memory.
I have just run @Torsten's script, and broke out the navigation component from the create record component. The time taken does not increase over time for either of those components. ( I shouldn't say that because the time taken for those tasks increases to 2ms after approx 3500 loops. At about 4500 loops the time increases to 3ms )
2 Likes
I should mention that running the original script brought my tiny test server to its knees, suggesting that it is memory related. If it were properly purging then the resource usage would've remained somewhat constant.
3 Likes
One of the features of the graphs on my machine is the regular spike. I'm assuming the spike is associated with FMP internal mechanisms doing their housekeeping.
2 Likes
Well then it "missed a spot"! 
3 Likes
PS, I don't wanna chalk it up to "bad" memory management, because it might actually be an intentional optimization in some, more common scenarios. But it definitely causes problems in this one.
3 Likes
LOL 
1 Like