For solutions NOW, You can work around this problem in just about any version of FMP that supports JDBC. At least back to version 11.
I more or less gave up hoping FMI will fix gaping functionality holes or bugs and now just work around them.
JDBC is a good solution and it gets record looping OUT of FMP. 
Going JDBC. This sounds like a typically Claris approach of how to do things:
Set the nail on the wall and then repeatedly move wall and nail against the hammer 
I'm just suggesting getting the work done and not waiting for FMI to get around to fixing anything (as they often don't). My clients don't care about documenting FMP's bugs: they want working functionality. Now.
Instead of these back and forths, ...why don't we do an actual benchmark? The ones I've done have always been at least 3X on SQL performance alone. But, with looping, it's many, many times faster.
I wrote some code a while back to help a poster where that OP had about 18 Billion calculations to do on a main table and then for every record in the main table, scan every record of a child table (and find the optimum child record after doing computations on every child record). Another poster wrote a custom plug-in to speed things up from the six or seven HOURS FMP took. His plug-in completed in 1 hour. Using JDBC and some simple optimization, my code completed in 30 minutes. The JDBC code could also run standalone (be scheduled to run at various times automatically) and could run with other DBMS' with the same structure.
NO FMS REQUIRED (but of course, same JDBC code would work fine with FMS or other JDBC DB.)
I also moved the data to MySQL and used the SAME JDBC code where the same task took only 12 minutes.
So, 6 or 7 hours with FMP or 12 minutes with MySQL. Hmmm....
So, ...JDBC has the added advantage for when you move off FMP to an Enterprise database with the same schema --> your code still works! All RDBMS support JDBC. JDBC Is a standard (non-proprietary).
A quick thanks to everyone who contributed to this post.
I was sleeping while most of it was happening, but some brilliant testing, above and beyond the call of duty contributions and some excellent information for others.
Well done 
Andy
The last task is classification of the observed.
A - my script is misconfigured
B - it is a bug
C - it is a feature and intended behaviour
I will classify it as a bug, because the sorting cannot be controlled and happens out of scope.
Job well done gentlemen. It is a privilege and a pleasure sharing this space with you!
Meanwhile in a Universe far far away:
https://community.filemaker.com/en/s/question/0D70H000005eGlX/has-the-new-community-lost-some-of-its-best-contributes-or-am-i-using-it-wrong?s1oid=00Di0000000eyqY&s1nid=0DB0H000000fxYs&emkind=chatterCommentNotification&s1uid=0050H00000Bq5vP&emtm=1578947015003&fromEmail=1&s1ext=0
The pleasure is mine. It is a well-managed forum which truly supports collaborative work both technically and by spirit.
A sad story.
Davidâs observations/questions:
1-how big in bits the source table?
2- Did anyone try to create the records in another window ( which would add time but would save time because there wouldnât be any context change)?
Beware using too many repeatedly opened windows Cecile. Multiple windows significantly increases the RAM footprint of FileMaker, the RAM used never goes down until it is quit. Iâd be wary of the high numbers involved here.
Kind regards
Andy
If you go to window instead of opening it every time?
Thereâs a sample file in the OP for reference.
I donât use additional windows for data processing.
Ok Iâll look into it in a bit
I am of the camp of fm devs who prefer to avoid opening a new window to do processing, and typically I have a couple of other go-to methods that I try to use first. As mentioned/implied by @AndyHibbs's comment, I have seen clients experience performance penalties for using Open Window too many times during a client session. I have tried an approach that recycled a single hidden window so that I could have the separate processing without constantly opening up new windows, but that approach never really grew on me.
Having said the above:
In this particular scenario that we are looking at:
Then I would go against my norms, and entertain the idea of doing some processing in a new window -- perhaps especially if this were a server-side script, meaning that window-flashing would not be an issue.
I did not test this option, but it did occur to me, after the fact, that it could be a possible workaround.
Thanks, @Cecile for raising this point.
Kind regards,
-stee
13,171,400 bytes or ~13.7MB
No, I didn't test opening a second window and keeping it open.
I'm not sure about that assumption. Switching from one context to another context is going to have similar requirements when it is switching layouts within one window or switching to a second window. In both cases the data for the current context has to be loaded.
The fact that the curve disappeared when the sort was removed makes me think that the problem isn't the amount of memory required for data but the amount of processing time required for mapping the sort to the data.
It turned out that the issue is located outside the loop. I suggest a new title for this thread in order to make itâs subject matter clearer.
âSort Records By Field () script step causes performance degradation in loop sequencesâ
All, do you agree, have different proposals? Thank you for feedback.
Hello @Torsten
Rather than tying the performance degradation to the granular level of "Sort Records By Field", I might consider tying it to the found set being in a sorted state. This would also encompass other scenarios, e.g. where the records were sorted by Sort Records.
I do not feel strongly about this, however. I am sure that whatever you rename the thread to, it will be a good choice.
Kind regards,
-steve
I'm very late to the party, but I had a thought...
I wondered if when returning to the layout that is sorted, there is an overhead in checking if it is still sorted correctly or something similar. Whilst I accept that there can be memory issues with loading and closing a lot of windows, I wonder if using a tiny card window which opens directly to the target layout might be worth testing. (I'd turn off the dim parent window option)
just a thought...
PJ
Hi @apjcs/Paul
I agree with you, my first gut guess was that FMP is doing a check to see if the sort order needs updating -- but, that's just a guess.
Enough of us have tossed around the idea of a new window such that it seems worth testing. If no one else does so before then, I'll try to test this out before the end of the week.
On another note:
I'm really glad to see you posting here at this forum. I remember and appreciate all the work you did with respect to figuring out and comparing the different ways to scrape a found set into JSON.
Welcome!