Overriding Variables using the Data Viewer

Variables defined in the data viewer are available to all objects in scope

If it is used carelessly this can cause problems which range from mild to severe.

Yesterday I was using the debugger to test a new script. It was parsing JSON pulled in from an API. When I ran the script without the data viewer open, it populated the records. When I ran the script with the data viewer it threw up JSON evaluation errors.

What was happening?

The Data Viewer can over-ride variables defined in scripts and layouts. This allows us to inject data into running scripts, layout level evaluations, and menus. It's a powerful tool.

In this case, I did not want to inject the data. It was happening because I had left calculations in the data viewer. The data viewer contained LET expressions that defined local variables with names that were the same as the local variables defined in the script. When the data viewer was open and active it triggered the calculation of the variables. The new value overrides the values set in the script and because the new data was not valid JSON, the JSON parsing steps would generate errors.

What's the Solution

  1. Don't run scripts with the Data Viewer open.

  2. Remove your tests from the Data Viewer.
    Don't think of it as a library of code snippets.
    Treat your data viewer like a scratch-pad.
    Use it for immediate tests, and even then, use it with caution

  3. Comment your tests
    If you want to preserve your test code, wrap it in C style comments.
    /* Let ( $msg ="like this" ; $msg ) */

If you are Deliberately invoking this Functionality

There are gotchas for you too

  1. Watched variables are not always evaluated.
  2. When your watched variable list is long enough that variables are not visible in the window they may not be evaluated.
  3. This could cause your test to fail because the over-ride values you specify have not been brought into scope.
  4. Scrolling through your watched variables to bring them into view will normally trigger the evaluation.

Here's a file that you can use for testing.

OverrideDataVariable.fmp12 (260 KB)

7 Likes

Quick note about C style comments. Also be sure to add another valid expression before/after your commented-out code, otherwise when you reopen that DV expression, it will have wrapped your expression in quotes without your permission!

Screen Shot 2021-07-28 at 7.00.51 PM

6 Likes

Does switching off 'Automatically evaluate' prevent the injection of values in variables of the current scope?

Agreed on this -- but if you forget, you can get your original calc back fairly painlessly by taking the quoted expression, and evaluating it in the Data Viewer. The act of evaluating it should return your original calculation expression.

3 Likes

Good question!

No. That switch does not prevent the data injection.

If you have the data viewer open to the "Current" tab/pane you do not get data injection. If you have the "Watch" pane open you will get data injection.

1 Like

Oops, this is defying my sense of logic. When I tell the system to NOT evaluate, it still evaluates?

Yeah, "Automatically Evaluate" only applies while you have the data viewer calculation window open. When it's ticked, the calc result updates as you type. When it's unticked, the calc result only updates when you click "Evaluate".

But, when you close the calc window for a Watch expression, it starts automatically evaluating in the Watch list. I wish you could turn off specific watch expressions!

2 Likes

This behavior makes sense to me. I use the data viewer to SEE what are the values while a script running in debugging mode.

1 Like

That doesn’t make sense to me. The debugger needs the data viewer to observe the correct value results otherwise why debugging at all - just for watching instruction flow?

Instead I would claim to strictly code side-effect free (functional) in anything within the calculation engine therefore the data viewer could be completely tamed and the data flow in calculations would be easier to be proofed for correctness …

Good Point. We have to be specific about the two tabs displayed by the Data Viewer: Current and Watch. When the Current tab is open, the script variables are not over-ridden. When the Watch tab is open, the variables will be over-ridden.

  1. Don't have the data viewer open when it is not needed
  2. Watching current values with the Current tab is safe
  3. Watching values with the Watch tab will override variables.

Only if side effects are implied in code (which can be easily avoided if properly expressed (following functional programming) ..

I'm discussing side-effects. So, yes, this is possible. And functional programmers get that wrong from time to time. This information is for everyone who is not a code-perfect functional programmer.

Please elaborate on the dos and don'ts of functional programming in FM. This would be helpful for all who want to improve their skills (maybe in a new OP like 'functional programming, best practise').
Thanks!

3 Likes

This is very simple. Just avoid any $var or $$var assignments via let or implicitly within Evaluate. Otherwise FM is very safe. (That’s why ExecuteSQL only returns results without side effects .. )

What is functional programming and what are side-effects. What needs to be properly expressed and how do you express properly.

(Can you tell I didn’t understand a thing in the last several posts—not that i understood very much before, aside maybe that the watch tab is dangerous although I don’t really get how scripts running would get values inputs from the Data Viewer (where does it get those anyway?!))

For an explanation of Functional Programming see Functional programming - Wikipedia

I stopped reading when I saw that Lisp was the first Functional Programming language :nauseated_face:. I used it a bit a long time ago, what a strange language.

In the Watch window one can change the content of a variable.

1 Like

Lisp is one of the most powerful programming languages allowing to alter its own code during execution. It has influenced Swift, Python and others majorly. FM Let and Evaluate functions are originated there most likely - wherelse?

1 Like

everything you can do in the Data Viewer (or in the 'calc engine') is functional programming within FM.

A side-effects is the result of a script step like Set Field or Replace where data will be altered or a state change is implied outside the current scope like Print, Goto Layout, GTRR ..
In contrast a function has an input (or non = void) and returns a result (output). A script step just changes the state or data.

1 Like