Python evaluation of a Sympy expression

[Alexishul]

Hi, i want to make a python code able to evaluate expressions from sympy (typically with derivatives). However it doesn't work, i guess Mathics is not able to parse "Derivative"

Here's my code :

session = MathicsSession()

x,y = symbols("x,y")
undefined = y
proposed = x

ode = Derivative(undefined, x, x) + 100*undefined
print(ode)

print(25*"*")

ode_subs = ode.subs(undefined, proposed)
print(ode_subs)

print(25*"*")

ode_subs = str(ode_subs)
print(ode_subs)

print(25*"*")

ode_eval = session.evaluate(ode_subs).to_sympy()
print(ode_eval)

Here's the output :

100*y + Derivative(y, (x, 2))
*************************
100*x + Derivative(x, (x, 2))
*************************
100*x + Derivative(x, (x, 2))
*************************
Traceback (most recent call last):
  File "C:\Users\alexi\OneDrive\Bureau\Thales\testFunction.py", line 31, in <module>
    ode_eval = session.evaluate(ode_subs).to_sympy()
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\session.py", line 90, in evaluate
    expr = parse(self.definitions, MathicsSingleLineFeeder(str_expression))
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\util.py", line 21, in parse
    return parse_returning_code(definitions, feeder)[0]
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\util.py", line 31, in parse_returning_code
    ast = parser.parse(feeder)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 64, in parse
    return self.parse_e()
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 101, in parse_e
    result.append(self.parse_exp(0))
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 123, in parse_exp
    new_result = self.parse_binary(result, token, p)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 242, in parse_binary
    expr2 = self.parse_exp(q)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 131, in parse_exp
    child = self.parse_exp(q + 1)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 110, in parse_exp
    result = self.parse_p()
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 146, in parse_p
    return method(token)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 286, in p_RawLeftParenthesis
    self.expect("RawRightParenthesis")
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 92, in expect
    raise InvalidSyntaxError()
mathics_scanner.errors.InvalidSyntaxError

Maybe there is a way to convert expressions to be parsable or something like that ?

[rocky]

Function parameters in Mathics3 and WMA are surrounded in square brackets and lists are surrounded by braces Derivative[y, {x, 2}] is valid while what you have shown in the print statement is not.

Whenever you get an InvalidSyntaxError, paste the string inside a mathics or mathicsscript session and it will tell you why it thinks the syntax is incorrect.

[Alexishul]

Ok i understand, but is there a function which convert sympy expression such as :
Derivative(y, (x, 2))
into Mathics/WMA expression like :
Derivative[y, {x, 2}]
? (In general cases, for example for Integrate and more)

[Alexishul]

That's weird i still have the same issue, maybe I did not understand very well :
Here's the code :

session = MathicsSession()

x,y = symbols("x,y")
undefined = y
proposed = 5*x

ode = Derivative(undefined, x, x) + 100*undefined
print(ode)

print(25*"*")

ode_subs = ode.subs(undefined, proposed)
print(ode_subs)

print(25*"*")

ode_suite = from_sympy(ode_subs)
ode_suite = str(ode_suite)
print(ode_suite)

print(25*"*")

ode_eval = session.evaluate(ode_suite)
print(ode_eval)

Here's the output :

100*y + Derivative(y, (x, 2))
*************************
500*x + Derivative(5*x, (x, 2))
*************************
System`Plus[System`Times[500, System`x], System`D[System`Times[5, System`x], <ListExpression: (<Symbol: System`x>, <Integer: 2>)>]]
*************************
Traceback (most recent call last):
  File "C:\Users\alexi\OneDrive\Bureau\Thales\testFunction.py", line 41, in <module>
    ode_eval = session.evaluate(ode_suite)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\session.py", line 90, in evaluate
    expr = parse(self.definitions, MathicsSingleLineFeeder(str_expression))
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\util.py", line 21, in parse
    return parse_returning_code(definitions, feeder)[0]
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\util.py", line 31, in parse_returning_code
    ast = parser.parse(feeder)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 64, in parse
    return self.parse_e()
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 101, in parse_e
    result.append(self.parse_exp(0))
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 119, in parse_exp
    new_result = method(result, token, p)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 577, in e_RawLeftBracket
    seq = self.parse_seq()
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 198, in parse_seq
    result.append(self.parse_exp(0))
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 119, in parse_exp
    new_result = method(result, token, p)
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 577, in e_RawLeftBracket
    seq = self.parse_seq()
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 198, in parse_seq
    result.append(self.parse_exp(0))
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 110, in parse_exp
    result = self.parse_p()
  File "c:\users\alexi\anaconda3\lib\site-packages\mathics\core\parser\parser.py", line 154, in parse_p
    raise InvalidSyntaxError()
mathics_scanner.errors.InvalidSyntaxError

Do you have some clue ?

[mmatera]

Notice that
SystemPlus[SystemTimes[500, Systemx], SystemD[SystemTimes[5, Systemx], <ListExpression: (<Symbol: Systemx>, <Integer: 2>)>]]`

is not a valid expression. This is issue from the implementation of ListExpression.__str__, and would be fixed by #847

[rocky]

The phrase "vaid expression" is a bit vague and I am not sure if there is an issue here. Although we may decide to change how str() works, I am less certain about repr() which is what is done in that PR. Although we changed how str() works, this is not without some cost of confusion.

The part of this PR that is not good is that it blurs the distinction between a Mathics3 input string and a string presentation of internal representation of its M-expression. And this is likely to aid in the kind of confusion that we see here. In fact, in the bad old days, there was even more of this going on which also cost us performance.

On input, we do not allow <Integer: 2> or <Symbol: System X>. As a repr() object this is fine and I would argue desirable.

We could decide to make str() to be more "helpful" and turn this into "2" and "X", but once we go down this route, then there will be a temptation to convert Global`x into the more normal x and maybe a desire to turn x^-1 into 1/x. In other words, trying to force the str() function to work the way that some particular Form (maybe InputForm?) works.

I do not see this as helping programmers separate the two concepts of the internal data structure used versus some desired output Form - whether it is InputForm or, more likely, some other kind Form that looks nicer or more natural.

What the developer probably wants and should use is to pass the internal structure through Form processing.

[mmatera]

"valid expression" === "expression that can be successfully parsed by the interpreter".

And the aim is not to use str to "format" the expression, but just to produce a str representation that can be evaluated afterward. So, the string value should contain the context for all the symbols (like in FullForm), but eventually using some short-hand notation (like {a,b,c} instead of List[a,b,c]). In any case, at this moment, I would be happy to have something that can be reentered into the parser.

[rocky]

And the aim is not to use str to "format" the expression, but just to produce a str representation that can be evaluated afterward

My point is that what was there was not incorrect, it is just that there is some new property that one might think nice to have, but is done at the expense of leading down a path that should not be done this way and causing possible confusion. And this is why the discussion continues over further things that need to be done with the string output. In the end, what is getting done is to simulate some sort of hacky Form processing, without going through the Form process that is typically done to display results.

[rocky]

There are some fundamental concepts going on that are getting glossed over. session.evaluate takes a string argument, parses that into the internal form which for a compound object is an Expression; it then uses the evaluate() method from the parsed object/expression. You can think of a Mathics3 Expression as roughly an S-expression. But for things like Derivatives, it is more properly called an M-expression, because the left-hand side (CAR in Lisp), is not a single symbol containing a function name.

The mathics.core.covert.sympy.from_sympy() function starts with the internal form which is generally that Expression kind of object. And that is from_sympy()'s and to_sympy()'s main purpose: to be able to translate back and forth between internal forms for purposes of evaluation.

In order to go from a native sympy or an internal Mathics3 object to a naturally formatted Mathics3 language string, Form processing is done; this involves calling format methods.

Some of this is described in the developer guide: https://mathics-development-guide.readthedocs.io/en/latest/extending/code-overview.html so please see that for more information.

[mmatera]

Actually, this should work if we fix an inconsistency in the way the method ListExpression.__str__ is implemented, compared with Expression.__str__

[rocky]

While it may be the case that we might change the string representations of some for str() and repr(), I think there is still some confusion here. A string representation of the internal data structure produced is not the same thing as the WMA or Mathics-like input. More detail should be discussed in that specific PR.

[rocky]

Looking at the PR, I think it is good with some very minor changes. But I do want to emphasize that this is not a replacement for Form processing, and this should be be used to accomplish to produce WMA or Mathics3 input or output.