🧮 Enhanced `param` Class for Safe Evaluations in Pizza 1.0x

Enhanced param Class for Safe 🧮 Mathematical Expressions in Pizza 1.0x

read also the updated Wiki page: Mathematical Notations and Capabilities in param():

The param class within the pizza.private.mstruct module has been significantly extended to manage complex mathematical expressions securely. This enhancement allows for the evaluation of expressions both within and outside ${...} placeholders without relying on Python's eval function, thereby mitigating potential security risks. The solution employs a param.safe_fstring() method to safely evaluate expressions, ensuring that mathematical computations are handled efficiently and securely.

Table of Contents

• Overview
• Key Features
• Shorthand Syntax for NumPy Operations
• Supported Functions and Operators
• Usage Examples
  • Basic Scalar Operations
  • Vector and Matrix Operations
  • Advanced Matrix Computations
  • Evaluating safely expressions with context
• Implementation Details
• Important Considerations
• Conclusion

---

Overview

The enhanced param class facilitates the secure and intuitive evaluation of mathematical expressions within string templates using ${...} placeholders. Expressions can be embedded within strings or used as standalone scalar expressions outside of ${...}. While expressions inside ${...} support complex operations, including matrix indexing and slicing, those outside are primarily limited to scalar computations. This dual capability ensures flexibility while maintaining system security.

---

Key Features

• Safe Evaluation: Utilizes the safe_fstring() method to evaluate expressions without using eval, ensuring system security.
• Intuitive Syntax: Employs ${...} placeholders for embedding expressions within strings, making the syntax familiar and easy to use.
• Shorthand Syntax: Provides simplified expressions for creating and manipulating vectors, matrices, and operations.
• Dynamic Context: Variables and expressions can depend on each other, with automatic resolution and type preservation.
• Supported NumPy Operations: Includes advanced operations like matrix multiplication (@), transposition (.T), and slicing.
• Comprehensive Function Set: Offers a rich set of mathematical and random functions for computations.
• Error Handling: Provides robust error management for undefined variables or invalid operations.

---

Shorthand Syntax for NumPy Operations

1. Vectors:

   • $[...]: Creates a NumPy vector and ensures it is at least 2D.
     • Example: $[1, 2, 3] → np.atleast_2d(np.array([1, 2, 3])).

2. Matrices:

   • $[[...]]: Creates a 2D NumPy array.
     • Example: $[[1, 2], [3, 4]] → np.array([[1, 2], [3, 4]]).

3. Variable Conversion:

   • @{var}: Forces a variable (var) to be recognized as a NumPy array and ensures it is at least 2D.
     • Example: @{n} → np.atleast_2d(np.array(n)).

4. Indexing and Slicing:

   • ${expr}: Supports indexing and slicing operations.
     • Example: ${p[1,1]} → Accesses the element at (1, 1) in matrix p.
     • Example: ${p[:,1]} → Retrieves the second column of p.

5. Operations:

   • Supports standard arithmetic (+, -, *, /, **) and advanced matrix operations (@, .T).

---

Supported Functions and Operators

Mathematical Functions

• Trigonometric Functions: sin, cos, tan, asin, acos, atan, atan2, radians, degrees
• Exponential and Logarithmic Functions: exp, log, log10, pow, sqrt
• Rounding and Modulus Functions: ceil, floor, fmod, modf
• Absolute and Hypotenuse Functions: fabs, hypot
• Constants: pi, e

Random Functions

• gauss, uniform, randint, choice

Operators

• Arithmetic Operators: +, -, *, /, **
• Matrix Operators: @ (multiplication), .T (transpose)
• Indexing and Slicing: [ ], [:, ]

---

Usage Examples

Basic Scalar Operations

from pizza.private.mstruct import param

p = param(debug=True) # to show eventual evaluation errors
p.a = 1.0
p.b = "10.0"
p.c = "${a} + ${b}"  # Scalar addition
print(repr(p))

Output

# repr(p)
  -------------:----------------------------------------
              a: 1.0
              b: 10.0
               = 10.0
              c: ${a} + ${b}
               = 11.0
  -------------:----------------------------------------
parameter list (param object) with 3 definitions

---

Vector and Matrix Operations

p.c = "$[${a}, 2, 3] * ${b}"  # Creates a NumPy vector and scales it
p.n = "$[0, 0, 1]"            # Another vector
p.o = "@{n}.T * 2"            # Transpose and scale
print(repr(p))

Output

# repr(p): 
  -------------:----------------------------------------
              a: 1.0
              b: 10.0
               = 10.0
              c: $[${a}, 2, 3] * ${b}
               = [10 20 30] (double)
              n: $[0, 0, 1]
               = [0 0 1] (int64)
              o: @{n}.T * 2
               = [0 0 2]T (int64)
  -------------:----------------------------------------
parameter list (param object) with 5 definitions

---

Advanced Matrix Computations

p.p = "$[[1, 2], [3, 4]]"      # Create a 2D NumPy array
p.q = "${p[1, 1]}"             # Indexing: retrieves 4
p.r = "@{p}[:,1] + 1"          # Add 1 to the second column
p.s = "@{p}[:, 1].reshape(-1, 1) @ @{r}" # perform p(:,1)'*s in Matlab sense
p.t = "np.linalg.eig(@{s})"    # Compute eigenvalues and eigenvectors
p.u = "The first eigenvalue is ${t.eigenvalues[0]}" # Print the first eigenvalue
p.v = "The second is ${t.eigenvalues[1]}" # Print the second eigenvalue
print(repr(p))

Ouput

# repr(p):
  -------------:----------------------------------------
              a: 1.0
              b: 10.0
               = 10.0
              c: $[${a}, 2, 3] * ${b}
               = [10 20 30] (double)
              p: $[[1, 2], [3, 4]]
               = [2×2 int64]
              q: ${p[1, 1]}
               = 4
              r: @{p}[:,1] + 1
               = [3 5] (int64)
              s: @{p}[:, 1].reshape(-1, 1) @ @{r}
               = [2×2 int64]
              t: np.linalg.eig(@{s})
               = EigResult(eigenvalue [...] 576, -0.89442719]]))
              u: The first eigenvalue [...]  ${t.eigenvalues[0]}
               = The first eigenvalue is 0.0
              v: The second is ${t.eigenvalues[1]}
               = The second is 26.0
  -------------:----------------------------------------
parameter list (param object) with 10 definitions

Error Messages

Some error messages are shown before the output when debug=True:

DEBUG u: Error in Evaluating: The first eigenvalue is 0.0
< invalid syntax (<unknown>, line 1) >
DEBUG v: Error in Evaluating: The second is 26.0
< invalid syntax (<unknown>, line 1) >

They appear because the string content “The first value…” and “The second…” cannot be interpreted anymore as valid mathematical or NumPy expressions. The content is converted into a string when they cannot be anymore interpreted.

Extensions

The mathematical interpretation can be pushed further:

p.w = "${t.eigenvalues[0]} + ${t.eigenvalues[1]}" # sum of eigen values
p.x = "$[[0,${t.eigenvalues[0]}+${t.eigenvalues[1]}]]" # horizontal concat à la Matlab

Output

  -------------:----------------------------------------
			  ...
              w: ${t.eigenvalues[0]}  [...]  ${t.eigenvalues[1]}
               = 26.0
              x: $[[0,${t.eigenvalues [...] {t.eigenvalues[1]}]]
               = [0 26] (double)
    -------------:----------------------------------------           

Evaluating safely expressions with context

Creating a param Instance

Create a param instance named context, initializing it with a NumPy array f:

from pizza.private.mstruct import param
import numpy as np
# Create a Param instance with a NumPy array
context = param(
    f = np.array([
        [1, 2, 3, 4],
        [5, 6, 7, 8],
        [9, 10, 11, 12],
        [13, 14, 15, 16]
    ])
)

Defining Variables and Expressions

Define additional variables a and b, and create a list of expressions using ${...} placeholders to reference these variables:

# Define additional variables 'a' and 'b' in the context
context.update(
    a =[1.0, 0.2, 0.03, 0.004],
    b = np.array([[1, 0.2, 0.03, 0.004]])
)

# Example expressions using ${...} placeholders
expressions = [
    "${a[1]}",                # Should return 0.2
    "${b[0,1]} + ${a[0]}",    # Should return 1.2
    "${f[0:2,1]}"              # Should return the second column of 'f' as [2, 6]
]

** Evaluating Expressions in Placeholders ${...}**

Iterate over the list of expressions, evaluating each one using the safe_fstring() method and printing the results:

# Evaluate each expression ${} and print the results
for expr in expressions:
    result = param.safe_fstring(expr, context)
    print(f"Expression (1): {expr} => Result: {result}")

Expected Output:

# (1): Substitutions are applied only to placeholders
# note that placeholders cannot be nested
Expression (1): ${a[1]} => Result: 0.2
Expression (1): ${b[0,1]} + ${a[0]} => Result: 0.2 + 1.0
Expression (1): ${f[0:2,1]} => Result: [2, 6]

Generalization to Full Evaluation

The previous expressions can be fully evaluated with formateval() method

# (2): Using formateval()
for expr in expressions:
    result = context.formateval(expr)
    print(f"Expression (2): {expr} => Result: {result}")

Expected Output:

# (2): Using full evaluation
Expression (2): ${a[1]} => Result: 0.2
Expression (2): ${b[0,1]} + ${a[0]} => Result: 1.2
Expression (2): ${f[0:2,1]} => Result: [2, 6]

Explanation:

1. ${a[1]}:

   • Evaluation: Accesses the second element of list a, which is 0.2.
   • Result: 0.2

2. ${b[0,1]} + ${a[0]}:

   • Evaluation: Adds b[0,1] (0.2) to a[0] (1.0).
   • Result: 1.2

3. ${f[0:2,1]}:

   • Evaluation: Slices array f to extract elements from rows 0 to 1 (inclusive) and column 1, resulting in [2, 6].
   • Result: [2 6]

---

Implementation Details

The param.eval() method drives the evaluation process, supported by the SafeEvaluator class for secure expression parsing. The replace_matrix_shorthand method ensures proper conversion of shorthands to valid NumPy syntax.

---

Important Considerations

1. Dynamic Context:

   • Variables and expressions can depend on each other and are evaluated dynamically.

2. Error Handling:

   • Robust error detection for undefined variables and invalid operations.

3. Precision:

   • Numerical precision may vary when embedding values in text strings.

4. Security:

   • Eliminates the need for eval, ensuring safe execution of expressions.

---

Conclusion

The enhanced param class offers a robust, secure, and intuitive system for managing mathematical expressions and NumPy operations. By supporting shorthand syntax, advanced operations, and a comprehensive function set, it is a powerful tool for dynamic computations in Python.

---

$2024-01-20$