Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Code Generator] Simplify Expressions #318

Merged
merged 49 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
f980f0e
Create draft PR for #25
rihi Aug 17, 2023
6414d67
Implement substitute_visitor
rihi Aug 23, 2023
a3e060e
Replace expression_simplification with expression_simplification_rules
rihi Aug 23, 2023
58ef1af
Implement expression simplification rules
rihi Aug 23, 2023
49fe49d
Add documentation to visit_mem_phi in substitute_visitor.py
rihi Sep 6, 2023
db813cf
Add docstring to modification.py/normalize_int(int,int,bool)
rihi Sep 6, 2023
509298e
Rename modification.py/FOLDABLE_CONSTANTS to FOLDABLE_OPERATIONS
rihi Sep 6, 2023
d577ad9
Rename modification.py/_FOLD_HANDLER to _OPERATION_TO_FOLD_FUNCTION
rihi Sep 6, 2023
7f8f1d1
Move _simplify_instructions and _simplify_instructions_with_rule_set …
rihi Sep 6, 2023
e8992dc
Extract code from _simplify_instructions_with_rule_set to _simplify_i…
rihi Sep 6, 2023
69060a9
Improve documentation in default.json for expression simplification m…
rihi Sep 6, 2023
3111578
Add documentation to substitute_visitor.py
rihi Sep 6, 2023
4e3cd71
Add additional test cases to test_substitute_visitor.py
rihi Sep 6, 2023
a9bbe64
Improve readability of _visit_operation(self,Operation) in substitute…
rihi Sep 6, 2023
fff6964
Improve readability of visit_unary_operation(self,UnaryOperation) in …
rihi Sep 6, 2023
54dcae2
Use visit instead of mapper function in visit_phi(self,Phi)
rihi Sep 6, 2023
f26128b
Lift constraints of SubstituteVisitor that tried to uphold validity o…
rihi Sep 6, 2023
f3f012a
Slight syntax changes to test_substitute_visitor.py
rihi Sep 6, 2023
60dbc26
Improve syntax in _OPERATION_TO_FOLD_FUNCTION
rihi Sep 6, 2023
b544b83
Assert that instructions are not substituted in expression_simplifica…
rihi Sep 6, 2023
28cf1d1
Add clarifying comment to _simplify_instruction_with_rule
rihi Sep 6, 2023
b4670cc
Rename ExpressionSimplificationRules to ExpressionSimplification
rihi Sep 6, 2023
3791dc1
Create copies while substituting with SubstituteVisitor.equality
rihi Sep 6, 2023
624066d
Remove unused function multiply_int_with_constant
rihi Sep 6, 2023
ce7da33
Add documentation to modification.py
rihi Sep 6, 2023
5c62662
Rename modification.py to constant_folding.py
rihi Sep 6, 2023
b0e9920
Extract method from visit_unary_operation
rihi Sep 6, 2023
ef561d1
Add documentation to collect_terms.py
rihi Sep 7, 2023
807679c
Add tests for ExpressionSimplification stage
rihi Sep 7, 2023
d2917b7
Update documentation for '_constant_fold_arithmetic_binary'
rihi Sep 20, 2023
892c4fc
Update documentation for 'normalize_int'
rihi Sep 20, 2023
1600341
Add comment explaining 'max_iterations'
rihi Sep 20, 2023
36215f0
Improve documentation of CollectTerms
rihi Sep 20, 2023
8ddd6a1
Rename variable in collect_terms.py for readability
rihi Sep 20, 2023
4f318ec
Rename variable in simplify_redundant_reference.py for readability
rihi Sep 20, 2023
1c06528
Improve documentation of TermOrder
rihi Sep 20, 2023
4efeebb
Remove unused collapse_mult_neg_one.py
rihi Sep 20, 2023
3146331
Fix test in test_substitute_visitor.py
rihi Sep 20, 2023
df201b9
Rename simplification rule FixAddSubSign to PositiveConstants
rihi Sep 20, 2023
a1a7a1e
Rename variables in positive_constants.py
rihi Sep 20, 2023
182913a
Rename operands to constant in collect_terms.py
rihi Sep 20, 2023
9207491
Rename CollectTerms to CollapseNestedConstants
rihi Sep 20, 2023
97ed807
Add missing unsigned multiply case to simplify_trivial_arithmetic.py
rihi Sep 20, 2023
b62a423
Use 'neg' operation in SubToAdd
rihi Sep 20, 2023
3679a36
Delete test_collapse_mult_neg_one.py
rihi Sep 20, 2023
4a5392d
Fix test_stage.py
rihi Sep 20, 2023
61523ce
Slight change to test_stage.py
rihi Sep 20, 2023
5f22eb1
Ignore MemPhi in SubstituteVisitor
blattm Sep 21, 2023
9e1123d
Merge branch 'main' into issue-25-_Code_Generator_Simplify_Expressions
blattm Sep 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion decompiler/pipeline/controlflowanalysis/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from .expression_simplification import ExpressionSimplification
from .expression_simplification_rules import ExpressionSimplificationRulesAst, ExpressionSimplificationRulesCfg
from .instruction_length_handler import InstructionLengthHandler
from .readability_based_refinement import ReadabilityBasedRefinement
from .variable_name_generation import VariableNameGeneration
142 changes: 0 additions & 142 deletions decompiler/pipeline/controlflowanalysis/expression_simplification.py

This file was deleted.

rihi marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
from typing import Callable, Optional

from decompiler.structures.pseudo import BinaryOperation, Constant, Expression, Integer, OperationType


def multiply_int_with_constant(expression: Expression, constant: Constant) -> Expression:
rihi marked this conversation as resolved.
Show resolved Hide resolved
"""
Multiply an integer expression with an integer constant.

:param expression: The integer expression to be multiplied.
:param constant: The constant value to multiply the expression by.
:return: A simplified expression representing the multiplication result.
"""

if not isinstance(expression.type, Integer):
raise ValueError(f"Expression must have integer type, got {expression.type}.")
if not isinstance(constant.type, Integer):
raise ValueError(f"Constant must have integer type, got {constant.type}.")
if expression.type != constant.type:
raise ValueError(f"Expression and constant type must equal. {expression.type} != {constant.type}")

if isinstance(expression, Constant):
return constant_fold(OperationType.multiply, [expression, constant])
else:
return BinaryOperation(OperationType.multiply, [expression, constant])


_FOLD_HANDLER: dict[OperationType, Callable[[list[Constant]], Constant]] = {
rihi marked this conversation as resolved.
Show resolved Hide resolved
OperationType.minus: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x - y),
OperationType.plus: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x + y),
OperationType.multiply: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x * y, True),
OperationType.multiply_us: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x * y, False),
OperationType.divide: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x // y, True),
OperationType.divide_us: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x // y, False),
OperationType.negate: lambda constants: _constant_fold_arithmetic_unary(constants, lambda value: -value),
OperationType.left_shift: lambda constants: _constant_fold_shift(constants, lambda value, shift, size: value << shift),
OperationType.right_shift: lambda constants: _constant_fold_shift(constants, lambda value, shift, size: value >> shift),
OperationType.right_shift_us: lambda constants: _constant_fold_shift(
constants, lambda value, shift, size: normalize_int(value >> shift, size - shift, False)
),
OperationType.bitwise_or: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x | y),
OperationType.bitwise_and: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x & y),
OperationType.bitwise_xor: lambda constants: _constant_fold_arithmetic_binary(constants, lambda x, y: x ^ y),
OperationType.bitwise_not: lambda constants: _constant_fold_arithmetic_unary(constants, lambda x: ~x),
rihi marked this conversation as resolved.
Show resolved Hide resolved
}


FOLDABLE_CONSTANTS = _FOLD_HANDLER.keys()
rihi marked this conversation as resolved.
Show resolved Hide resolved


def constant_fold(operation: OperationType, constants: list[Constant]) -> Constant:
"""
Fold operation with constants as operands.

:param operation: The operation.
:param constants: All constant operands of the operation.
:return: A constant representing the result of the operation.
"""

if operation not in _FOLD_HANDLER:
raise ValueError(f"Constant folding not implemented for operation '{operation}'.")

return _FOLD_HANDLER[operation](constants)


def _constant_fold_arithmetic_binary(
rihi marked this conversation as resolved.
Show resolved Hide resolved
constants: list[Constant],
fun: Callable[[int, int], int],
norm_sign: Optional[bool] = None
) -> Constant:
if len(constants) != 2:
raise ValueError(f"Expected exactly 2 constants to fold, got {len(constants)}.")
if not all(constant.type == constants[0].type for constant in constants):
raise ValueError(f"Can not fold constants with different types: {(constant.type for constant in constants)}")
if not all(isinstance(constant.type, Integer) for constant in constants):
raise ValueError(f"All constants must be integers, got {list(constant.type for constant in constants)}.")

left, right = constants

left_value = left.value
right_value = right.value
if norm_sign is not None:
left_value = normalize_int(left_value, left.type.size, norm_sign)
right_value = normalize_int(right_value, right.type.size, norm_sign)

return Constant(
normalize_int(fun(left_value, right_value), left.type.size, left.type.signed),
left.type
)


def _constant_fold_arithmetic_unary(constants: list[Constant], fun: Callable[[int], int]) -> Constant:
if len(constants) != 1:
raise ValueError("Expected exactly 1 constant to fold")
if not isinstance(constants[0].type, Integer):
raise ValueError(f"Constant must be of type integer: {constants[0].type}")

return Constant(normalize_int(fun(constants[0].value), constants[0].type.size, constants[0].type.signed), constants[0].type)


def _constant_fold_shift(constants: list[Constant], fun: Callable[[int, int, int], int]) -> Constant:
if len(constants) != 2:
raise ValueError("Expected exactly 2 constants to fold")
if not all(isinstance(constant.type, Integer) for constant in constants):
raise ValueError("All constants must be integers")

left, right = constants

return Constant(normalize_int(fun(left.value, right.value, left.type.size), left.type.size, left.type.signed), left.type)


def normalize_int(v: int, size: int, signed: bool) -> int:
rihi marked this conversation as resolved.
Show resolved Hide resolved
value = v & ((1 << size) - 1)
if signed and value & (1 << (size - 1)):
return value - (1 << size)
else:
return value
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
from decompiler.pipeline.controlflowanalysis.expression_simplification.rules.rule import SimplificationRule
from decompiler.structures.pseudo import BinaryOperation, Expression, Operation, OperationType, UnaryOperation


class CollapseAddNeg(SimplificationRule):
"""
Simplifies additions/subtraction with negated expression.

- `e0 + -(e1) -> e0 - e1`
- `e0 - -(e1) -> e0 + e1`
"""

def apply(self, operation: Operation) -> list[tuple[Expression, Expression]]:
if operation.operation not in [OperationType.plus, OperationType.minus]:
return []
if not isinstance(operation, BinaryOperation):
raise TypeError(f"Expected BinaryOperation, got {type(operation)}")

right = operation.right
if not isinstance(right, UnaryOperation) or right.operation != OperationType.negate:
return []

return [(
operation,
BinaryOperation(
OperationType.minus if operation.operation == OperationType.plus else OperationType.plus,
[operation.left, right.operand],
operation.type
)
)]
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from decompiler.pipeline.controlflowanalysis.expression_simplification.modification import FOLDABLE_CONSTANTS, constant_fold
from decompiler.pipeline.controlflowanalysis.expression_simplification.rules.rule import SimplificationRule
from decompiler.structures.pseudo import Constant, Expression, Operation


class CollapseConstants(SimplificationRule):
"""
Fold operations with only constants as operands:
"""

def apply(self, operation: Operation) -> list[tuple[Expression, Expression]]:
if not all(isinstance(o, Constant) for o in operation.operands):
return []
if operation.operation not in FOLDABLE_CONSTANTS:
return []

return [(
operation,
constant_fold(operation.operation, operation.operands)
)]
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
from decompiler.pipeline.controlflowanalysis.expression_simplification.rules.rule import SimplificationRule
from decompiler.structures.pseudo import BinaryOperation, Constant, Expression, Operation, OperationType, UnaryOperation


class CollapseMultNegOne(SimplificationRule):
rihi marked this conversation as resolved.
Show resolved Hide resolved
"""
Simplifies expressions multiplied with -1.

`e0 * -1 -> -(e0)`
"""

def apply(self, operation: Operation) -> list[tuple[Expression, Expression]]:
if operation.operation != OperationType.multiply:
return []
if not isinstance(operation, BinaryOperation):
raise TypeError(f"Expected BinaryOperation, got {type(operation)}")

right = operation.right
if not isinstance(right, Constant) or right.value != -1:
return []

return [(
operation,
UnaryOperation(
OperationType.negate,
[operation.left],
operation.type
)
)]
Loading