Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

String encoding for return values #185

Merged
merged 10 commits into from
Nov 25, 2024
Merged

String encoding for return values #185

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
7a25875
String encoding for return values
virgil-serbanuta Nov 7, 2024
bd3767d
cleanup
virgil-serbanuta Nov 8, 2024
28bf950
cleanup
virgil-serbanuta Nov 8, 2024
72dc6b6
tmp
virgil-serbanuta Nov 11, 2024
f14a42c
Fix tests
virgil-serbanuta Nov 11, 2024
384aef2
Remove unneeded identifier
virgil-serbanuta Nov 12, 2024
8254fa2
Renamee encodeStatements to codegenValuesEncoder
virgil-serbanuta Nov 20, 2024
5f4b61d
Rename data to log_buffer
virgil-serbanuta Nov 20, 2024
65ac93e
More documentation
virgil-serbanuta Nov 20, 2024
d40e15d
Remove spaces
virgil-serbanuta Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions rust-semantics/error.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ module RUST-ERROR-SYNTAX
syntax TypePathOrError ::= injectOrError(TypePathSegmentsOrError) [function, total]
syntax TypePathSegmentsOrError ::= concat(TypePathSegment, TypePathSegmentsOrError) [function, total]
syntax ExpressionOrError ::= andOrError(ExpressionOrError, ExpressionOrError) [function, total]
syntax ExpressionOrError ::= addOrError(ExpressionOrError, ExpressionOrError) [function, total]
syntax ValueOrError ::= tupleOrError(ValueListOrError) [function, total]
endmodule

Expand Down Expand Up @@ -52,6 +53,10 @@ module RUST-ERROR
rule andOrError(e(E:SemanticsError), v(_:Expression)) => e(E)
rule andOrError(v(E1:Expression), v(E2:Expression)) => v(E1 && E2)

rule addOrError(_:ExpressionOrError, e(E:SemanticsError)) => e(E)
rule addOrError(e(E:SemanticsError), v(_:Expression)) => e(E)
rule addOrError(v(E1:Expression), v(E2:Expression)) => v(E1 + E2)

rule tupleOrError(L:ValueList) => tuple(L)
rule tupleOrError(E:SemanticsError) => E
endmodule
Expand Down
3 changes: 3 additions & 0 deletions tests/ulm-contracts/bytes_hooks.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,16 @@ extern "C" {
fn append_bool(bytes_id: u64, value: bool) -> u64;
fn append_str(bytes_id: u64, value: &str) -> u64;

fn append_bytes_raw(bytes_id: u64, to_append: u64) -> u64;

fn decode_u256(bytes_id: u64) -> (u64, u256);
fn decode_u160(bytes_id: u64) -> (u64, u160);
fn decode_u128(bytes_id: u64) -> (u64, u128);
fn decode_u64(bytes_id: u64) -> (u64, u64);
fn decode_u32(bytes_id: u64) -> (u64, u32);
fn decode_u16(bytes_id: u64) -> (u64, u16);
fn decode_u8(bytes_id: u64) -> (u64, u8);
fn decode_str(bytes_id: u64) -> (u64, str);

fn decode_signature(bytes_id: u64) -> (u64, u64);
fn hash(bytes_id: u64) -> u256;
Expand Down
12 changes: 12 additions & 0 deletions tests/ulm-contracts/test_helpers.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,15 @@ fn decode_single_u256(bytes_id: u64) -> u256 {
};
value
}

fn decode_single_str(bytes_id: u64) -> str {
let (remaining_id, prefix_size) = :: bytes_hooks :: decode_u256(bytes_id);
if prefix_size != 32_u256 {
fail();
};
Comment on lines +20 to +22
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this prefix is supposed to be an integer equal to the length of the string. If so, is that a property worth validating?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more complicated. See this for more details: https://docs.soliditylang.org/en/latest/abi-spec.html but, as I understand it, the summary is something like this:

First, values are encoded in two bytes objects (called the "prefix" and the "suffix"), which are concatenated at the end to produce the full encoding. Fixed length values (such as int256) are encoded fully in the prefix (their suffix part is empty). Variable length values (such as strings) have both prefix and suffix parts. The prefix part is equal to the-final-length-of-the-prefix + the-length-of-the-suffix-before-the-current-value (I assume that this makes it easy to find the start of the suffix part of the value). The suffix part contains the length of the value followed by the value bytes.

So, prefix_size here is the prefix encoding of the string, which, for a single encoded string, should be the total length of the prefix. It can only be 32, i.e. the number of bytes needed for any fixed size int with at most 256 bytes.

The :: bytes_hooks :: decode_str call below will receive the suffix, and it will read the length, then it will extract that many bytes, and so on.

let (remaining_id, value) = :: bytes_hooks :: decode_str(remaining_id);
if :: bytes_hooks :: length(remaining_id) > 0_u32 {
fail();
};
value
}
6 changes: 3 additions & 3 deletions tests/ulm-no-contract/test_bytes_hooks.appendstr.run
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ call :: test_bytes_hooks :: append_str;
return_value_to_arg;
call :: bytes_hooks :: length;
return_value;
check_eq 7_u32;
check_eq 64_u32;

push "Everyone";
push "Everyone12345678901234567890123456789012";
call :: test_bytes_hooks :: append_str;
return_value_to_arg;
call :: bytes_hooks :: length;
return_value;
check_eq 10_u32
check_eq 96_u32
44 changes: 22 additions & 22 deletions tests/ulm-with-contract/erc_20_token.1.run
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
list_mock GetAccountStorageHook ( 11059124170230400219182520145867018696635045254189466511817698468614930762963 ) ulmIntResult(0, u256);
list_mock SetAccountStorageHook ( 11059124170230400219182520145867018696635045254189466511817698468614930762963 , 10000 ) ulmNoResult();
list_mock GetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 ) ulmIntResult(0, u256);
list_mock SetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 , 10000 ) ulmNoResult();
list_mock GetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 ) ulmIntResult(10000, u256);
list_mock GetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 ) ulmIntResult(10000, u256);
list_mock GetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 ) ulmIntResult(10000, u256);
list_mock SetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 , 9900 ) ulmNoResult();
list_mock GetAccountStorageHook ( 17171626450567201640701347902808840427582371480602455275836469707331258301780 ) ulmIntResult(0, u256);
list_mock SetAccountStorageHook ( 17171626450567201640701347902808840427582371480602455275836469707331258301780 , 100 ) ulmNoResult();
list_mock GetAccountStorageHook ( 13032332009337290780939164280742955473285243463246449969298152790977293574652 ) ulmIntResult(0, u256);
list_mock SetAccountStorageHook ( 13032332009337290780939164280742955473285243463246449969298152790977293574652 , 10000 ) ulmNoResult();
list_mock GetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 ) ulmIntResult(0, u256);
list_mock SetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 , 10000 ) ulmNoResult();
list_mock GetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 ) ulmIntResult(10000, u256);
list_mock GetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 ) ulmIntResult(10000, u256);
list_mock GetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 ) ulmIntResult(10000, u256);
list_mock SetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 , 9900 ) ulmNoResult();
list_mock GetAccountStorageHook ( 107935355587284645605585039702009686485686559005730386557515578263121021727641 ) ulmIntResult(0, u256);
list_mock SetAccountStorageHook ( 107935355587284645605585039702009686485686559005730386557515578263121021727641 , 100 ) ulmNoResult();
list_mock Log3Hook ( 100389287136786176327247604509743168900146139575972864366142685224231313322991 , 1010101 , 2020202 , b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00d" ) ulmNoResult();
list_mock GetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 ) ulmIntResult(9900, u256);
list_mock GetAccountStorageHook ( 17171626450567201640701347902808840427582371480602455275836469707331258301780 ) ulmIntResult(100, u256);
list_mock SetAccountStorageHook ( 97321503972240892886219376522881926110074550168465622121824657360422868161783 , 200 ) ulmNoResult();
list_mock GetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 ) ulmIntResult(9900, u256);
list_mock GetAccountStorageHook ( 107935355587284645605585039702009686485686559005730386557515578263121021727641 ) ulmIntResult(100, u256);
list_mock SetAccountStorageHook ( 6299478510657022222390882510197411465571447485061513230872540120223652878023 , 200 ) ulmNoResult();
list_mock Log3Hook ( 63486140976153616755203102783360879283472101686154884697241723088393386309925 , 1010101 , 3030303 , b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xc8" ) ulmNoResult();
list_mock GetAccountStorageHook ( 97321503972240892886219376522881926110074550168465622121824657360422868161783 ) ulmIntResult(200, u256);
list_mock SetAccountStorageHook ( 97321503972240892886219376522881926110074550168465622121824657360422868161783 , 0 ) ulmNoResult();
list_mock GetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 ) ulmIntResult(9900, u256);
list_mock GetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 ) ulmIntResult(9900, u256);
list_mock SetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 , 9700 ) ulmNoResult();
list_mock GetAccountStorageHook ( 17171626450567201640701347902808840427582371480602455275836469707331258301780 ) ulmIntResult(100, u256);
list_mock SetAccountStorageHook ( 17171626450567201640701347902808840427582371480602455275836469707331258301780 , 300 ) ulmNoResult();
list_mock GetAccountStorageHook ( 6299478510657022222390882510197411465571447485061513230872540120223652878023 ) ulmIntResult(200, u256);
list_mock SetAccountStorageHook ( 6299478510657022222390882510197411465571447485061513230872540120223652878023 , 0 ) ulmNoResult();
list_mock GetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 ) ulmIntResult(9900, u256);
list_mock GetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 ) ulmIntResult(9900, u256);
list_mock SetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 , 9700 ) ulmNoResult();
list_mock GetAccountStorageHook ( 107935355587284645605585039702009686485686559005730386557515578263121021727641 ) ulmIntResult(100, u256);
list_mock SetAccountStorageHook ( 107935355587284645605585039702009686485686559005730386557515578263121021727641 , 300 ) ulmNoResult();
list_mock Log3Hook ( 100389287136786176327247604509743168900146139575972864366142685224231313322991 , 1010101 , 2020202 , b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xc8" ) ulmNoResult();
list_mock GetAccountStorageHook ( 115494845046723600673943990597816811669707739681772931244236289759170204726560 ) ulmIntResult(9700, u256);
list_mock GetAccountStorageHook ( 17171626450567201640701347902808840427582371480602455275836469707331258301780 ) ulmIntResult(300, u256);
list_mock GetAccountStorageHook ( 61159765391625381932700807532824927387505868579121206115135228514320059425645 ) ulmIntResult(9700, u256);
list_mock GetAccountStorageHook ( 107935355587284645605585039702009686485686559005730386557515578263121021727641 ) ulmIntResult(300, u256);

push "uint256";
hold_list_values_from_test_stack;
Expand Down
4 changes: 2 additions & 2 deletions tests/ulm-with-contract/storage.256.run
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
mock SetAccountStorageHook ( 47586510880955265726688953653028435069486243252926274554489699639913455891751 , 1000000000000000000000000000000000000000000000000000000000000 ) ulmNoResult();
mock GetAccountStorageHook ( 47586510880955265726688953653028435069486243252926274554489699639913455891751 ) ulmIntResult(1000000000000000000000000000000000000000000000000000000000000, u256);
mock SetAccountStorageHook ( 81899343944223093934060911023235694134954301913235101281428181165721203435003 , 1000000000000000000000000000000000000000000000000000000000000 ) ulmNoResult();
mock GetAccountStorageHook ( 81899343944223093934060911023235694134954301913235101281428181165721203435003 ) ulmIntResult(1000000000000000000000000000000000000000000000000000000000000, u256);

push "setMyData256";
hold_string_from_test_stack;
Expand Down
4 changes: 2 additions & 2 deletions tests/ulm-with-contract/storage.key.run
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
mock SetAccountStorageHook ( 10244917065751466042072482011607924022453580028001931068470547844249081769915 , 123 ) ulmNoResult();
mock GetAccountStorageHook ( 10244917065751466042072482011607924022453580028001931068470547844249081769915 ) ulmIntResult(123, u256);
mock SetAccountStorageHook ( 89307555350153532481150451243271458944170707741354029868021544305102042370102 , 123 ) ulmNoResult();
mock GetAccountStorageHook ( 89307555350153532481150451243271458944170707741354029868021544305102042370102 ) ulmIntResult(123, u256);

push "setMyDataKey";
hold_string_from_test_stack;
Expand Down
4 changes: 2 additions & 2 deletions tests/ulm-with-contract/storage.simple.run
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
mock SetAccountStorageHook ( 73866358942311723187445361469509963283744060252826659833950887638461707973283 , 123 ) ulmNoResult();
mock GetAccountStorageHook ( 73866358942311723187445361469509963283744060252826659833950887638461707973283 ) ulmIntResult(123, u256);
mock SetAccountStorageHook ( 2566766180763832229052383091904627608042490431692515280991237530682175269032 , 123 ) ulmNoResult();
mock GetAccountStorageHook ( 2566766180763832229052383091904627608042490431692515280991237530682175269032 ) ulmIntResult(123, u256);

push "setMyData";
hold_string_from_test_stack;
Expand Down
15 changes: 15 additions & 0 deletions tests/ulm-with-contract/types.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#![no_std]

#[allow(unused_imports)]
use ulm::*;

#[ulm::contract]
pub trait Types {
#[init]
fn init(&self) {}

#[endpoint(strEndpoint)]
fn str_endpoint(&self) -> str {
"Hello"
}
}
21 changes: 21 additions & 0 deletions tests/ulm-with-contract/types.str-raw.run
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
push "strEndpoint";
hold_string_from_test_stack;
hold_list_values_from_test_stack;
hold_list_values_from_test_stack;
encode_call_data;

return_value;
mock CallData;

call_contract 12345;
return_value;
check_eq ();

push_status;
check_eq 2;

check_raw_output
b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x20" +
b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05" +
b"Hello\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
Comment on lines +18 to +20
Copy link

@sskeirik sskeirik Nov 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my understanding of how encoded values work, this seems like it has an extra value in it?

I expect that the endpoint returns a string, where a str is encoded as follows:

| 32: str.len() | 32x: str payload |

where:

  1. the notation | n: payload | describes a byte field of length n with the given payload
  2. x is chosen to be the smallest value of x such that 32x > str.len().

I don't understand where the initial value x20 is coming from.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the answer here: #185 (comment) , but a single string is encoded like this (when part of a request, response, log entry, and probably other stuff):

| 32: prefix-length | 32: str.len() | 32x: str payload |
|-------prefix------|-------------suffix---------------|


19 changes: 19 additions & 0 deletions tests/ulm-with-contract/types.str.run
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
push "strEndpoint";
hold_string_from_test_stack;
hold_list_values_from_test_stack;
hold_list_values_from_test_stack;
encode_call_data;

return_value;
mock CallData;

call_contract 12345;
return_value;
check_eq ();

push_status;
check_eq 2;
output_to_arg;
call :: test_helpers :: decode_single_str;
return_value;
check_eq "Hello"
Loading