Add hyperfridge milestone 3 delivery

[dastansam]

Milestone Delivery Checklist

• The milestone-delivery-template.md has been copied and updated.
• This pull request is being made by the same account as the accepted application.
• I have disclosed any and all sources of reused code in the submitted repositories and have done my due diligence to meet its license requirements.
• In case of acceptance, invoices must be submitted and payments will be transferred to the Polkadot AssetHub and/or fiat account provided in the application.
• The delivery is according to the Guidelines for Milestone Deliverables.

Link to the application pull request: w3f/Grants-Program#2096

cc @wstrametz

[wstrametz]

author check is failing; should be resubmit? thx alot

[PieWol]

Hey @wstrametz , no problem. Thanks to your comment we see that it is indeed you trying to hand in this delivery. No need for changes. Would be nice to see you creating the PR for any future deliverables though.

[PieWol]

Have you already conducted test in production mode? How did that work out?

[wstrametz]

Have you already conducted test in production mode? How did that work out?

Hi @PieWol - yes; we tested with and without CUDA on linux (not macOS metal yet) - here are some findings:

https://github.com/element36-io/hyperfridge-r0/blob/main/docs/runtime.md

I also did test with the API key (Bonsai) but did not measure that. Anyway want to add a productive version using Bonsai (with real bank account) for the last milestone - will add it measurement for bonsai then.

Cheers, Walter

[PieWol]

Hey @wstrametz and @dastansam please check out my current evaluation I have put up for you. Looking forward to your comments. I sadly got stuck while following the testing guide.

[dastansam]

thank you for swift evaluation @PieWol!

I identified the issues and just merged the fix. Looking forward to your feedback!

[dastansam]

thank you for swift evaluation @PieWol!

I identified the issues and just merged the fix. Looking forward to your feedback!

sorry, I missed one more fix. Could you please try from the latest main commit when you evaluate, thanks! @PieWol

[PieWol]

Hey @dastansam , thanks for the fixes, I'm getting further than before. Sadly I'm still runnning into an issue when trying to execute the API call from the testing guide. I have strictly used the values from the screenshot. (Bob transfers 2 tokens to Alice). Looks like for some reason the ebics-service is started in a simluation mode which provides fake statements but the runtimes isn't able to process them. The one I created doesn't even show up. How do I start the ebics service correctly so that only the specific statements I create in the swagger-ui are submitted?

Here is the error log from the node:

2024-05-23 16:56:42 ✨ Imported #166 (0x600f…a967)    
2024-05-23 16:56:42 [OCW] Instantiating offchain worker    
2024-05-23 16:56:42 [OCW] Syncing: true    
2024-05-23 16:56:42 [OCW] Current activity: <wasm:stripped>    
2024-05-23 16:56:42 [OCW] Activity: <wasm:stripped>    
2024-05-23 16:56:42 Requested started id=1348 method=GET uri=http://localhost:8093/ebics/api-v1/bankstatements
2024-05-23 16:56:42 [OCW] JSON received: [{"iban":"CH4308307000289537312","balanceOP":80842.45,"balanceOPCurrency":"CHF","balanceCL":80097.2,"balanceCLCurrency":"CHF","balanceCLDate":"2021-02-28","bookingDate":"2021-02-28","validationDate":"2021-02-28","incomingTransactions":[],"outgoingTransactions":[{"iban":null,"name":null,"addrLine":["VISECA CARD SERVICES SA \nHagenholzstrasse 56 \nPostfach 7007 \n8050 Zuerich","ich"],"currency":"CHF","amount":745.25,"reference":null,"endToEndId":null,"instrId":null,"msgId":null,"pmtInfId":null}]},{"iban":"CH2108307000289537320","balanceOP":0,"balanceOPCurrency":"CHF","balanceCL":110,"balanceCLCurrency":"CHF","balanceCLDate":"2021-03-14","bookingDate":"2021-03-14","validationDate":"2021-03-14","incomingTransactions":[{"iban":"CH4308307000289537312","name":null,"addrLine":["element36 AG \nBahnmatt 25 \n6340 Baar"],"currency":"CHF","amount":100,"reference":"Testanweisung","endToEndId":null,"instrId":null,"msgId":null,"pmtInfId":null}],"outgoingTransactions":[]}]    
2024-05-23 16:56:42 panicked at /Users/Desktop/ocw-ebics/pallets/fiat-ramps/src/utils.rs:25:78:
called `Option::unwrap()` on a `None` value    
2024-05-23 16:56:42 Error running offchain workers at 0x600fac6ef24e18a2e24d5eab1cdc5ceca623e86c215c2674066ba494ee92a967: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed
WASM backtrace:
error while executing at wasm backtrace:
    0: 0x16ea51 - <unknown>!rust_begin_unwind
    1: 0x5e99 - <unknown>!core::panicking::panic_fmt::h3aff855fe938c13f
    2: 0x6510 - <unknown>!core::panicking::panic::h11a2021d92dc1cbb
    3: 0x64c8 - <unknown>!core::option::unwrap_failed::h0e0b231623e0d004
    4: 0xb0bd0 - <unknown>!<fiat_ramps::pallet::Pallet<T> as frame_support::traits::misc::OffchainWorker<<<<T as frame_system::pallet::Config>::Block as sp_runtime::traits::HeaderProvider>::HeaderT as sp_runtime::traits::Header>::Number>>::offchain_worker::h1b7953c444e0366c
    5: 0x4837b - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::offchain_worker::hd04f58194f863336
    6: 0x991b3 - <unknown>!OffchainWorkerApi_offchain_worker

[dastansam]

Hey @dastansam , thanks for the fixes, I'm getting further than before. Sadly I'm still runnning into an issue when trying to execute the API call from the testing guide. I have strictly used the values from the screenshot. (Bob transfers 2 tokens to Alice). Looks like for some reason the ebics-service is started in a simluation mode which provides fake statements but the runtimes isn't able to process them. The one I created doesn't even show up. How do I start the ebics service correctly so that only the specific statements I create in the swagger-ui are submitted?

Here is the error log from the node:

2024-05-23 16:56:42 ✨ Imported #166 (0x600f…a967)    

2024-05-23 16:56:42 [OCW] Instantiating offchain worker    

2024-05-23 16:56:42 [OCW] Syncing: true    

2024-05-23 16:56:42 [OCW] Current activity: <wasm:stripped>    

2024-05-23 16:56:42 [OCW] Activity: <wasm:stripped>    

2024-05-23 16:56:42 Requested started id=1348 method=GET uri=http://localhost:8093/ebics/api-v1/bankstatements

2024-05-23 16:56:42 [OCW] JSON received: [{"iban":"CH4308307000289537312","balanceOP":80842.45,"balanceOPCurrency":"CHF","balanceCL":80097.2,"balanceCLCurrency":"CHF","balanceCLDate":"2021-02-28","bookingDate":"2021-02-28","validationDate":"2021-02-28","incomingTransactions":[],"outgoingTransactions":[{"iban":null,"name":null,"addrLine":["VISECA CARD SERVICES SA \nHagenholzstrasse 56 \nPostfach 7007 \n8050 Zuerich","ich"],"currency":"CHF","amount":745.25,"reference":null,"endToEndId":null,"instrId":null,"msgId":null,"pmtInfId":null}]},{"iban":"CH2108307000289537320","balanceOP":0,"balanceOPCurrency":"CHF","balanceCL":110,"balanceCLCurrency":"CHF","balanceCLDate":"2021-03-14","bookingDate":"2021-03-14","validationDate":"2021-03-14","incomingTransactions":[{"iban":"CH4308307000289537312","name":null,"addrLine":["element36 AG \nBahnmatt 25 \n6340 Baar"],"currency":"CHF","amount":100,"reference":"Testanweisung","endToEndId":null,"instrId":null,"msgId":null,"pmtInfId":null}],"outgoingTransactions":[]}]    

2024-05-23 16:56:42 panicked at /Users/Desktop/ocw-ebics/pallets/fiat-ramps/src/utils.rs:25:78:

called `Option::unwrap()` on a `None` value    

2024-05-23 16:56:42 Error running offchain workers at 0x600fac6ef24e18a2e24d5eab1cdc5ceca623e86c215c2674066ba494ee92a967: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed

WASM backtrace:

error while executing at wasm backtrace:

    0: 0x16ea51 - <unknown>!rust_begin_unwind

    1: 0x5e99 - <unknown>!core::panicking::panic_fmt::h3aff855fe938c13f

    2: 0x6510 - <unknown>!core::panicking::panic::h11a2021d92dc1cbb

    3: 0x64c8 - <unknown>!core::option::unwrap_failed::h0e0b231623e0d004

    4: 0xb0bd0 - <unknown>!<fiat_ramps::pallet::Pallet<T> as frame_support::traits::misc::OffchainWorker<<<<T as frame_system::pallet::Config>::Block as sp_runtime::traits::HeaderProvider>::HeaderT as sp_runtime::traits::Header>::Number>>::offchain_worker::h1b7953c444e0366c

    5: 0x4837b - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::offchain_worker::hd04f58194f863336

    6: 0x991b3 - <unknown>!OffchainWorkerApi_offchain_worker

simple docker compose up is enough to start the ebics service. Did you run it like this? Also, could you try this in hyperfridge branch?

[PieWol]

@dastansam I ran it like this:
docker run -p 8093:8093 e36io/ebics-service

[dastansam]

@dastansam I ran it like this: docker run -p 8093:8093 e36io/ebics-service

yeah, it's an old one. I will add this in the documentation as well. sorry for the inconvenience

[dastansam]

@dastansam I ran it like this: docker run -p 8093:8093 e36io/ebics-service

yeah, the main README seems a bit outdaded. I added the correct link for launching the whole suite of services. This is basically a docker compose up in the hyperfridge branch.

[PieWol]

Got it, thanks @dastansam. I will retry it from scratch now with your updated commands and the latest commits of the hyperfridge branch.

[PieWol]

Hey @dastansam , regarding your fix with the ebics-service readme. Is it correct that as long as I use the docker commands listed in the readme instead of building it myself it should run fine on arm (m1)?

[PieWol]

Hey @dastansam ,
sadly even after retrying it with your latest changes it doesn't work for me to mint the stable coins. Can we please setup a quick call to figure out together what is going wrong? Please schedule a meeting with [email protected]

[dastansam]

Hey @dastansam , regarding your fix with the ebics-service readme. Is it correct that as long as I use the docker commands listed in the readme instead of building it myself it should run fine on arm (m1)?

yes, it should work fine on ARM if you specify platform x86_64 (I also have ARM so, pretty sure).

[dastansam]

Hey @dastansam , sadly even after retrying it with your latest changes it doesn't work for me to mint the stable coins. Can we please setup a quick call to figure out together what is going wrong? Please schedule a meeting with [email protected]

I have sent an email

[PieWol]

Hey @dastansam, thanks for the latest change with the remot API. Now I was able to test everything and gave you some feedback. It worked great 🎉 ! Only question I still have was about the balance storage item within the fiat-ramps pallet. You can find it in the notes. evaluation

Once this is answered the milestone will be approved.

[dastansam]

Hey @dastansam, thanks for the latest change with the remot API. Now I was able to test everything and gave you some feedback. It worked great 🎉 ! Only question I still have was about the balance storage item within the fiat-ramps pallet. You can find it in the notes. evaluation

Once this is answered the milestone will be approved.

hey thanks for the swift evaluation! glad, it's finally working!

as for the balances, currently, that storage field is not used at all. The reason is that we don't need to track the balance, since it is already tracked by System::account() storage and EBICS validates every transaction, i.e if someone has low balance transfer won't be allowed anyway. But I agree that it is confusing, so we can probably remove it in the next milestone.

[github-actions]

🪙 Please fill out the invoice form in order to initiate the payment process. Thank you!

[PieWol]

Thanks again for your latest improvements. Looking forward to the next milestone deliveries! 🎉

[RouvenP]

hi @dastansam we just sent the payment.