From c8c77dcb50d86843fbb318b1ce852ffa3d96e56f Mon Sep 17 00:00:00 2001 From: Joe Date: Tue, 21 Nov 2023 10:32:51 +0000 Subject: [PATCH 01/11] simd: enable core bpf programs --- proposals/0088-enable-core-bpf-programs.md | 143 +++++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100644 proposals/0088-enable-core-bpf-programs.md diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md new file mode 100644 index 000000000..f177508c5 --- /dev/null +++ b/proposals/0088-enable-core-bpf-programs.md @@ -0,0 +1,143 @@ +--- +simd: '0088' +title: Enable Core BPF Programs +authors: + - Joe Caulfield +category: Standard +type: Core +status: Draft +created: 2023-11-07 +feature: (fill in with feature tracking issues once accepted) +--- + +## Summary + +This proposal introduces the concept of Core BPF programs: programs which are +essential to network operations. Currently, these are embedded programs known as +"native" programs. + +This SIMD also outlines the processes by which engineers will manage upgrades to +these enshrined Core BPF programs, as well as migrate existing native programs +to core BPF. + +## Motivation + +With the advent of Program Runtime v2 and Firedancer, native programs will +eventually become BPF programs. + +Program Runtime v2 will introduce a brand-new ABI for on-chain programs, and the +native programs will need to be revised to support it. + +BPF programs offer less complexity for other clients, such as Firedancer, since +developers will no longer have to keep up with program changes in their runtime +implementations. Instead, the program can just be modified once. + +For these reasons, it makes sense to introduce the concept of Core BPF programs: +BPF programs the network depends on that should be upgraded with a special +process. + +With an explicit process in place for upgrading Core BPF programs, engineers can +safely manage changes to essential programs through feature gates. The feature +gate process requries incremental activations on each cluster: testnet, then +devnet, then mainnet-beta. This is extremely valuable for ensuring a change has +been safely integrated and will not have negative effects on mainnet-beta. + +This same explicit process can be used to migrate existing native programs to +their new Core BPF versions. + +## Alternatives Considered + +An alternative approach to managing upgrades to essential core programs could be +to make these core BPF programs _upgradeable_ and establish a multi-signature +upgrade authority. This authority could be comprised of key-holders from each +validator client: Solana Labs, Jito, Jump, and possibly Syndica (Sig). + +However, this approach adds additional layers of complexity around upgrading +these programs behind a feature-gate. + +## New Terminology + +- `Core BPF Program`: A **non-upgradeable** BPF program relied on by any part of + the Solana stack, including (but not limited to) consensus, transaction + processing, voting, staking, and account creation. +- `Target Program`: The core program intending to be changed by some + feature activation. +- `Source Program`: The modified version of a core program intending to be moved + in place of the existing core program. + +## Detailed Design + +Core BPF programs shall be non-upgradeable BPF programs deployed with +`BPFLoader2111111111111111111111111111111111`. + +Core BPF programs must only be modified through feature gates. + +**Upgrading a core BPF program** shall consist of deploying the modified program +source code to a new arbitrary address as a **non-upgradeable** BPF program and +creating a feature gate to, using the runtime, move the modified program in +place of the existing program. + +Note, since this deployed program will be deployed using the non-upgradeable BPF +loader (`BPFLoader2111111111111111111111111111111111`), it will consist of only +a program account, with no additional program _data_ account. + +This process must adhere to the following steps: + +1. Verifiably build the modified program. +2. Generate a new keypair for the **source** program. +3. Deploy the program to the **source** address. +4. Generate a new keypair for the **feature gate**. +5. Create a new feature gate for replacing the **target** program with the + **source** (modified) program. +6. Follow the existing process for activating features. + +An additional optional field could be added to feature gate issues for the +**target program** being upgraded. + +**Migrating a native program** to a core BPF program shall consist of following +the exact steps outlined above with the modified BPF version of the program. +This will effectively replace the program at the target address with its core +BPF implementation. + +Some additional checks should be run when doing a migration instead of an +upgrade, such as: + +- Validating the native program's owner is + `NativeLoader1111111111111111111111111111111` +- Ensuring the core BPF program has the correct state + +## Impact + +This proposed change would result in a new process for upgrading core programs. +Although this process will still require feature gates as it does now, the +change is handled completely differently. + +The act of deploying the modified program and using a runtime feature gate to +move it into the proper account will be an entirely new way of upgrading +programs that were previously native. + +Core contributors will have to become comfortable with this process. + +## Security Considerations + +This proposal provides a secure means for upgrading core BPF programs - a +process that will be increasingly valuable as new core BPF programs are created. + +However, this change also introduces a critical security consideration +surrounding the replacement of a core program with the contents of another +account. This is an extremely sensitive process that must be handled with +maximum caution. + +If a core program is modified incorrectly, or somehow erased during migration, +it could have immediate and fatal consequences for the network. + +## Backwards Compatibility + +This feature itself does not directly introduce any breaking changes. + +When this proposed process is used to migrate a native program to core BPF, that +program will be completely backwards compatible. + +However, once a program has been migrated to core BPF, the process by which this +program is upgraded will not be backward compatible. Core contributors must +follow the new process. From 6017a1dea2576f3a94724637c9ec4b6d8b8c7829 Mon Sep 17 00:00:00 2001 From: Joe Date: Tue, 5 Dec 2023 12:25:58 -0600 Subject: [PATCH 02/11] address jon initial feedback --- proposals/0088-enable-core-bpf-programs.md | 82 +++++++++++----------- 1 file changed, 41 insertions(+), 41 deletions(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index f177508c5..629681cc0 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -12,70 +12,72 @@ feature: (fill in with feature tracking issues once accepted) ## Summary -This proposal introduces the concept of Core BPF programs: programs which are +This proposal outlines the process by which engineers will manage upgrades to +enshrined Core BPF programs, as well as migrate existing native programs to core +BPF. + +This SIMD also introduces the concept of Core BPF programs: programs which are essential to network operations. Currently, these are embedded programs known as "native" programs. -This SIMD also outlines the processes by which engineers will manage upgrades to -these enshrined Core BPF programs, as well as migrate existing native programs -to core BPF. - -## Motivation +With an explicit process in place for upgrading Core BPF programs, engineers can +safely manage changes to essential programs through feature gates. The feature +gate process requries incremental activations on each cluster: testnet, then +devnet, then mainnet-beta. This is extremely valuable for ensuring a change has +been safely integrated and will not have negative effects on mainnet-beta. -With the advent of Program Runtime v2 and Firedancer, native programs will -eventually become BPF programs. +This same explicit process can be used to migrate existing native programs to +their new Core BPF versions. -Program Runtime v2 will introduce a brand-new ABI for on-chain programs, and the -native programs will need to be revised to support it. +## Motivation BPF programs offer less complexity for other clients, such as Firedancer, since developers will no longer have to keep up with program changes in their runtime implementations. Instead, the program can just be modified once. -For these reasons, it makes sense to introduce the concept of Core BPF programs: +For this reason, it makes sense to introduce the concept of Core BPF programs: BPF programs the network depends on that should be upgraded with a special process. -With an explicit process in place for upgrading Core BPF programs, engineers can -safely manage changes to essential programs through feature gates. The feature -gate process requries incremental activations on each cluster: testnet, then -devnet, then mainnet-beta. This is extremely valuable for ensuring a change has -been safely integrated and will not have negative effects on mainnet-beta. - -This same explicit process can be used to migrate existing native programs to -their new Core BPF versions. - ## Alternatives Considered -An alternative approach to managing upgrades to essential core programs could be +An alternative approach to managing upgrades to Core BPF programs could be to make these core BPF programs _upgradeable_ and establish a multi-signature upgrade authority. This authority could be comprised of key-holders from each validator client: Solana Labs, Jito, Jump, and possibly Syndica (Sig). -However, this approach adds additional layers of complexity around upgrading -these programs behind a feature-gate. +Gating upgrades behind a feature-gate has numerous benefits, including a long +lead time to deployment and requiring validators to upgrade their software to +signal their acceptance of the change. Allowing Core BPF programs to be upgraded +by a multi-sig without feature-gating strips core contributors of these +benefits. + +It's possible that multi-sig program upgrades could be combined with +feature-gates, however this implementation would add additional layers of +complexity to the upgrade process. ## New Terminology - `Core BPF Program`: A **non-upgradeable** BPF program relied on by any part of the Solana stack, including (but not limited to) consensus, transaction processing, voting, staking, and account creation. -- `Target Program`: The core program intending to be changed by some - feature activation. -- `Source Program`: The modified version of a core program intending to be moved - in place of the existing core program. ## Detailed Design +In the context of this design, **source program** shall be the modified version +of a core BPF program intending to be moved in place of the existing program, +while **target program** is that existing program. + Core BPF programs shall be non-upgradeable BPF programs deployed with `BPFLoader2111111111111111111111111111111111`. Core BPF programs must only be modified through feature gates. **Upgrading a core BPF program** shall consist of deploying the modified program -source code to a new arbitrary address as a **non-upgradeable** BPF program and -creating a feature gate to, using the runtime, move the modified program in -place of the existing program. +to a new arbitrary address as a **non-upgradeable** BPF program and using a +feature gate to move the modified program in place of the existing program. The +feature gate is required to circumvent normal transaction processing rules and +replace the contents of one account with another directly at the runtime level. Note, since this deployed program will be deployed using the non-upgradeable BPF loader (`BPFLoader2111111111111111111111111111111111`), it will consist of only @@ -100,11 +102,8 @@ This will effectively replace the program at the target address with its core BPF implementation. Some additional checks should be run when doing a migration instead of an -upgrade, such as: - -- Validating the native program's owner is - `NativeLoader1111111111111111111111111111111` -- Ensuring the core BPF program has the correct state +upgrade, such as validating the native program's owner is +`NativeLoader1111111111111111111111111111111`. ## Impact @@ -116,8 +115,6 @@ The act of deploying the modified program and using a runtime feature gate to move it into the proper account will be an entirely new way of upgrading programs that were previously native. -Core contributors will have to become comfortable with this process. - ## Security Considerations This proposal provides a secure means for upgrading core BPF programs - a @@ -133,11 +130,14 @@ it could have immediate and fatal consequences for the network. ## Backwards Compatibility -This feature itself does not directly introduce any breaking changes. +This proposal itself does not directly introduce any breaking changes. The code +introduced to migrate native programs or upgrade core BPF programs will exist +off the runtime's "hot path" until it's actually used for a migration/upgrade. -When this proposed process is used to migrate a native program to core BPF, that -program will be completely backwards compatible. +When the mechanism is used _specifically_ to migrate a native program to core +BPF, the core BPF version will be more than backwards compatible. It must +provide the exact same results as the native program it aims to replace. However, once a program has been migrated to core BPF, the process by which this -program is upgraded will not be backward compatible. Core contributors must +program is upgraded will not be backwards compatible. Core contributors must follow the new process. From 02074b82d398637d3834eac0df2346568036e3a4 Mon Sep 17 00:00:00 2001 From: Joe Date: Wed, 13 Dec 2023 09:35:49 -0500 Subject: [PATCH 03/11] add recent sync feedback --- proposals/0088-enable-core-bpf-programs.md | 122 +++++++++------------ 1 file changed, 52 insertions(+), 70 deletions(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index 629681cc0..7444f57fe 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -20,15 +20,6 @@ This SIMD also introduces the concept of Core BPF programs: programs which are essential to network operations. Currently, these are embedded programs known as "native" programs. -With an explicit process in place for upgrading Core BPF programs, engineers can -safely manage changes to essential programs through feature gates. The feature -gate process requries incremental activations on each cluster: testnet, then -devnet, then mainnet-beta. This is extremely valuable for ensuring a change has -been safely integrated and will not have negative effects on mainnet-beta. - -This same explicit process can be used to migrate existing native programs to -their new Core BPF versions. - ## Motivation BPF programs offer less complexity for other clients, such as Firedancer, since @@ -41,16 +32,10 @@ process. ## Alternatives Considered -An alternative approach to managing upgrades to Core BPF programs could be -to make these core BPF programs _upgradeable_ and establish a multi-signature -upgrade authority. This authority could be comprised of key-holders from each -validator client: Solana Labs, Jito, Jump, and possibly Syndica (Sig). - -Gating upgrades behind a feature-gate has numerous benefits, including a long -lead time to deployment and requiring validators to upgrade their software to -signal their acceptance of the change. Allowing Core BPF programs to be upgraded -by a multi-sig without feature-gating strips core contributors of these -benefits. +An alternative approach to managing upgrades to Core BPF programs could be to +manually upgrade these programs behind a feature gate using the runtime. This +process is more complex and involves circumventing the established program +upgrade process. It's possible that multi-sig program upgrades could be combined with feature-gates, however this implementation would add additional layers of @@ -58,32 +43,40 @@ complexity to the upgrade process. ## New Terminology -- `Core BPF Program`: A **non-upgradeable** BPF program relied on by any part of +- `Core BPF Program`: An **upgradeable** BPF program relied on by any part of the Solana stack, including (but not limited to) consensus, transaction processing, voting, staking, and account creation. ## Detailed Design -In the context of this design, **source program** shall be the modified version -of a core BPF program intending to be moved in place of the existing program, -while **target program** is that existing program. +Core BPF programs shall be upgradeable BPF programs deployed with +`BPFLoaderUpgradeab1e11111111111111111111111`. + +The upgrade authority shall be a multi-sig comprised of keyholders from each +validator client. Right now, that list includes Solana Labs, Jito, and Jump. +This list can be updated in the future to include newer clients like Sig. + +**Upgrading a core BPF program** shall consist of a coordinated effort amongst +core contributors from all validator clients. Similar to the feature gate +process, this upgrade should occur on testnet, then devnet, then mainnet-beta. -Core BPF programs shall be non-upgradeable BPF programs deployed with -`BPFLoader2111111111111111111111111111111111`. +Upgraded versions of core BPF programs shall always be compiled using a +verifiable build process. -Core BPF programs must only be modified through feature gates. +**Migrating a native program to core BPF** shall consist of deploying a modifed +version of the native program to a new arbitrary address as an **upgradeable** +BPF program and using a feature gate to move the modified program in place of +the existing program. -**Upgrading a core BPF program** shall consist of deploying the modified program -to a new arbitrary address as a **non-upgradeable** BPF program and using a -feature gate to move the modified program in place of the existing program. The -feature gate is required to circumvent normal transaction processing rules and -replace the contents of one account with another directly at the runtime level. +The feature gate is required to circumvent normal transaction processing rules +and replace the contents of one account with another directly at the runtime +level. -Note, since this deployed program will be deployed using the non-upgradeable BPF -loader (`BPFLoader2111111111111111111111111111111111`), it will consist of only -a program account, with no additional program _data_ account. +In the context of this design, **source program** shall be the modified BPF +version of a native program intending to be moved in place of the existing +native program, while **target program** is that existing program. -This process must adhere to the following steps: +The migration process must adhere to the following steps: 1. Verifiably build the modified program. 2. Generate a new keypair for the **source** program. @@ -93,51 +86,40 @@ This process must adhere to the following steps: **source** (modified) program. 6. Follow the existing process for activating features. -An additional optional field could be added to feature gate issues for the -**target program** being upgraded. - -**Migrating a native program** to a core BPF program shall consist of following -the exact steps outlined above with the modified BPF version of the program. -This will effectively replace the program at the target address with its core -BPF implementation. - -Some additional checks should be run when doing a migration instead of an -upgrade, such as validating the native program's owner is -`NativeLoader1111111111111111111111111111111`. - ## Impact -This proposed change would result in a new process for upgrading core programs. -Although this process will still require feature gates as it does now, the -change is handled completely differently. +With this change, validator clients would no longer be required to implement +changes to essential programs. Instead these programs could be modified just +once. This reduces some engineering overhead on validator teams. -The act of deploying the modified program and using a runtime feature gate to -move it into the proper account will be an entirely new way of upgrading -programs that were previously native. +This also introduces a multi-sig process, which core contributors across all +validator teams must, in a timely manner, participate in. ## Security Considerations -This proposal provides a secure means for upgrading core BPF programs - a -process that will be increasingly valuable as new core BPF programs are created. - -However, this change also introduces a critical security consideration -surrounding the replacement of a core program with the contents of another -account. This is an extremely sensitive process that must be handled with -maximum caution. +This proposal establishes the concept of relying on BPF programs that are not +built into the runtime for essential cluster operations. With these programs +being upgradeable, there are some obvious security considerations around who can +upgrade these programs and when. With a proper multi-sig process in place, these +risks are mitigated. -If a core program is modified incorrectly, or somehow erased during migration, -it could have immediate and fatal consequences for the network. +When it comes to migrating native programs to core BPF, this change introduces a +serious security consideration surrounding the replacement of an essential +program with the contents of another account. This is an extremely sensitive +process that must be handled with maximum caution. If a core program is modified +incorrectly, or somehow erased during migration, it could have immediate and +fatal consequences for the network. ## Backwards Compatibility This proposal itself does not directly introduce any breaking changes. The code -introduced to migrate native programs or upgrade core BPF programs will exist -off the runtime's "hot path" until it's actually used for a migration/upgrade. +introduced to migrate native programs to core BPF programs will exist off of the +runtime's "hot path" until it's actually used for a migration. -When the mechanism is used _specifically_ to migrate a native program to core -BPF, the core BPF version will be more than backwards compatible. It must -provide the exact same results as the native program it aims to replace. +When a migration is conducted, the core BPF version will be more than backwards +compatible. It must provide the exact same results as the native program it aims +to replace. -However, once a program has been migrated to core BPF, the process by which this -program is upgraded will not be backwards compatible. Core contributors must -follow the new process. +However, once a native program has been migrated to core BPF, the process by +which this program is upgraded will not be backwards compatible. Core +contributors must follow the new process. From 8191e4b2a97e59c192f505f077698c7f6f24a741 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 14 Dec 2023 21:03:39 -0500 Subject: [PATCH 04/11] remove upgrade authority details --- proposals/0088-enable-core-bpf-programs.md | 70 ++++++++++------------ 1 file changed, 30 insertions(+), 40 deletions(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index 7444f57fe..18962b55c 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -12,13 +12,13 @@ feature: (fill in with feature tracking issues once accepted) ## Summary -This proposal outlines the process by which engineers will manage upgrades to -enshrined Core BPF programs, as well as migrate existing native programs to core -BPF. +This proposal introduces the concept of Core BPF programs: programs which are +essential to network operations. Currently, these exist as built-in programs +known as "native" programs. -This SIMD also introduces the concept of Core BPF programs: programs which are -essential to network operations. Currently, these are embedded programs known as -"native" programs. +This SIM also details the process by which existing native programs shall be +ported to Core BPF, as well as the process for introducing brand-new Core BPF +programs. ## Motivation @@ -27,46 +27,39 @@ developers will no longer have to keep up with program changes in their runtime implementations. Instead, the program can just be modified once. For this reason, it makes sense to introduce the concept of Core BPF programs: -BPF programs the network depends on that should be upgraded with a special -process. +BPF programs the network depends on that should be treated with special care. ## Alternatives Considered -An alternative approach to managing upgrades to Core BPF programs could be to -manually upgrade these programs behind a feature gate using the runtime. This -process is more complex and involves circumventing the established program -upgrade process. - -It's possible that multi-sig program upgrades could be combined with -feature-gates, however this implementation would add additional layers of -complexity to the upgrade process. +The alternative to Core BPF programs is to keep these essential programs as +native programs. This would mean each validator client implementation would have +to build and maintain these built-in programs with their runtime +implementations, including any future changes to these programs introduced via +SIMDs or other fixes. ## New Terminology -- `Core BPF Program`: An **upgradeable** BPF program relied on by any part of - the Solana stack, including (but not limited to) consensus, transaction - processing, voting, staking, and account creation. +- `Core BPF Program`: A BPF program relied on by any part of the Solana stack, + including (but not limited to) consensus, transaction processing, voting, + staking, and account creation. ## Detailed Design -Core BPF programs shall be upgradeable BPF programs deployed with -`BPFLoaderUpgradeab1e11111111111111111111111`. - -The upgrade authority shall be a multi-sig comprised of keyholders from each -validator client. Right now, that list includes Solana Labs, Jito, and Jump. -This list can be updated in the future to include newer clients like Sig. +Core BPF programs in many ways will be designed no differently than any other +BPF program on Solana. However, some programs may require special privileges, +which is beyond the scope of this SIMD. -**Upgrading a core BPF program** shall consist of a coordinated effort amongst -core contributors from all validator clients. Similar to the feature gate -process, this upgrade should occur on testnet, then devnet, then mainnet-beta. +When an existing native program is being proposed to migrate to Core BPF, or +when a new Core BPF program is being introduced, at least one SIMD shall be +published outlining at least the following details: -Upgraded versions of core BPF programs shall always be compiled using a -verifiable build process. +- How any required special privileges will be granted to the program in its BPF + form +- How this program's upgrades will be managed after it becomes Core BPF **Migrating a native program to core BPF** shall consist of deploying a modifed -version of the native program to a new arbitrary address as an **upgradeable** -BPF program and using a feature gate to move the modified program in place of -the existing program. +version of the native program to a new arbitrary address and using a feature +gate to move the modified program in place of the existing program. The feature gate is required to circumvent normal transaction processing rules and replace the contents of one account with another directly at the runtime @@ -92,16 +85,13 @@ With this change, validator clients would no longer be required to implement changes to essential programs. Instead these programs could be modified just once. This reduces some engineering overhead on validator teams. -This also introduces a multi-sig process, which core contributors across all -validator teams must, in a timely manner, participate in. - ## Security Considerations This proposal establishes the concept of relying on BPF programs that are not -built into the runtime for essential cluster operations. With these programs -being upgradeable, there are some obvious security considerations around who can -upgrade these programs and when. With a proper multi-sig process in place, these -risks are mitigated. +built into the runtime for essential cluster operations. Depending on how these +programs are elected to be upgraded, there are some obvious security +considerations around who can upgrade these programs and when. Any new Core BPF +program should follow a fully-fledged SIMD process addressing these concerns. When it comes to migrating native programs to core BPF, this change introduces a serious security consideration surrounding the replacement of an essential From 8533b54d812eb5d485e88997b92d80e97977aa0d Mon Sep 17 00:00:00 2001 From: Joe Date: Fri, 15 Dec 2023 15:04:03 -0500 Subject: [PATCH 05/11] address tyera review feedback --- proposals/0088-enable-core-bpf-programs.md | 47 +++++++++++----------- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index 18962b55c..871d9868b 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -16,7 +16,7 @@ This proposal introduces the concept of Core BPF programs: programs which are essential to network operations. Currently, these exist as built-in programs known as "native" programs. -This SIM also details the process by which existing native programs shall be +This SIMD also details the process by which existing native programs shall be ported to Core BPF, as well as the process for introducing brand-new Core BPF programs. @@ -24,7 +24,7 @@ programs. BPF programs offer less complexity for other clients, such as Firedancer, since developers will no longer have to keep up with program changes in their runtime -implementations. Instead, the program can just be modified once. +implementations. Instead, the program can just be updated once. For this reason, it makes sense to introduce the concept of Core BPF programs: BPF programs the network depends on that should be treated with special care. @@ -49,40 +49,41 @@ Core BPF programs in many ways will be designed no differently than any other BPF program on Solana. However, some programs may require special privileges, which is beyond the scope of this SIMD. -When an existing native program is being proposed to migrate to Core BPF, or -when a new Core BPF program is being introduced, at least one SIMD shall be +When an existing native program is being proposed to be migrated to Core BPF, +or when a new Core BPF program is being introduced, at least one SIMD shall be published outlining at least the following details: - How any required special privileges will be granted to the program in its BPF form - How this program's upgrades will be managed after it becomes Core BPF -**Migrating a native program to core BPF** shall consist of deploying a modifed -version of the native program to a new arbitrary address and using a feature -gate to move the modified program in place of the existing program. +**Migrating a native program to core BPF** shall consist of deploying a +BPF version of the native program to a new arbitrary address and, using a +feature gate, move the BPF program, replacing the existing native program at +its program address. -The feature gate is required to circumvent normal transaction processing rules -and replace the contents of one account with another directly at the runtime -level. +In the context of this design, **target prorgam** refers to an existing native +program, while **source program** refers to the BPF version to be moved into the +existing program account. -In the context of this design, **source program** shall be the modified BPF -version of a native program intending to be moved in place of the existing -native program, while **target program** is that existing program. +In the context of this design, **source program** shall be the BPF version of +a native program intending to be moved in place of the existing native program, +while **target program** is that existing program. The migration process must adhere to the following steps: -1. Verifiably build the modified program. -2. Generate a new keypair for the **source** program. -3. Deploy the program to the **source** address. -4. Generate a new keypair for the **feature gate**. -5. Create a new feature gate for replacing the **target** program with the - **source** (modified) program. +1. Verifiably build the source BPF program. +2. Generate a new keypair for the source program. +3. Deploy the program to the source program address. +4. Generate a new keypair for the feature gate. +5. Create a new feature gate to replace the target program with the source BPF + program. 6. Follow the existing process for activating features. ## Impact With this change, validator clients would no longer be required to implement -changes to essential programs. Instead these programs could be modified just +changes to essential programs. Instead these programs could be updated just once. This reduces some engineering overhead on validator teams. ## Security Considerations @@ -96,9 +97,9 @@ program should follow a fully-fledged SIMD process addressing these concerns. When it comes to migrating native programs to core BPF, this change introduces a serious security consideration surrounding the replacement of an essential program with the contents of another account. This is an extremely sensitive -process that must be handled with maximum caution. If a core program is modified -incorrectly, or somehow erased during migration, it could have immediate and -fatal consequences for the network. +process that must be handled with maximum caution. If a core program is +reimplemented incorrectly, or somehow erased during migration, it could have +immediate and fatal consequences for the network. ## Backwards Compatibility From be47e900815211a6683b6600f88f0678fe72e69e Mon Sep 17 00:00:00 2001 From: Joe Date: Wed, 3 Jan 2024 10:11:58 -0600 Subject: [PATCH 06/11] drop duplicate paragraph --- proposals/0088-enable-core-bpf-programs.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index 871d9868b..f5bef8251 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -62,14 +62,10 @@ BPF version of the native program to a new arbitrary address and, using a feature gate, move the BPF program, replacing the existing native program at its program address. -In the context of this design, **target prorgam** refers to an existing native +In the context of this design, **target program** refers to an existing native program, while **source program** refers to the BPF version to be moved into the existing program account. -In the context of this design, **source program** shall be the BPF version of -a native program intending to be moved in place of the existing native program, -while **target program** is that existing program. - The migration process must adhere to the following steps: 1. Verifiably build the source BPF program. From 2c13cd01be18d890c1a35b6cd2c513eacadbbb51 Mon Sep 17 00:00:00 2001 From: Joe Date: Wed, 3 Jan 2024 16:02:35 -0600 Subject: [PATCH 07/11] language suggestions --- proposals/0088-enable-core-bpf-programs.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index f5bef8251..91a4e7121 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -16,15 +16,16 @@ This proposal introduces the concept of Core BPF programs: programs which are essential to network operations. Currently, these exist as built-in programs known as "native" programs. -This SIMD also details the process by which existing native programs shall be +This SIMD details the process by which existing native programs can be ported to Core BPF, as well as the process for introducing brand-new Core BPF programs. ## Motivation -BPF programs offer less complexity for other clients, such as Firedancer, since -developers will no longer have to keep up with program changes in their runtime -implementations. Instead, the program can just be updated once. +BPF programs offer less complexity than native programs for other clients, such +as Firedancer, since developers will no longer have to keep up with program +changes in their runtime implementations. Instead, the program can just be +updated once. For this reason, it makes sense to introduce the concept of Core BPF programs: BPF programs the network depends on that should be treated with special care. @@ -58,8 +59,8 @@ published outlining at least the following details: - How this program's upgrades will be managed after it becomes Core BPF **Migrating a native program to core BPF** shall consist of deploying a -BPF version of the native program to a new arbitrary address and, using a -feature gate, move the BPF program, replacing the existing native program at +BPF version of the native program to a new arbitrary address and using a +feature gate to move the BPF program, replacing the existing native program at its program address. In the context of this design, **target program** refers to an existing native From fadff46562796615a6ee39a3a1bbd53c53fbf84a Mon Sep 17 00:00:00 2001 From: Joe Date: Tue, 9 Jan 2024 12:30:00 -0600 Subject: [PATCH 08/11] add more requirements to migration SIMDs --- proposals/0088-enable-core-bpf-programs.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index 91a4e7121..cb42042ea 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -54,9 +54,13 @@ When an existing native program is being proposed to be migrated to Core BPF, or when a new Core BPF program is being introduced, at least one SIMD shall be published outlining at least the following details: +- The interface for the program +- A precise and complete specification of its behavior - How any required special privileges will be granted to the program in its BPF form -- How this program's upgrades will be managed after it becomes Core BPF +- Whether or not this program will be an upgradeable or non-upgradeable BPF + program +- How this changes to this program will be managed after it becomes BPF **Migrating a native program to core BPF** shall consist of deploying a BPF version of the native program to a new arbitrary address and using a From 385401e9bdacb65a80cff80770793d8aff9d4465 Mon Sep 17 00:00:00 2001 From: Joe Date: Tue, 9 Jan 2024 12:33:58 -0600 Subject: [PATCH 09/11] explicitly state program IDs are unchanged --- proposals/0088-enable-core-bpf-programs.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index cb42042ea..d14289825 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -65,7 +65,8 @@ published outlining at least the following details: **Migrating a native program to core BPF** shall consist of deploying a BPF version of the native program to a new arbitrary address and using a feature gate to move the BPF program, replacing the existing native program at -its program address. +its original program address. No program IDs for existing native programs would +be changed by this migration process. In the context of this design, **target program** refers to an existing native program, while **source program** refers to the BPF version to be moved into the From aa23787c513aa1e83b1b96c80a4029fdfe1aaff4 Mon Sep 17 00:00:00 2001 From: Joe Date: Tue, 9 Jan 2024 12:39:38 -0600 Subject: [PATCH 10/11] add mention of resource metrics backwards compatibility --- proposals/0088-enable-core-bpf-programs.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index d14289825..862e98853 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -113,6 +113,16 @@ When a migration is conducted, the core BPF version will be more than backwards compatible. It must provide the exact same results as the native program it aims to replace. -However, once a native program has been migrated to core BPF, the process by -which this program is upgraded will not be backwards compatible. Core -contributors must follow the new process. +When a migration is conducted, the BPF version of a native program will be +absolutely backwards compatible *functionally*. Its BPF version must provide the +exact same results as the original native program it aims to replace. + +However, since BPF programs cannot precisely match the compute meter and other +resource limits of their original native counterparts, some of these metrics may +be slightly different when a native program becomes BPF, thereby affecting +backwards compatibility in that regard. + +Additionally, once a native program has been migrated to core BPF, the process +by which this program is upgraded will not be backwards compatible. Core +contributors must follow the upgrade process outlined in each program's +migration SIMD. From 3824771b37e2b2515d2bc4ee028baf08ecdccd70 Mon Sep 17 00:00:00 2001 From: Joe Date: Tue, 9 Jan 2024 15:06:11 -0600 Subject: [PATCH 11/11] dedupe backward compat paragraph --- proposals/0088-enable-core-bpf-programs.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/proposals/0088-enable-core-bpf-programs.md b/proposals/0088-enable-core-bpf-programs.md index 862e98853..2aebafbf7 100644 --- a/proposals/0088-enable-core-bpf-programs.md +++ b/proposals/0088-enable-core-bpf-programs.md @@ -109,10 +109,6 @@ This proposal itself does not directly introduce any breaking changes. The code introduced to migrate native programs to core BPF programs will exist off of the runtime's "hot path" until it's actually used for a migration. -When a migration is conducted, the core BPF version will be more than backwards -compatible. It must provide the exact same results as the native program it aims -to replace. - When a migration is conducted, the BPF version of a native program will be absolutely backwards compatible *functionally*. Its BPF version must provide the exact same results as the original native program it aims to replace.