From 0ec4c758966c5cf363b1b18e87976f9053919ea2 Mon Sep 17 00:00:00 2001 From: bji Date: Mon, 16 Oct 2023 22:14:02 -0700 Subject: [PATCH 1/5] add: simd-33 feature issue. (#70) --- proposals/0033-timely-vote-credits.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0033-timely-vote-credits.md b/proposals/0033-timely-vote-credits.md index 227590ae2..4e51ed276 100644 --- a/proposals/0033-timely-vote-credits.md +++ b/proposals/0033-timely-vote-credits.md @@ -7,7 +7,7 @@ category: Standard type: Core status: Draft created: 2023-01-30 -feature: (fill in with feature tracking issues once accepted) +feature: https://github.com/solana-labs/solana/issues/32857 --- ## Summary From f34a0902174118e9bb9a0d54c3f296f0ea93d76e Mon Sep 17 00:00:00 2001 From: Anoushk Kharangate <32778608+anoushk1234@users.noreply.github.com> Date: Fri, 20 Oct 2023 21:51:50 +0530 Subject: [PATCH 2/5] SIMD-0064: Transaction Receipts (#64) * create transaction receipt simd * update * Update 0063-transaction-receipt.md * Update 0063-transaction-receipt.md * fix name * tree spec wip * receipt tree spec wip * fixup * remove logs from receipt * update * update * update * fix lint * fix bankhash * Update 0064-transaction-receipt.md * Update 0064-transaction-receipt.md * Update 0064-transaction-receipt.md * update tree spec * change receipt structure to use Message hash instead of a signature. * bench: add benchmarks on receipt tree with message hashes instead of signatures Removed signatures and added message hashes for our benchmarks. * minor fixes * minor fixes * Update 0064-transaction-receipt.md * fix: add len of receipts to tree * fix: add byte ordering for length suffix * fix: lint * change Receipt to TransactionReceipt Co-authored-by: Trent Nelson * change slot to block Co-authored-by: Trent Nelson * optimisations and clean up * remove redundant comment Co-authored-by: ripatel-fd * remove redundant comment for version Co-authored-by: ripatel-fd * add root for empty set. Co-authored-by: Richard Patel * fix typo Co-authored-by: Richard Patel * append justification for sha256 Co-authored-by: Richard Patel * change terminology for receipts Co-authored-by: Richard Patel * fix receipt terminology for tree Co-authored-by: Richard Patel Co-authored-by: Trent Nelson * Minor clean up * fix * Update 0064-transaction-receipt.md * grammar * clarify * typo * clarify * company name * precision * add layout and fix lint Co-authored-by: Richard Patel * fix typo in transaction Co-authored-by: lheeger-jump <126003637+lheeger-jump@users.noreply.github.com> * make layout section more explicit Co-authored-by: Richard Patel * nit: mention theoretical perf in hash function choice Co-authored-by: ripatel-fd * change should to must - rf2119 Co-authored-by: Trent Nelson * update should to must - rfc2119 Co-authored-by: Trent Nelson * replace "fixed" with "avoided" to be more clear Co-authored-by: Trent Nelson * More details in terminology section Co-authored-by: Trent Nelson * fix lint and add missing node in tree spec Co-authored-by: Trent Nelson * Clarify impact of receipts Co-authored-by: Trent Nelson * fix lint * add intermediate_node when leaf count is zero Co-authored-by: Trent Nelson * Author list * Fix hash notation Co-authored-by: Trent Nelson * add empty intermediate root for empty vector illustration * Empty tree --------- Co-authored-by: harsh4786 <50767810+harsh4786@users.noreply.github.com> Co-authored-by: Trent Nelson Co-authored-by: ripatel-fd Co-authored-by: Richard Patel Co-authored-by: lheeger-jump <126003637+lheeger-jump@users.noreply.github.com> Co-authored-by: Trent Nelson --- proposals/0064-transaction-receipt.md | 426 ++++++++++++++++++++++++++ 1 file changed, 426 insertions(+) create mode 100644 proposals/0064-transaction-receipt.md diff --git a/proposals/0064-transaction-receipt.md b/proposals/0064-transaction-receipt.md new file mode 100644 index 000000000..7f918fa7e --- /dev/null +++ b/proposals/0064-transaction-receipt.md @@ -0,0 +1,426 @@ +--- +simd: '0064' +title: Transaction Receipts +authors: + - Anoushk Kharangate (Tinydancer) + - Richard Patel (Jump) + - Harsh Patel (Tinydancer) +category: Standard +type: Core +status: Draft +created: 2023-06-20 +feature: (fill in with feature tracking issues once accepted) +--- + +## Summary + +Here we propose a mechanism for proving transaction inclusion into a block in +the Solana protocol. This is a pre-requisite for several use-cases that would +like to build upon a [Simple Payment Verification](https://en.wikipedia.org/wiki/Bitcoin_network#Payment_verification) +like construction. + +We employ the well-known [Merkle Tree](https://en.wikipedia.org/wiki/Merkle_tree) +data structure to compress a block's transactions and their results into a compact +identifier, with which inclusion proofs can be generated + +## New Terminology + +TransactionReceiptData: A deterministic encoding of state changes induced by a +transaction that includes the version, the status and a message hash of the transaction. + +Transaction Receipt Tree: A commitment scheme over all transaction receipts +in a slot. + +## Motivation + +One of the fundamental requirements of a Solana client is access to the status +of a confirmed transaction. This is due to the fact that transactions are not +guaranteed to be confirmed once submitted, and may fail once executed. Virtually +all Solana clients therefore use a subscription-based or polling mechanism to +inquire whether a transaction was successfully executed. + +The only standard mechanism to retrieve transaction statuses remotely is via the +RPC protocol. However, the RPC protocol only serves replay information of the +RPC service itself. It does not provide information whether the validator +network at large has derived the same information. This may allow an RPC +provider to send incorrect information to a client, such as marking a failed +transaction as successful. + +Solana validator nodes replay all transactions and thus have access to +transaction status information. To improve security, clients should verify that +transaction status information received via RPC matches the validator network at +large. + +This proposal introduces a new “transaction receipt” data structure, which +contains a subset of the information available via RPC. The derivation and +serialization functions for transaction receipts are defined to be deterministic +to prevent malleability. + +To succinctly detect transaction receipt mismatches, this proposal further +introduces a commitment scheme based on a binary hash tree that is +constructed once per slot. + +### Design Goals + +1. Transaction Receipts must be deterministic. + Given a transaction T and ledger history leading up to it, serializing the + receipt generated for T should result in the same byte vector for all nodes + in the network. + *Rationale:* Determinism is required for cluster-wide consensus. + +2. Transaction Receipts must not be required during block construction. + *Rationale:* Future upgrades propose tolerating asynchronous replay during + block construction. In other words, validators should be allowed to produce + and distribute a block before replaying said block. It is impossible to + introduce such a tolerance if transaction receipts are mandatory components + of blocks. + +## Alternatives Considered + +### Using TransactionStatusMeta + +An alternative to introducing a new transaction receipt type is reusing the transaction +status data as it appears in RPC ([TransactionStatusMeta]). This would reduce +complexity for clients. + +However, *TransactionStatusMeta* has no strict definition and is thus malleable +in violation of design goal 1. Technical reasons are as follows: + +- Available fields vary between node releases. +- Log data is truncated based on node configuration. +- The [TransactionResult] type includes error codes, which are + implementation-defined (Thus breaks multi-client compatibility). + +This would make a commitment scheme impractical as-is. Addressing these concerns +is a breaking change. + + [TransactionStatusMeta]: https://docs.rs/solana-transaction-status/1.16.1/solana_transaction_status/struct.TransactionStatusMeta.html + [TransactionResult]: https://docs.rs/solana-sdk/1.16.1/solana_sdk/transaction/type.Result.html + +### Proof-of-History / Block Hash + +Another alternative to introducing a new commitment scheme is reusing the +proof-of-history (PoH) hash chain. This was proposed in +[SPV Proposal](https://docs.solana.com/proposals/simple-payment-and-state-verification#transaction-merkle). + +The PoH chain currently commits to the signatures of all transactions added to +the ledger. The consensus layer then periodically votes on the last state of the +PoH chain for each bankhash which is an extension of the blockhash. +Expanding the PoH hash is the least complex option as of today but +is consequential for future upgrades. + +Such a change would be incompatible with design goal 2 because it redefines +the PoH hash to additionally commit to execution results, instead of only +ledger content. Block producers are then forced to synchronously replay while +appending PoH ticks. + +Furthermore, it significantly changes behavior in the event of an execution +disagreement (e.g. due to a difference in execution behavior between +validators). Mixing execution results into PoH forces execution disagreements to +result in a chain split. + +## Detailed Design + +### TransactionReceiptData + +A TransactionReceiptData v1 object deterministically encodes the effects of executing +a transaction. It contains the following information: + +- Transaction Receipt Version (currently v1, other values reserved for future upgrades) +- Message Hash identifying the transaction it refers to +- The Execution Status, a boolean identifying whether the transaction was successfully + executed or failed without extrinsic state changes. + +It is defined by the following pseudocode. + +```rust +// Scalars are encoded in little-endian order + +const RECEIPT_VERSION_V1: u64 = 1; + +const RECEIPT_STATUS_SUCCESS: u8 = 0; +const RECEIPT_STATUS_FAILURE: u8 = 1; + +// Size: 0x29 +struct TransactionReceiptData { + // Offset: 0x00 + // Must be RECEIPT_VERSION_V1 + version: u64, + + // Offset: 0x08 + message_hash: [u8;32], + + // Offset: 0x28 + // Must be one of RECEIPT_STATUS_{SUCCESS,FAILURE} + status: u8, +} +``` + +#### Binary layout + +The layout of the transaction receipt data when being hashed is as follows: +0x00 Prefix +0x01 Version +0x09 Status +0x0a Message Hash + +Since SHA-256 prefers 32 byte blocks with non-overlapping data, we considered padding +the data to be aligned. However, without precomputation this would be worse in +terms efficiency as it would require on block more. With precomputation the +efficiency would be roughly the same. Below are the two layouts: + +**Naive Case:** + +```txt +// SHA block 0 +[0x00..0x01] Merkle Node Type +[0x01..0x09] Version +[0x09..0x0a] Status +[0x0a..0x20] Message Hash +// SHA block 1 +[0x20..0x2a] Message Hash + +[0x2a..0x33] Merkle-Damgard Suffix +[0x33..0x40] Merkle-Damgard Padding +``` + +**Potentially Optimised Case:** + +```txt +// SHA block 0 (precomputed) +[0x00..0x01] Merkle Node Type +[0x01..0x09] Version +[0x09..0x0a] Status +[0x0a..0x20] Padding +// SHA block 1 +[0x20..0x40] Message Hash +// SHA block 2 +[0x40..0x49] Merkle-Damgard Suffix +[0x49..0x60] Merkle-Damgard Padding +``` + +The leaf is constructed by hashing the struct fields in this order: +hash(0x0, version, status, message_hash) + +##### Key considerations + +- When inserted as a leaf node into the tree defined below, + this structure requires hashing two SHA-256 blocks. +- The `version` integer can safely be shortened to `u8` in + a future upgrade (Due to little-endian encoding). Using a + u64 allows for this flexibilty. Futhermore, we could also + fit a domain hash separator in the `version` bits to prevent + re-interpreting the leaf nodes as a different structure in + the future as a good practice. +- In future versions, hashes for a subset of fields exhibiting + a small set of possible combinations could be aligned to the + hash function block size and precomputed. (Such as `version` + and `status`). For now, this micro-optimization would yield + no performance improvements as mentioned above. + +### Transaction Receipt Tree + +The transaction receipt tree is a binary merkle tree of transaction receipts, where +each node is a 32 byte hash. + +The tree is derived from a vector of TransactionReceiptData objects. +It is designed to feature the following properties: + +- The Merkle tree construction used is an extension of the tree construction used + in PoH. (With different input data) + +- The tree derivation function is surjective: + Each vector of transaction receipts results in a unique tree, + thereby making it deterministic and immalleable. + +- The order of the leaf nodes matches the block's transaction order. + (As it unamiguously appears in the PoH construction) + +- Succinct inclusion proofs are constructed by providing a hash path + from a leaf node to the root node. The inclusion proof format is defined + separately from this SIMD. + +#### Specification + +- Input: Vector of TransactionReceiptData objects in PoH order + +- Pre-condition: Each element's `message_hash` is unique + +- Output: Transaction receipt tree root (32 byte hash) + +- Definitions: + - The `intermediate_root` is the 32 byte root of the binary merkle tree + as externally specified. + [Specification: Binary Merkle Tree](https://github.com/solana-foundation/specs/blob/main/core/merkle-tree.md) + - The `root_prefix` is the byte `0x80` + - The function `u64_le_encode` encodes a 64-bit integer in little-endian + byte order. + - The `leaf_count` is the number of TransactionReceiptData objects to + serve as leaf nodes in the tree. + - `sentinel_root` is a byte array of zeros with length 32 + +- If the leaf count is zero, the output is + `sha256(root_prefix || sentinel_root || u64_le_encode(0))`. + +- If the leaf count is non-zero, the output is + `sha256(root_prefix || intermediate_root || u64_le_encode(leaf_count))`. + +#### Tree design considerations + +- *Use of the PoH hash tree construction* + - Allows reusing existing code for constructing the transaction receipt tree. + - Avoids introducing a new cryptographic construction. + - Alternatives to SHA-256 based constructions, such as BLAKE3 or SHA-3 would + offer superior theoretical performance. + - Existing optimized SHA-256 hash tree implementations are readily available. + - As of 2023-Oct, hardware implementations of SHA-256 are currently available + (via SHA-NI on x86, via Firedancer's SystemVerilog implementation on AWS F1 + FPGA), whereas BLAKE3 and SHA-3 are not. (SHA-3 is only available on recent + Arm cores) + - As of 2023-Oct, the Merkle tree root for an empty input vector is + unspecified, which is a specification bug. The canonical PoH Merkle tree + in the Solana Labs implementation does not define the empty tree either. + Therefore, we introduce zero as the sentinel value for the root of an empty + hash tree. + +- *Leaf count suffix*: + The PoH tree implicitly expands the internal leaf count to a power of two, + causing the `intermediate_root` two have more than one pre-image for certain + leaf counts. This avoided by instead including the leaf count suffix in the + final hash. + +- *Exclusion proofs are not provided*: + Although it is possible to construct Merkle-based set exclusion proofs, this + feature was not part of this proposal's design criteria. A proposed exclusion + proof mechanism involves sorting transaction receipts by their message hash. + This change may be considered in a future version of the transaction receipt + tree. + +#### Examples + +The following illustrates transaction receipt hash trees for various inputs. + +```txt +Transaction receipt tree with an empty vector of transaction receipts +where Nα is the root. +Nα := sha256(0x80 || intermediate_root([0u8;32]) || u64_le_encode(0)) + +Nα +| +0 +``` + +```txt +Transaction receipt tree with four transaction receipts as leaf +nodes [L0, L1, L2, L3] where R0, R1, R2, R3 are the serialized +TransactionReceiptData objects and Nδ is the root. + + Nδ + / \ + / \ + Nγ N(transaction_receipts) + / \ + / \ + Nα Nβ + / | / \ + / | / \ +L0 L1 L2 L3 + +L0 := sha256(concat(0x00, R0)) +L1 := sha256(concat(0x00, R1)) +L2 := sha256(concat(0x00, R2)) +L3 := sha256(concat(0x00, R3)) +Nα := sha256(concat(0x01, L0, L1)) +Nβ := sha256(concat(0x01, L2, L3)) +Nγ := sha256(concat(0x01, Nα, Nβ)) +Nδ := sha256(concat(0x80, Nγ, u64_le_encode(4))) # leaf count +``` + +```txt +Transaction receipt tree with five transaction receipts +as leaf nodes [L0, L1, L2, L3, L4] where R0, R1, R2, R3, R4 are the +serialized TransactionReceiptData objects and Nτ is the root. + + Nτ + / \ + / \ + Nζ N(transaction_receipts) + / \ + / \ + Nδ Iε + / \ \\ + / \ \\ + Nα Nβ Nγ + / \ / \ || + L0 L1 L2 L3 L4 + +L0 := sha256(concat(0x00, R0)) +L1 := sha256(concat(0x00, R1)) +L2 := sha256(concat(0x00, R2)) +L3 := sha256(concat(0x00, R3)) +L4 := sha256(concat(0x00, R4)) +Nα := sha256(concat(0x01, L0, L1)) +Nβ := sha256(concat(0x01, L2, L3)) +Nγ := sha256(concat(0x01, L4, L4)) +Nδ := sha256(concat(0x01, Nα, Nβ)) +Iε := sha256(concat(0x01, Nγ, Nγ)) +Nζ := sha256(concat(0x01, Nδ, Iε)) +Nτ := sha256(concat(0x80, Nζ, u64_le_encode(5))) # leaf count +``` + +#### Benchmarks + +Tinydancer has prepared benchmarks comparing two merkle tree implementations. +The input used was a vector of 500k TransactionReceiptData objects. + +1) Solana Labs Merkle Tree: This is the pure Rust implementation that is currently + used by the Solana Labs client. + [Solana Labs GitHub](https://github.com/solana-labs/solana/tree/master/merkle-tree) +2) Firedancer binary merkle tree (bmtree): Implemented in C and uses Firedancer's + optimised SHA-256 implementation. Benchmarks were performed via Rust FFI bindings, + and thus may not be representative of the upper bound. + An improved version using AVX-512 instructions is in progress. + [Firedancer GitHub](https://github.com/firedancer-io/firedancer/tree/main/src/ballet/bmtree) + +[Benchmark Source](https://github.com/tinydancer-io/merkle-bench) +![TINY_bench](https://github.com/tinydancer-io/solana-improvement-documents/assets/50767810/637dc83a-b3d2-4616-b70e-4fbb8a9e17fd) + +## Impact + +The generation of a Transaction Receipt Tree is a prerequisite for providing proof +that a transaction was included in a block. Itself a step toward providing proof +that a transaction was executed and accepted under consensus by a Solana cluster. +A major improvement in trust-minimization for the ecosystem, opening the door to +new use-cases and reducing infrastructure requirements for some of today's. + +## Security Considerations + +The transaction receipt tree is expected to be used for transaction inclusion proofs. +For example, an inclusion proof might attest that a token transfer has succeeded. +Such proofs may then be relied on to provide financial services. + +It is thus important to ensure that proofs cannot be forged. + +Common forgery attacks against Merkle trees include: + +- Various forms of pre-image attacks against the underlying hash functions. + As of 2023-Oct, no practical collision attacks against SHA-256 are known. +- Malleability and type confusion in the hash tree construction. + These are prevented via two mechanisms: + 1. Hash domain separation via one byte prefixes (for leaf nodes, branch + nodes, and the final hash respectively) + 2. A node count suffix to prevent malleable leaf count attacks + +Further conerns include: + +- Implementation bugs: To reduce the risk of such, this proposal deliberately + keeps the amount of newly introduced logic low. +- Performance-related attacks (DoS): The computational complexity of transaction + receipt tree construction is `O(n+log n)` where `n` is the number of + transactions. There are no other user controllable components such as + variable-length inputs. + +## Backwards Compatibility + +This change does not impact the Solana ledger, and thus introduces no backwards +compatibility concerns. From a85a0dae75b3f2ee0f22447624614f99834b9060 Mon Sep 17 00:00:00 2001 From: Jacob Creech <82475023+jacobcreech@users.noreply.github.com> Date: Fri, 20 Oct 2023 14:37:48 -0500 Subject: [PATCH 3/5] Add living status, supercede, extends, and rfc 2119 recommendation to template (#73) * add: living status * add: key work language to template When writing your SIMD, you should follow keyword usage as specified in rfc 2119 and rfc 8174 * add: supercede and extends added the ability to denote if a specific SIMD proposed supercedes or extends a previous SIMD * fix: lint issues * added blurb on scrutiny and review added blurb on scrutiny and review --- XXXX-template.md | 9 +++++++++ proposals/0001-simd-process.md | 13 +++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/XXXX-template.md b/XXXX-template.md index 5e7be7c8f..b63eb67f4 100644 --- a/XXXX-template.md +++ b/XXXX-template.md @@ -8,6 +8,9 @@ type: Core/Networking/Interface/Meta status: Draft created: (fill me in with today's date, YYYY-MM-DD) feature: (fill in with feature tracking issues once accepted) +supercedes: (optional - fill this in if the SIMD supercedes a previous SIMD) +extends: (optional - fill this in if the SIMD extends the design of a previous + SIMD) --- ## Summary @@ -39,6 +42,12 @@ to another Solana core contributor. The generally means: - Interaction with other features - Edge cases +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this +document are to be interpreted as described in [RFC +2119](https://www.ietf.org/rfc/rfc2119.txt) and [RFC +8174](https://www.ietf.org/rfc/rfc8174.txt). + ## Impact How will the implemented proposal impacts dapp developers, validators, and core contributors? diff --git a/proposals/0001-simd-process.md b/proposals/0001-simd-process.md index 405e8761e..2966352ea 100644 --- a/proposals/0001-simd-process.md +++ b/proposals/0001-simd-process.md @@ -5,7 +5,7 @@ authors: - Jacob Creech (Solana Foundation) category: Meta type: Meta -status: Draft +status: Living created: 2022-10-18 --- @@ -68,7 +68,7 @@ Proposals but apply to areas other than the Solana protocol itself. They may propose an implementation, but not to Solana's codebase; they often require community consensus and users are typically not free to ignore them. Examples include procedures, guidelines, changes to the decision-making process, and -changes to the tools or environment used in Solana development. Any meta-EIP is +changes to the tools or environment used in Solana development. Any meta-SIMD is also considered a Process Proposals. ## Proposal Lifecycle @@ -79,6 +79,7 @@ The stages in a lifecycle of a proposal are as follows: - Draft - Review - Accepted +- Living - Stagnant - Withdrawn - Implemented @@ -101,6 +102,7 @@ flowchart LR Idea ---> Draft; Draft ---> Review; Review ---> Accepted; + Review ---> Living; Draft ---> Stagnant; Review ---> Stagnant; @@ -158,6 +160,13 @@ far the most effective way to see a proposal through to completion: authors should not expect that other project developers will take on responsibility for implementing their accepted feature. +### Living + +A special status for SIMDs that are designed to be continually updated and not +reach a state of finality. This includes most notably SIMD-1. This status must +undergo extra scrutiny and review when updating the status from review to +living. + ### Stagnant If a proposal reaches 6 months without activity, the proposal will be From 93322ef3d8ab35329fbce09c80eac30ccdae7122 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 26 Oct 2023 14:28:20 +0200 Subject: [PATCH 4/5] init SIMD address wording feedback revise motivation section one more pass on wording --- proposals/0077-programify-feature-gate.md | 216 ++++++++++++++++++++++ 1 file changed, 216 insertions(+) create mode 100644 proposals/0077-programify-feature-gate.md diff --git a/proposals/0077-programify-feature-gate.md b/proposals/0077-programify-feature-gate.md new file mode 100644 index 000000000..8d0db11f0 --- /dev/null +++ b/proposals/0077-programify-feature-gate.md @@ -0,0 +1,216 @@ +--- +simd: '0077' +title: Programify Feature Gate Program +authors: + - Joe Caulfield + - Tyera Eulberg +category: Standard +type: Core +status: Draft +created: 2023-10-26 +feature: #33547 +--- + +## Summary + +### Roadmap + +This is SIMD 1/3 expected for **Multi-Client Feature Gates**. See + + +**Goals:** + +- Decentralized control over queuing new runtime features +- Automatic feature activation selection based on stake weight with supporting + software +- Decentralized control of the mechanism itself + +**Resulting Architecture:** + +1. 👉 **Feature Creation:** Features can be created by anyone. Each is owned by + an upgradeable BPF program at `Feature111111111111111111111111111111111111` +2. **Feature Queuing:** A governance process nominates features that should be + queued for activation +3. **Feature Recognition & Activation:** Features are activated based on stake + support of nodes who recognize the feature in their software version + +### Proposal + +This SIMD outlines a proposal to replace the non-existent program at address +`Feature111111111111111111111111111111111111`, which is the owner of all +feature accounts, with an upgradeable BPF program. + +It defines the program's initial functionality - which consists solely of the +capability to revoke pending feature activations - and an initial upgrade +control process for managing upgrades of the program. + +Important note: the process by which core contributors *activate* features +would remain completely unchanged by this SIMD. + +## Motivation + +The complete Multi-Client Feature Gate architecture - mentioned under "Roadmap" +above - will require several changes to the way feature accounts are handled. +The most obvious approach is to place a program in charge of these accounts. +An upgradeable BPF program presents a more logical solution over a native +program for this task, since it provides a built-in system for upgrades. + +In the case of this particular proposal, this BPF program will provide +engineers the capability to revoke pending feature activations. + +Currently, a feature is queued for activation by a keypair holder creating an +empty account and assigning it to the +`Feature111111111111111111111111111111111111` program. Because there is no +actual program at that address, the queuing is irreversible; if the runtime +knows about a feature gate at that address, it will activate it at the next +epoch boundary. This means there is no recourse in the case of a mistaken +queuing, discovery of a bug in the feature's code, or simply a desire to manage +the cadence and schedule of activations. + +A fully-fledged BPF program would take ownership of those accounts and support +revoking queued features, giving engineers more flexibility and safeguards. + +## Alternatives Considered + +The Feature Gate program could instead be a native program, rather than a BPF +program. However, this would mean any changes to the program would need to be +implemented by all validator clients in coordination. This makes upgrading the +program to support the complete Multi-Client Feature Gate architecture +cumbersome and potentially dangerous. + +## New Terminology + +- Feature Gate program: The BPF program that will own all feature accounts. +- “Revoke” or “revoke pending activation”: The act of reallocating a feature + account’s data to zero, assigning it to the system program, and defunding + its lamports balance - effectively removing it from the runtime’s recognized + set of features pending activation. + +## Detailed Design + +The design for this proposal consists of three components: + +- Deployment of the program by the runtime +- Revoking pending features using the BPF program +- Upgrade control process for the BPF program + +### Deploying the Program + +In order to deploy an upgradeable BPF program to the address at +`Feature111111111111111111111111111111111111`, a runtime change is required. +This change would allow the runtime, upon feature activation, to move an +already deployed upgradeable BPF program into the account at address +`Feature111111111111111111111111111111111111`. + +For maximum security, the initial program can be a no-op program with an +intentionally large allocation (to allow for larger programs in the future). + +Once the program is moved into place, the program can be upgraded using the +conventional `BpfUpgradeableLoader` method to include the revoke functionality +defined below. + +The specific no-op program to be initially moved into +`Feature111111111111111111111111111111111111` by the runtime should be +verifiably built and then deployed to devent, testnet, and mainnet-beta. + +### Revoking Pending Features + +As mentioned above under “New Terminology”, the act of revoking a pending +feature activation consists of reallocating a feature account’s data to zero, +assigning it to the system program, and defunding its lamports balance. This +causes the feature account to be “revoked” since the runtime will no longer +detect it as an account owned by `Feature111111111111111111111111111111111111`. + +When a core contributor executes the `solana feature activate` command, a +signature from the feature keypair is required to activate it, since its state +will change. Similarly, we can require the same feature keypair’s signature to +revoke said feature. + +Consider the instruction as it would appear in the Feature Gate Program: + +```rust +pub enum FeatureGateInstruction { + /// Revoke a pending feature activation. + /// + /// A "pending" feature activation is a feature account that has been + /// allocated and assigned, but hasn't yet been updated by the runtime + /// with an `activation_slot`. + /// + /// Features that _have_ been activated by the runtime cannot be revoked. + /// + /// Accounts expected by this instruction: + /// + /// 0. `[w+s]` Feature account + /// 1. `[w]` Destination (for rent lamports) + RevokePendingActivation, +} +``` + +When this instruction is invoked with the proper signature, the feature account +would be reallocated, defunded, and returned to the System Program, like so: + +```rust +/* Checks */ + +let new_destination_lamports = feature_info + .lamports() + .checked_add(destination_info.lamports()) + .ok_or::(FeatureGateError::Overflow.into())?; + +**feature_info.try_borrow_mut_lamports()? = 0; +**destination_info.try_borrow_mut_lamports()? = new_destination_lamports; + +feature_info.realloc(0, true)?; +feature_info.assign(&system_program::id()); +``` + +### Controlling the Program Upgrades + +Because the Feature Gate program requires special runtime support, upgrading +the Feature Gate program will initially be controlled by a 2-of-3 +multi-signature authority, consisting of key-holders distributed across +validator-client teams. + +- Solana Labs: 1 key-holder +- Jump: 1 key-holder +- Jito: 1 key-holder + +Only when 2 out of 3 key-holders have authorized an upgrade will the Feature +Gate program be upgraded. + +**Note:** This includes the upgrade required to upgrade the initial no-op +program to the first released Feature Gate Program supporting revoke. + +## Impact + +Core contributors are positively impacted by this change, since the ability to +revoke pending feature activations is a significant security advantage. + +There is otherwise no change to the activation process whatsoever. This +includes queuing features for activation with the CLI and the timing of their +activation by the runtime. + +This proposal also increases decentralized control over some components of the +feature activation process, but can be more decentralized in the future. + +## Security Considerations + +Currently the accounts used for feature-gating are owned by a program ID that +does not exists. This means that there’s no on-chain authority that can modify +feature accounts once they’ve been created. This allows the runtime to +confidently update their state upon activation. + +With this proposal, a live BPF program - which can accept instructions from +anyone and execute code - will be the owner of these accounts. This does create +some risk if *both* the program’s processor code *and* its upgrade authority +are not properly managed. But thoroughly reviewed and safe processor code, as +well as a decentralized system for upgrading the program, will together +mitigate these new risks as much as possible. + +## Backwards Compatibility + +As mentioned under "Summary", the process by which core contributors *activate* +features would remain completely unchanged by this SIMD. + +This SIMD only *adds* the capability to revoke pending activations, so it's +completely backwards compatible. \ No newline at end of file From 32a6f91a74332671ee6f5f0716a05fa1c735985a Mon Sep 17 00:00:00 2001 From: Joe Date: Sun, 29 Oct 2023 14:40:55 +0100 Subject: [PATCH 5/5] add more context for native vs BPF --- proposals/0077-programify-feature-gate.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/proposals/0077-programify-feature-gate.md b/proposals/0077-programify-feature-gate.md index 8d0db11f0..367d16b19 100644 --- a/proposals/0077-programify-feature-gate.md +++ b/proposals/0077-programify-feature-gate.md @@ -78,6 +78,19 @@ implemented by all validator clients in coordination. This makes upgrading the program to support the complete Multi-Client Feature Gate architecture cumbersome and potentially dangerous. +However, one of the main benefits gained by instead opting for a native program +would be an easier ability to upgrade via feature gate. + +Another alternative considered is to use the deployment process outlined in the +"Deploying the Program" section under "Detailed Design" to upgrade the program +behind feature gates in the future. In short, contributors would stage changes +to the program in another account and create a runtime feature gate to swap +this upgraded version of the program into the current program's place. + +Note this may render the multi-signature program upgrade authority setup +useless, except for less common use cases, such as a critical upgrade that +could not be done through a feature gate. + ## New Terminology - Feature Gate program: The BPF program that will own all feature accounts.