Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NIT-2640] Add EVM tracing for Stylus programs #2530

Merged
merged 31 commits into from
Aug 16, 2024
Merged

Conversation

gligneul
Copy link
Contributor

@gligneul gligneul commented Jul 29, 2024

Stylus programs perform HostIO calls when they need to interact with the EVM state, for instance, when loading a value from storage. When performing these calls, we want to capture EVM traces as if they were being executed on the EVM. So, when a stylus program loads a value from storage, we want to emit a trace for the SLOAD opcode.

We should set up a fake EVM stack for each opcode as if it were being executed in the EVM. We should also emit a POP opcode after emitting a trace for an opcode that leaves a value on the stack.

When a Stylus program performs a HostIO (mostly in arbitrator/wasm-libraries/user-host-trait/src/lib.rs), the host-calling library also calls a special CaptureHostIO HostIO, passing the call's name, arguments, and outputs.

On the host side, Nitro now handles this call on arbos/programs/api.go. The API now calls the TracingInfo.CaptureEVMTraceForHostio method to generate the EVM traces for all HostIOs based on their names.

This PR centralizes all the EVM traces in the CaptureEVMTraceForHostio method and removes redundant tracing code on the programs' API. It also adds tests specifically for the EVM traces on Stylus.

Example

Here is an example trace for a transaction that performs a write and then a read to the storage contract using the multi-call contract.

{
  "gas": 249657,
  "failed": false,
  "returnValue": "117094d34db10679fa57f2146afe957557635232129eef332049aeed23a53c98",
  "structLogs": [
    {
      "pc": 0,
      "op": "SLOAD",
      "gas": 0,
      "gasCost": 0,
      "depth": 1,
      "stack": [
        "0x0"
      ],
      "storage": {
        "0000000000000000000000000000000000000000000000000000000000000000": "0000000000000000000000000000000000000000000000000000000000000000"
      }
    },
    {
      "pc": 0,
      "op": "SLOAD",
      "gas": 0,
      "gasCost": 0,
      "depth": 1,
      "stack": [
        "0x3c79da47f96b0f39664f73c0a1f350580be90742947dddfa21ba64d578dfe600"
      ],
      "storage": {
        "0000000000000000000000000000000000000000000000000000000000000000": "0000000000000000000000000000000000000000000000000000000000000000",
        "3c79da47f96b0f39664f73c0a1f350580be90742947dddfa21ba64d578dfe600": "0000000000000000000000000000000000000000000000000000000000000000"
      }
    },
    {
      "pc": 0,
      "op": "CALLDATACOPY",
      "gas": 8798120,
      "gasCost": 1,
      "depth": 1,
      "stack": [
        "0xd5",
        "0x0",
        "0x0"
      ]
    },
    {
      "pc": 0,
      "op": "CALL",
      "gas": 7712916,
      "gasCost": 7715516,
      "depth": 1,
      "stack": [
        "0x0",
        "0x0",
        "0x41",
        "0x0",
        "0x0",
        "0x457b1ba688e9854bdbed2f473f7510c476a3da09",
        "0x75b094"
      ]
    },
    {
      "pc": 0,
      "op": "CALLDATACOPY",
      "gas": 8643741,
      "gasCost": 1,
      "depth": 2,
      "stack": [
        "0x41",
        "0x0",
        "0x0"
      ]
    },
    {
      "pc": 0,
      "op": "SSTORE",
      "gas": 8643732,
      "gasCost": 1,
      "depth": 2,
      "stack": [
        "0x117094d34db10679fa57f2146afe957557635232129eef332049aeed23a53c98",
        "0x6102537d4b5c24fa4582f62653740c8e110e5b5d3b037f7f5dba539ed460550d"
      ],
      "storage": {
        "6102537d4b5c24fa4582f62653740c8e110e5b5d3b037f7f5dba539ed460550d": "117094d34db10679fa57f2146afe957557635232129eef332049aeed23a53c98"
      }
    },
    {
      "pc": 0,
      "op": "STOP",
      "gas": 7700411,
      "gasCost": 0,
      "depth": 2,
      "stack": []
    },
    {
      "pc": 0,
      "op": "CALL",
      "gas": 7700481,
      "gasCost": 7700581,
      "depth": 1,
      "stack": [
        "0x0",
        "0x0",
        "0x21",
        "0x0",
        "0x0",
        "0x457b1ba688e9854bdbed2f473f7510c476a3da09",
        "0x758001"
      ]
    },
    {
      "pc": 0,
      "op": "CALLDATACOPY",
      "gas": 8629783,
      "gasCost": 1,
      "depth": 2,
      "stack": [
        "0x21",
        "0x0",
        "0x0"
      ]
    },
    {
      "pc": 0,
      "op": "SLOAD",
      "gas": 8629774,
      "gasCost": 67097,
      "depth": 2,
      "stack": [
        "0x6102537d4b5c24fa4582f62653740c8e110e5b5d3b037f7f5dba539ed460550d"
      ],
      "storage": {
        "6102537d4b5c24fa4582f62653740c8e110e5b5d3b037f7f5dba539ed460550d": "0000000000000000000000000000000000000000000000000000000000000000"
      }
    },
    {
      "pc": 0,
      "op": "POP",
      "gas": 8562677,
      "gasCost": 0,
      "depth": 2,
      "stack": [
        "0x117094d34db10679fa57f2146afe957557635232129eef332049aeed23a53c98"
      ]
    },
    {
      "pc": 0,
      "op": "RETURN",
      "gas": 7628190,
      "gasCost": 0,
      "depth": 2,
      "stack": [
        "0x20",
        "0x0"
      ]
    },
    {
      "pc": 0,
      "op": "RETURNDATACOPY",
      "gas": 8699820,
      "gasCost": 7,
      "depth": 1,
      "stack": [
        "0x20",
        "0x0",
        "0x0"
      ]
    },
    {
      "pc": 0,
      "op": "RETURN",
      "gas": 7750343,
      "gasCost": 0,
      "depth": 1,
      "stack": [
        "0x20",
        "0x0"
      ]
    }
  ]
}

@cla-bot cla-bot bot added the s Automatically added by the CLA bot if the creator of a PR is registered as having signed the CLA. label Jul 29, 2024
@gligneul gligneul requested review from eljobe and diegoximenes July 29, 2024 20:34
Copy link
Collaborator

@tsahee tsahee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally, this is a very good direction. Great work.

Two comments/questions:

  1. Will return data be captured if the contract exists normally and not via exit early? (and if so - is there a test that covers that?)
  2. I would be happy if we could test some sort of equivalence between stylus and non-stylus. At least for a few instructions, at least at first.
    One way would be to deploy rustFile("multicall") and mocksgen.DeployMultiCallTest. These two contracts have the same interface to wrap some of opcodes - at least SSTORE/SLOAD/CALL/LOG. Calling both contracts with the same data should not create identical trace.. but both should have the basic opcode tested.

@@ -534,7 +534,7 @@ pub trait UserHost<DR: DataReader>: GasMeteredMachine {
fn return_data_size(&mut self) -> Result<u32, Self::Err> {
self.buy_ink(HOSTIO_INK)?;
let len = *self.evm_return_data_len();
trace!("return_data_size", self, be!(len), &[], len)
trace!("return_data_size", self, &[], be!(len), len)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't I see a signature change for the trace! macro in this PR?
Was it modified in an earlier PR?

Copy link
Contributor Author

@gligneul gligneul Jul 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trace macro was already working correctly on the Stylus side. This PR adds the trace handling on the host/nitro side.

Regarding this change, I swapped the arguments and outputs to better reflect the semantics of the fields. The return value was being sent as an argument instead of an output. It wasn't a problem until now because this was not being used before.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you might need a corresponding change to cargo-stylus here, but I'm not sure https://github.com/OffchainLabs/cargo-stylus/blob/794221e183cdfd791a2b5be04ec2ab762b381269/replay/src/trace.rs#L367

(also applies to the other tracing order changes)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened a PR on cargo stylus updating the API: OffchainLabs/cargo-stylus#73

@tsahee
Copy link
Collaborator

tsahee commented Jul 30, 2024

  1. Will return data be captured if the contract exists normally and not via exit early? (and if so - is there a test that covers that?)

The right way to log return might be from CallProgram in program.go (or from the native callProgram / etc). In this case - don't log exit_early and let the same place log normal and early exit

@gligneul
Copy link
Contributor Author

The right way to log return might be from CallProgram in program.go (or from the native callProgram / etc). In this case - don't log exit_early and let the same place log normal and early exit

Makes sense, I will fix it.

I would be happy if we could test some sort of equivalence between stylus and non-stylus. At least for a few instructions, at least at first.
One way would be to deploy rustFile("multicall") and mocksgen.DeployMultiCallTest. These two contracts have the same interface to wrap some of opcodes - at least SSTORE/SLOAD/CALL/LOG. Calling both contracts with the same data should not create identical trace.. but both should have the basic opcode tested.

Makes sense. I will add this test.

@gligneul gligneul requested review from tsahee and eljobe August 5, 2024 17:33
eljobe
eljobe previously approved these changes Aug 7, 2024
Copy link
Member

@eljobe eljobe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tsahee
tsahee previously approved these changes Aug 7, 2024
Copy link
Collaborator

@tsahee tsahee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left a minor request, but I'm very happy with it. Great work.
Not design-approving yet because I want Lee's take as well.

system_tests/stylus_trace_test.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@PlasmaPower PlasmaPower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach generally looks good to me, though I have a couple comments on details. I haven't reviewed the full details of each opcode tracing yet

@@ -534,7 +534,7 @@ pub trait UserHost<DR: DataReader>: GasMeteredMachine {
fn return_data_size(&mut self) -> Result<u32, Self::Err> {
self.buy_ink(HOSTIO_INK)?;
let len = *self.evm_return_data_len();
trace!("return_data_size", self, be!(len), &[], len)
trace!("return_data_size", self, &[], be!(len), len)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you might need a corresponding change to cargo-stylus here, but I'm not sure https://github.com/OffchainLabs/cargo-stylus/blob/794221e183cdfd791a2b5be04ec2ab762b381269/replay/src/trace.rs#L367

(also applies to the other tracing order changes)

arbos/util/tracing.go Outdated Show resolved Hide resolved
@gligneul gligneul dismissed stale reviews from tsahee and eljobe via d3273f8 August 9, 2024 14:07
eljobe
eljobe previously approved these changes Aug 12, 2024
Copy link
Member

@eljobe eljobe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gligneul
Copy link
Contributor Author

I tested this PR using the Nitro test node and the main branch of cargo-stylus, including the HostIO fixes in Stylus replay. I tested a few HostIOs. Here is an example of an opcode I tested that is broken in the current nitro version and is fixed on this PR.

I created a simple stylus contract based on the stylus-hello-world example repository. Then, I added a function to the contract that calls the gas_left HostIO and stores the value in storage. Here is the snippet of the function.

    pub fn set_gas_left(&mut self) {
        let gas_left = stylus_sdk::evm::gas_left();
        self.set_number(U256::from(gas_left));
        println!("set number to gas left: {}", gas_left);
    }

This HostIO is currently broken on replay because nitro passes the return value on the arguments field instead of the outputs field. (This PR fixes this issue.) If you were to make a transaction on Sepolia and try to trace the transaction, you would get the following error.

➜  stylus-example git:(main) ✗ cargo stylus trace -e $RPC_URL -t $TX | jq
thread 'main' panicked at replay/src/trace.rs:289:31:
range end index 8 out of range for slice of length 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

In the test node with this patch, I can fetch the trace using the cargo stylus trace and replay the execution with cargo stylus replay. The following snippet shows that I could set a breakpoint on lldb and verify the return value of the gas left opcode, as expected. (Running replay on Apple silicon requires the following patch OffchainLabs/cargo-stylus@main...stylus-debugging).

(lldb) br set -f lib.rs -l 82
Breakpoint 2: where = libstylus_hello_world.dylib`stylus_hello_world::Counter::set_gas_left::h4c42aead250c26d5 + 20 at lib.rs:82:24, address = 0x0000000100f48a84
(lldb) c
Process 22005 resuming
Process 22005 stopped
* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = breakpoint 2.1
    frame #0: 0x0000000100f48a84 libstylus_hello_world.dylib`stylus_hello_world::Counter::set_gas_left::h4c42aead250c26d5(self=0x000000016fdfa060) at lib.rs:82:24
   79  	    }
   80
   81  	    pub fn set_gas_left(&mut self) {
-> 82  	        let gas_left = stylus_sdk::evm::gas_left();
   83  	        self.set_number(U256::from(gas_left));
   84  	        println!("set number to gas left: {}", gas_left);
   85  	    }
(lldb) n
Process 22005 stopped
* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = step over
    frame #0: 0x0000000100f48a94 libstylus_hello_world.dylib`stylus_hello_world::Counter::set_gas_left::h4c42aead250c26d5(self=0x000000016fdfa060) at lib.rs:83:25
   80
   81  	    pub fn set_gas_left(&mut self) {
   82  	        let gas_left = stylus_sdk::evm::gas_left();
-> 83  	        self.set_number(U256::from(gas_left));
   84  	        println!("set number to gas left: {}", gas_left);
   85  	    }
   86  	}
(lldb) p gas_left
(unsigned long) 22325
(lldb) c
Process 22005 resuming
set number to gas left: 22325
call completed successfully
Process 22005 exited with status = 0 (0x00000000)

Copy link
Collaborator

@tsahee tsahee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tsahee tsahee merged commit 0de6aaf into master Aug 16, 2024
15 checks passed
@tsahee tsahee deleted the gligneul/stylus-tracing branch August 16, 2024 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design-approved s Automatically added by the CLA bot if the creator of a PR is registered as having signed the CLA.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants