Update code snippets and move to rust workspace #302

0xLucca · 2025-01-08T11:28:19Z

This PR reorganizes the code snippets in the zero-to-hero tutorial to be part of the Rustr workspace

0xLucca · 2025-01-08T13:52:51Z

@UtkarshBhardwaj007 I have migrated the existing code for the current tutorials. I'll wait until the benchmarking PR is merged so I can include that here

UtkarshBhardwaj007 · 2025-01-08T18:03:42Z

@UtkarshBhardwaj007 I have migrated the existing code for the current tutorials. I'll wait until the benchmarking PR is merged so I can include that here

Thanks, this is great. I believe you mean this PR? It is merged now and you should be able to use it. I'll review once you do :)

UtkarshBhardwaj007 · 2025-01-10T14:43:17Z

...utorials/polkadot-sdk/parachains/zero-to-hero/build-custom-pallet/tests/integration_tests.rs

@@ -1,176 +0,0 @@
-// This file is part of 'custom-pallet'.


If you remove all this, then this part of code is not being tested. I think we can go 2 ways for this:

Create a separate crate for each step in a tutorial (like you are doing in this PR). The advantage I can see is that a person following the tutorial will be able to checkout the code until that step. Basically checking out the pallet-unit-testing code would give me all the code till that step (including build-a-custom-pallet). However, since these are tutorials and we expect people to follow along, I don't expect many people to actually go to the Github repo of the tutorial to checkout a step (unless of-course the tutorial asks them to do that). There are 2 disadvantages that I can think of with this approach - a) A lot of code repetition (which is okay IMO) and b) We also need to copy tests in all the steps. Basically, the code in this crate is not being tested which is not good. If we go with this approach, we need to have tests here as well. We can copy the tests from pallet-unit-testing crate here.

Create 1 crate per tutorial: This is how I was planning to do this. Basically have 1 crate which follows the tutorial completely and use that in all .md files for code snippets. This way we would automatically have all the tests in place as add tests is usually one of the steps in a tutorial. Also gets rid of code duplication and leaves less stuff for us to maintain. This is the approach I would recommend following. What do you think? @0xLucca @kianenigma

I understand your concern but approach 2, while reducing duplication, wouldn't achieve one of the key goals of the tutorial - ensuring that the code shown at each step is exactly what developers need and can compile independently.

The main issue with copying the tests from the pallet-unit-testing crate to intermediate steps is that those tests likely depend on features and code that aren't introduced until later steps.

Adding these tests would require including additional dependencies and code that we deliberately haven't covered yet in the tutorial.

This would create a mismatch between what's shown in the documentation and what's actually needed to compile the code. This way, we cannot guarantee that the code shown at each step compiles independently

That being said, let's try to find the best way to ensure we meet both requirements. @UtkarshBhardwaj007 What do you think about this?

First off, I think ti is a great idea to have each step of the tutorial to conclude with:

you can find the entire code for this step of the tutorial here.

I couldn't form a strong opinion at first glance about which approach to take.

I can contribute by saying that our metrics for decision making should be:

All the code snippets rendered are correct

Have less maintenance burden on our side, by having fewer copy-pasted code.

It is nice-to-have if each section can exactly end with a copy of the code up until that step.

Criteria 2 and 3 are against one another, but I believe if we are more diligent and making sure what code snippets actually need to be pulled in (#302 (review)), we can have both.

So I would lean towards option 1.

wouldn't achieve one of the key goals of the tutorial - ensuring that the code shown at each step is exactly what developers need and can compile independently.

I don't think that individual compilation makes sense. These tutorial steps build on top of each other. For example, it doesn't make sense that someone would want to compile the pallet-unit-testing code independently without the build-a-custom-pallet code as it tests the very code in build-a-custom-pallet. What we should ensure is that all the steps work and compile at the end of each step. Having 1 crate for the tutorial would ensure that all the steps work and tie together as expected.

The main issue with copying the tests from the pallet-unit-testing crate to intermediate steps is that those tests likely depend on features and code that aren't introduced until later steps. Adding these tests would require including additional dependencies and code that we deliberately haven't covered yet in the tutorial.

pallet-unit-testing adds tests for the custom-pallet which is built in the first tutorial after setting-up-a-template. These tests don't depend on anything but the build-a-custom-pallet code. So adding these tests shouldn't cause any issues. The only extra dependencies we would be bringing into build-a-custom-pallet code would be the testing dependencies and it should be trivial to use mkdocs to not display these dependencies in earlier parts of the tutorials (for example how it is done in the build-a-custom-pallet prior to this PR.

This would create a mismatch between what's shown in the documentation and what's actually needed to compile the code. This way, we cannot guarantee that the code shown at each step compiles independently

Like I said above, it doesn't make sense that these steps compile independently. The tutorial has the following steps:

Set Up a Template: Clones the template and installs some cargo binaries.

Build a Custom Pallet: Build the actual pallet. If the code compiles till here, we can be sure that the first 2 steps are okay.

Pallet Unit Testing: Wouldn't make sense to compile independently anyway. You need the build-a-custom-pallet code. Hence suggesting the 1-crate-approach.

Benchmarking: Wouldn't make sense to compile independently anyway. You need the build-a-custom-pallet code. Hence suggesting the 1-crate-approach.

Add Pallets to the Runtime: You need a lot of previous code from set-up-a-template to compile this step as well.

Deploy to Paseo TestNet: no significant associated code

Obtain Coretime: no significant associated code

Basically having 1 crate would ensure that everything compiles not only individually (where it makes sense) but also the whole tutorial compiles together in the end and works as expected. What do you think? @0xLucca

I can contribute by saying that our metrics for decision making should be:

All the code snippets rendered are correct

Have less maintenance burden on our side, by having fewer copy-pasted code.

It is nice-to-have if each section can exactly end with a copy of the code up until that step.

For 2: I think having 1 crate like I am proposing above should solve that.
For 1&3: We will have to use mkdocs to selectively hide and show any lines that we want like in this PR. This should be doable unless function signatures or similar things change in the code anywhere in the tutorial (I don't think they do).

@kianenigma would you agree?

UtkarshBhardwaj007 · 2025-01-10T14:44:54Z

.snippets/code/tutorials/polkadot-sdk/parachains/zero-to-hero/pallet-unit-testing/src/lib.rs

@@ -0,0 +1,190 @@
+#![cfg_attr(not(feature = "std"), no_std)]


Code files should have license headers. See here.

UtkarshBhardwaj007 · 2025-01-10T14:53:10Z

tutorials/polkadot-sdk/parachains/zero-to-hero/pallet-benchmarking.md

@@ -5,8 +5,7 @@ description: Learn how to benchmark Polkadot SDK-based pallets, assigning precis

 ## Introduction

-After implementing and testing your pallet with a mock runtime in the [Pallet Unit Testing
-](/tutorials/polkadot-sdk/parachains/zero-to-hero/pallet-unit-testing/){target=\_blank} tutorial, the next crucial step is benchmarking. Benchmarking assigns precise [weight](/polkadot-protocol/glossary/#weight){target=\_blank} to each extrinsic, measuring their computational and storage costs. These derived weights enable accurate fee calculation and resource allocation within the runtime.


Worth keeping this line?

Benchmarking assigns precise [weight](/polkadot-protocol/glossary/#weight){target=\_blank} to each extrinsic, measuring their computational and storage costs. These derived weights enable accurate fee calculation and resource allocation within the runtime.

UtkarshBhardwaj007 · 2025-01-10T14:57:53Z

tutorials/polkadot-sdk/parachains/zero-to-hero/pallet-benchmarking.md

        ```

    2. Enable runtime benchmarking for your pallet in `runtime/Cargo.toml`:
        ```toml hl_lines="25"
-        --8<-- 'code/tutorials/polkadot-sdk/parachains/zero-to-hero/pallet-benchmarking/runtime-cargo.toml:136:161'


Also need to update this in Step 4b. mkdocs serve shows a blank output there.

UtkarshBhardwaj007 · 2025-01-10T15:05:52Z

tutorials/polkadot-sdk/parachains/zero-to-hero/pallet-benchmarking.md

        ```

 3. Add your pallet to the runtime's benchmark configuration:
    1.  Register your pallet in `runtime/src/benchmarks.rs`:
        ```rust hl_lines="11"
-        --8<-- 'code/tutorials/polkadot-sdk/parachains/zero-to-hero/pallet-benchmarking/benchmarks.rs:26:37'


This part of the tutorial seems to be changing the directory structure of the tutorial without giving any information. The last step before this step was pallet-unit-testing. This step should build on the directory structure that we ended up with after completing the pallet-unit-testing tutorial.

I think it is really good to have benchmarking as part of this tutorial to cover all steps of creating a launch ready custom pallet. We just need to be careful to build on the previous step to ensure consistency.

kianenigma · 2025-01-10T15:51:16Z

.snippets/code/Cargo.toml

-frame-system = { version = "38.0.0", default-features = false }
-scale-info = { version = "2.11.1", default-features = false }
-sp-runtime = { version = "39.0.5", default-features = false }
+color-print = { version = "0.3.4" }


FYI, if you use the umbrella crate here, you will have a much easier time updating dependencies.

kianenigma · 2025-01-10T15:52:21Z

...polkadot-sdk/parachains/zero-to-hero/add-pallets-to-runtime/pallets/custom-pallet/src/lib.rs

+#[cfg(test)]
+mod tests;
+
+#[frame_support::pallet(dev_mode)]


is this being noted somewhere? dev_mode removes some constraints from FRAME macros, but then if you take the same code to a parachain template and try and apply it, you will get a bunch of compiler errors.

I think the pallet actually doesn't need this -- you are already declaring weight and storage hashers, which are the main two things it relaxes.

kianenigma · 2025-01-10T15:56:29Z

...s/polkadot-sdk/parachains/zero-to-hero/pallet-benchmarking/runtime/src/configs/xcm_config.rs

@@ -0,0 +1,193 @@
+use crate::{


Double checking: are all of these files actually needed?

It seems like this step of the tutorial is about benchmarking.

kianenigma

I see a LOT of code being pulled here, sometimes multiple copies of the same. We need to double check if this is the right pattern.

I mean, why do we need code related to XCM config, a full working runtime and so on here?

Let's establish a framework around how we should do this task that is sane, minimal but still achieves to goal.

Imagine you have the old school doc in front of you, with a hardcoded code in the markdown file.

The question you should be asking is this:

What is the "smallest working rust project" that I can add, that can contain this snippet and make sure it is correct?

And only create that.

The only bottleneck/exception to this is that you might want to reference snippets of code in the original parachain template that this tutorial is based on, in which case we can embed the correct version of this template that we are relying on as a git submodule to this repo. This will actually improve our flow, as it will be super clear: "this toturial is based on the parachain template commit XXX", and every time there is a new parachain template release, we can also update it here.

kianenigma · 2025-01-10T16:08:45Z

can we hide the extra lines in this with mkdocs preprocessors?

0xLucca added 6 commits January 6, 2025 10:48

Add rust-test

1c9c2a9

asd

70a36e5

Update build-custom-pallet

e6f2010

format toml

d77895b

Update add-pallets-to-runtime

231f17b

Remove testing workflow

1480b1c

0xLucca added 2 commits January 9, 2025 10:48

Merge branch 'master' into 0xlucca/update-code-snippets

209ea92

Update pallet-benchmarking

e15079f

0xLucca marked this pull request as ready for review January 9, 2025 15:34

0xLucca requested a review from a team as a code owner January 9, 2025 15:34

0xLucca requested a review from UtkarshBhardwaj007 January 9, 2025 15:35

0xLucca added 2 commits January 9, 2025 12:46

Fix docs

dac5d3e

Merge branch 'master' into 0xlucca/update-code-snippets

061aa1a

UtkarshBhardwaj007 reviewed Jan 10, 2025

View reviewed changes

kianenigma reviewed Jan 10, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update code snippets and move to rust workspace #302

Update code snippets and move to rust workspace #302

0xLucca commented Jan 8, 2025

0xLucca commented Jan 8, 2025

UtkarshBhardwaj007 commented Jan 8, 2025

UtkarshBhardwaj007 Jan 10, 2025 •

edited

Loading

0xLucca Jan 10, 2025

kianenigma Jan 10, 2025

kianenigma Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025 •

edited

Loading

UtkarshBhardwaj007 Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025

kianenigma Jan 10, 2025

kianenigma Jan 10, 2025

kianenigma Jan 10, 2025

kianenigma Jan 10, 2025

kianenigma left a comment •

edited

Loading

kianenigma commented Jan 10, 2025

Update code snippets and move to rust workspace #302

Are you sure you want to change the base?

Update code snippets and move to rust workspace #302

Conversation

0xLucca commented Jan 8, 2025

0xLucca commented Jan 8, 2025

UtkarshBhardwaj007 commented Jan 8, 2025

UtkarshBhardwaj007 Jan 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

UtkarshBhardwaj007 Jan 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kianenigma left a comment • edited Loading

Choose a reason for hiding this comment

kianenigma commented Jan 10, 2025

UtkarshBhardwaj007 Jan 10, 2025 •

edited

Loading

UtkarshBhardwaj007 Jan 10, 2025 •

edited

Loading

kianenigma left a comment •

edited

Loading