Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add max coinbase out sigops to CoinbaseOutputDataSize in TP protocol #86

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Fi3
Copy link
Contributor

@Fi3 Fi3 commented Jun 16, 2024

The template provide need a way to know how many sigops the pool want to include in the coinbase outupts, otherwise is not able to create a template that if mined will result in a valid block.

The template provide need a way to know how many sigops the pool want to
include in the coinbase outupts, otherwise is not able to create a
template that if mined will result in a valid block.
@Sjors
Copy link
Contributor

Sjors commented Jun 17, 2024

I can implement this on the template provider side.

Let's recommend 400 as the default, as that's what Bitcoin Core has been using for ages (nBlockSigOpsCost = 400; in src/node/miner.cpp). It can be 0 only if all coinbase payout addresses are taproot.

This will need some coordination in deployment, especially if we don't change the protocol version number (see #82).

@Fi3
Copy link
Contributor Author

Fi3 commented Jun 18, 2024

@Sjors I addressed your comments. When we have consensus on this I can implement it in SRI and you can add it to TP. I think the most contentious point here is how to handle versioning

@Sjors
Copy link
Contributor

Sjors commented Jun 18, 2024

I'll let you know here once I have a branch with this change in it for the Template Provider. If you can open a pull request on the SRI repo with the same change, we can test if they're compatible.

@Fi3 Fi3 force-pushed the AddSigOpsCount branch from 7662ddd to 5014666 Compare June 18, 2024 09:29
@Sjors
Copy link
Contributor

Sjors commented Jun 18, 2024

Let's recommend 400 as the default

This is still missing. Perhaps worded as:

Use 400 for optimal backwards compatibility with v1. Note that taproot outputs consume 0 sigops.

@Fi3
Copy link
Contributor Author

Fi3 commented Jun 18, 2024

Let's recommend 400 as the default

This is still missing. Perhaps worded as:

Use 400 for optimal backwards compatibility with v1. Note that taproot outputs consume 0 sigops.

not sure if we should recommend a default at all. If so I would go with 0.

@Sjors
Copy link
Contributor

Sjors commented Jun 18, 2024

0 will create an invalid block if you your coinbase output is non-taproot. And you won't find out until someone starts spamming the mempool with bare multisig: https://bitcoinexplainedpodcast.com/@nado/episodes/bitcoin-explained-76-stamps-and-the-invalid-block-caused-by-it

The reason I suggest 400 is because that's what getblocktemplate does, so every stratum v1 miner does it by default - unless they patched Bitcoin Core. This has never been configurable before.


| Field Name | Data Type | Description |
| ----------------------------------- | --------- | ----------------------------------------------------------------------------------------------- |
| coinbase_output_max_additional_size | U32 | The maximum additional serialized bytes which the pool will add in coinbase transaction outputs |
| coinbase_output_max_sigops | U16 | The maximum additional sigops which the pool will add in coinbase transaction outputs |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add _additional for consistency?

@Fi3
Copy link
Contributor Author

Fi3 commented Jun 18, 2024

0 will create an invalid block if you your coinbase output is non-taproot. And you won't find out until someone starts spamming the mempool with bare multisig: https://bitcoinexplainedpodcast.com/@nado/episodes/bitcoin-explained-76-stamps-and-the-invalid-block-caused-by-it

The reason I suggest 400 is because that's what getblocktemplate does, so every stratum v1 miner does it by default - unless they patched Bitcoin Core. This has never been configurable before.

what about: legacy pools should use 400, new pools should use 0 ?

@Sjors
Copy link
Contributor

Sjors commented Jun 18, 2024

new pools should use 0

That assumes new pools want to use Taproot, that seems too DEMANDing :-)

@Sjors
Copy link
Contributor

Sjors commented Jun 18, 2024

Here you go, completely untested: Sjors/bitcoin#45 (branch sjors/sv2-sigops)

ryanofsky added a commit to bitcoin/bitcoin that referenced this pull request Jul 18, 2024
…ptions

c504b69 refactor: add coinbase constraints to BlockCreateOptions (Sjors Provoost)
6b4c817 refactor: pass BlockCreateOptions to createNewBlock (Sjors Provoost)
323cfed refactor: use CHECK_NONFATAL to avoid single-use symbol (Sjors Provoost)

Pull request description:

  When generating a block template through e.g. getblocktemplate RPC, we reserve 4000 weight units and 400 sigops. Pools use this space for their coinbase outputs.

  At least one pool patched their Bitcoin Core node to adjust these hardcoded values. They eventually [produced an invalid block](https://bitcoin.stackexchange.com/questions/117837/how-many-sigops-are-in-the-invalid-block-783426) which exceeded the sigops limit.

  The existince of such patches suggests it may be useful to make this value configurable. This PR would make such a change easier. However, the main motivation is that in the Stratum v2 spec requires the pool to communicate the maximum bytes they intend
  to add to the coinbase outputs.

  Specifically the `CoinbaseOutputDataSize` message which is part of the [Template Distribution Protocol](https://github.com/stratum-mining/sv2-spec/blob/main/07-Template-Distribution-Protocol.md#71-coinbaseoutputdatasize-client---server) has a field `coinbase_output_max_additional_size`.

  A proposed change to the spec adds the max additional sigops as well: stratum-mining/sv2-spec#86. Whether that change makes it into the spec is not important though, as adding both to `BlockAssembler::Options` makes sense.

  The first commit is a test refactor followup for #30335, related to the code that's changed here, but not required.

  The second commit introduces BlockCreateOptions, with just `use_mempool`.

  The thirds commit adds `coinbase_max_additional_weight` and `coinbase_output_max_additional_sigops` to  `BlockCreateOptions`. They use the originally hardcoded values, and no existing caller overrides these defaults. This changes in #29432.

ACKs for top commit:
  itornaza:
    tested ACK c504b69
  ryanofsky:
    Code review ACK c504b69
  ismaelsadeeq:
    Code review ACK c504b69

Tree-SHA512: de2fa085f47048c91d95524e03f909f6f27f175c1fefa3d6106445e7eb5cf5b710eda6ea5b641cf3b4704a4e4e0181a0c829003b9fd35465f2a46167e5d64487
@GitGab19
Copy link
Collaborator

@Fi3 how a JDC could know how many sigops has to signal to its TP? Shouldn't it receive them in the AllocateMiningJobToken.Success?

@Fi3
Copy link
Contributor Author

Fi3 commented Oct 30, 2024

@Fi3 how a JDC could know how many sigops has to signal to its TP? Shouldn't it receive them in the AllocateMiningJobToken.Success?

if it have the outputs it can parse them I'm wondering how it communicate it to the TP btw

@plebhash
Copy link
Contributor

@Fi3 how a JDC could know how many sigops has to signal to its TP? Shouldn't it receive them in the AllocateMiningJobToken.Success?

if it have the outputs it can parse them I'm wondering how it communicate it to the TP btw

JDS informs JDC about coinbase_output_max_additional_size via AllocateMiningJobToken.Success

then, JDC informs TP about coinbase_output_max_additional_size via CoinbaseOutputDataSize


this PR proposes introducing coinbase_output_max_sigops into CoinbaseOutputDataSize, while renaming the message into CoinbaseOutputConstraints

this means JDC is going to inform TP about coinbase_output_max_sigops via CoinbaseOutputConstraints

@GitGab19 question is: how is JDC informed about coinbase_output_max_sigops?

to me, the most logical answer simply follows the same pattern as coinbase_output_max_additional_size:

JDS informs JDC about coinbase_output_max_sigops via AllocateMiningJobToken.Success


therefore, IMO this PR should also propose adding a new coinbase_output_max_sigops field to AllocateMiningJobToken.Success

@Sjors
Copy link
Contributor

Sjors commented Oct 30, 2024

this PR should also propose

That makes sense.

The Mining interface in Bitcoin Core already supports passing passing coinbase_output_max_sigops to it along with coinbase_output_max_additional_size:

https://github.com/bitcoin/bitcoin/blob/97b790e844abd2f92c928239a7dc786d37fad18b/src/interfaces/mining.h#L88-L95

https://github.com/bitcoin/bitcoin/blob/97b790e844abd2f92c928239a7dc786d37fad18b/src/node/types.h#L30-L46

So this spec change won't require a change to the interface, which means there's no rush to get it finalised before the v29 Bitcoin Core release (which hopefully ships this interface).

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 6, 2024

Related to this, I realized a while ago that NewTemplate message contains coinbase_outputs field, but actually that is not set by TP, because it doesn't know the pool_outputs to add. Why couldn't we send the coinbase_outputs to TP, so that it can create a template with values already placed? Isn't this point related to this PR? What am I missing?

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 6, 2024

Related to this, I realized a while ago that NewTemplate message contains coinbase_outputs field, but actually that is not set by TP, because it doesn't know the pool_outputs to add. Why couldn't we send the coinbase_outputs to TP, so that it can create a template with values already placed? Isn't this point related to this PR? What am I missing?

I'm pretty sure that is setted by TP. That field should contain the output with the segwit commitment (only the TP know it if we don't request txs data)

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 6, 2024

Related to this, I realized a while ago that NewTemplate message contains coinbase_outputs field, but actually that is not set by TP, because it doesn't know the pool_outputs to add. Why couldn't we send the coinbase_outputs to TP, so that it can create a template with values already placed? Isn't this point related to this PR? What am I missing?

I'm pretty sure that is setted by TP. That field should contain the output with the segwit commitment (only the TP know it if we don't request txs data)

Actually it's not (at least from my findings) because they are never communicated to the TP. How could it know them by itself?
Here's the point in which the outputs are set in the jobs: https://github.com/stratum-mining/stratum/blob/73f51dba0d637f13a9dfb1fd7e4c3a7b58f59007/protocols/v2/roles-logic-sv2/src/job_creator.rs#L177

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 6, 2024

Related to this, I realized a while ago that NewTemplate message contains coinbase_outputs field, but actually that is not set by TP, because it doesn't know the pool_outputs to add. Why couldn't we send the coinbase_outputs to TP, so that it can create a template with values already placed? Isn't this point related to this PR? What am I missing?

I'm pretty sure that is setted by TP. That field should contain the output with the segwit commitment (only the TP know it if we don't request txs data)

Actually it's not (at least from my findings) because they are never communicated to the TP. How could it know them by itself? Here's the point in which the outputs are set in the jobs: https://github.com/stratum-mining/stratum/blob/73f51dba0d637f13a9dfb1fd7e4c3a7b58f59007/protocols/v2/roles-logic-sv2/src/job_creator.rs#L177

I'm pretty sure that you are wrong. Try to link that a JDC to a TP that is synced on testnet or mainnet and print that field

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 6, 2024

I did and I was not able to decode any tx output from those bytes. Can you tell me which message is used to communicate to TP the pool outputs?

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 6, 2024

I did and I was not able to decode any tx output from those bytes. Can you tell me which message is used to communicate to TP the pool outputs?

you don't have to communicate anyhting to the TP he already know what to put in there

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 6, 2024

How does it know by itself? I'm talking about the pool coinbase outputs which needs to be put into the coinbase tx

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 6, 2024

How does it know by itself? I'm talking about the pool coinbase outputs which needs to be put into the coinbase tx

aaaa ok. No we don't need the pool coinbase output there, the only thing that tp need to know is how much should be reserved for the pool's coinbase outputs. That field is for the output with the segwit commitment.

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 6, 2024

But the coinbase_tx_outputs field contained in the NewTemplate message is not used anywhere. The Pool (or JDC) use the coinbase_tx_outputs configured in the Pool (or the ones communicated by JDS). Am I missing something here? Shouldn't TP directly put the real coinbase_tx_outputs in that field (the one in NewTemplate message)?

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 6, 2024

But the coinbase_tx_outputs field contained in the NewTemplate message is not used anywhere. The Pool (or JDC) use the coinbase_tx_outputs configured in the Pool (or the ones communicated by JDS). Am I missing something here? Shouldn't TP directly put the real coinbase_tx_outputs in that field (the one in NewTemplate message)?

no it work like that, when you receive the template an you create a job you create a coinbase that have as outputs the pool outputs + the tp outputs
TP outputs must be placed at the end. Cause it cointains the output with the segwit commitment that must be the last output of the coinbase

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 6, 2024

you don't see it explicitly in the roles code, cause this is something that who develop a role do not need to worry about, if I rember correctly is done somwhere in the roles-logic library

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 7, 2024

you don't see it explicitly in the roles code, cause this is something that who develop a role do not need to worry about, if I rember correctly is done somwhere in the roles-logic library

I just found this comment on the function which computes the coinbase tx:
/// It assume that NewTemplate.coinbase_tx_outputs == 0

https://github.com/stratum-mining/stratum/blob/73f51dba0d637f13a9dfb1fd7e4c3a7b58f59007/protocols/v2/roles-logic-sv2/src/job_creator.rs#L323

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 7, 2024

you don't see it explicitly in the roles code, cause this is something that who develop a role do not need to worry about, if I rember correctly is done somwhere in the roles-logic library

I just found this comment on the function which computes the coinbase tx: /// It assume that NewTemplate.coinbase_tx_outputs == 0

https://github.com/stratum-mining/stratum/blob/73f51dba0d637f13a9dfb1fd7e4c3a7b58f59007/protocols/v2/roles-logic-sv2/src/job_creator.rs#L323

must be an old comment not updated?

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 7, 2024

I don't think so, because as I was saying earlier, the coinbase_tx_outputs field contained in the NewTemplate message is not used anywhere

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 7, 2024

I don't think so, because as I was saying earlier, the coinbase_tx_outputs field contained in the NewTemplate message is not used anywhere

    /// used to create new jobs when a new template arrives
    pub fn on_new_template(
        &mut self,
        template: &mut NewTemplate,
        version_rolling_allowed: bool,
        mut pool_coinbase_outputs: Vec<TxOut>,
        pool_signature: String,
    ) -> Result<NewExtendedMiningJob<'static>, Error> {
        let server_tx_outputs = template.coinbase_tx_outputs.to_vec();
        let mut outputs = tx_outputs_to_costum_scripts(&server_tx_outputs);
        pool_coinbase_outputs.append(&mut outputs);

        // This is to make sure that 0 is never used, so we can use 0 for
        // set_new_prev_hashes that do not refer to any future job/template if needed
        // Then we will do the inverse (-1) where needed
        let template_id = template.template_id + 1;
        self.lasts_new_template.push(template.as_static());
        let next_job_id = self.ids.next();
        self.job_to_template_id.insert(next_job_id, template_id);
        self.templte_to_job_id.insert(template_id, next_job_id);
        new_extended_job(
            template,
            &mut pool_coinbase_outputs,
            pool_signature,
            next_job_id,
            version_rolling_allowed,
            self.extranonce_len,
        )
    }

should be used here
didn't checked no time now

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 7, 2024

Thank you for pointing it out, I was missing it! So it is definitely used 👍
Why couldn't the TP directly send out the final coinbase_outputs? Instead of building it there? It would be a much cleaner approach imo

@Fi3
Copy link
Contributor Author

Fi3 commented Nov 7, 2024

Thank you for pointing it out, I was missing it! So it is definitely used 👍 Why couldn't the TP directly send out the final coinbase_outputs? Instead of building it there? It would be a much cleaner approach imo

I don't see it as something bad is pretty clean IMO btw I didn't write the spec so don't know.My best guess is to keep the TP logic as simple as possible. Maybe @TheBlueMatt can help here

@TheBlueMatt
Copy link

The TP protocol is designed to be as unidirectional as possible - it doesn't think it just dumps work. It isn't fully unidirectional as it considers the size of coinbase the client wants, but that's the best we can do.

Passing the coinbase tx output set from the JDS to the TP just to have the TP echo then back to the client doesn't accomplish anything, IMO, and reduces the unidirectionality goal. It's trivial to just have the TP+JD client figure it out.

@GitGab19
Copy link
Collaborator

GitGab19 commented Nov 7, 2024

The TP protocol is designed to be as unidirectional as possible - it doesn't think it just dumps work. It isn't fully unidirectional as it considers the size of coinbase the client wants, but that's the best we can do.

Passing the coinbase tx output set from the JDS to the TP just to have the TP echo then back to the client doesn't accomplish anything, IMO, and reduces the unidirectionality goal. It's trivial to just have the TP+JD client figure it out.

Makes sense, thanks for your answer!

@Sjors
Copy link
Contributor

Sjors commented Nov 8, 2024

TP outputs must be placed at the end. Cause it cointains the output with the segwit commitment that must be the last output of the coinbase

I don't think it needs to be the last one, just needs to have the right magic bytes.
https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure

However it may still be a sane default to put it last:

If there are more than one scriptPubKey matching the pattern, the one with highest output index is assumed to be the commitment.

@TheBlueMatt
Copy link

Yea, the design of Sv2 basically assumes that all things that require commitments (future soft forks and/or merged-mined chains) will do the same (or use the segwit commitment).

@@ -38,7 +38,7 @@ Notably, if the pool intends to change the space it requires for coinbase transa
| ----------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| request_id | U32 | Unique identifier for pairing the response |
| mining_job_token | B0_255 | Token that makes the client eligible for committing a mining job for approval/transaction declaration or for identifying custom mining job on mining connection. |
| coinbase_output_max_additional_size | U32 | The maximum additional serialized bytes which the pool will add in coinbase transaction outputs. See discussion in the Template Distribution Protocol's CoinbaseOutputDataSize message for more details. |
| coinbase_output_max_additional_size | U32 | The maximum additional serialized bytes which the pool will add in coinbase transaction outputs. See discussion in the Template Distribution Protocol's CoinbaseOutputConstraints message for more details. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as pointed out here #86 (comment)

we need to add coinbase_output_max_sigops to AllocateMiningJobToken.Success

@@ -38,7 +38,7 @@ Notably, if the pool intends to change the space it requires for coinbase transa
| ----------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| request_id | U32 | Unique identifier for pairing the response |
| mining_job_token | B0_255 | Token that makes the client eligible for committing a mining job for approval/transaction declaration or for identifying custom mining job on mining connection. |
| coinbase_output_max_additional_size | U32 | The maximum additional serialized bytes which the pool will add in coinbase transaction outputs. See discussion in the Template Distribution Protocol's CoinbaseOutputDataSize message for more details. |
| coinbase_output_max_additional_size | U32 | The maximum additional serialized bytes which the pool will add in coinbase transaction outputs. See discussion in the Template Distribution Protocol's CoinbaseOutputConstraints message for more details. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| coinbase_output_max_additional_size | U32 | The maximum additional serialized bytes which the pool will add in coinbase transaction outputs. See discussion in the Template Distribution Protocol's CoinbaseOutputConstraints message for more details. |
| coinbase_output_max_additional_size | U32 | The maximum additional serialized bytes which the pool will add in coinbase transaction outputs. See discussion in the Template Distribution Protocol's CoinbaseOutputConstraints message for more details. |
| coinbase_output_max_additional_sigops | U16 | The maximum additional sigops which the pool will add in coinbase transaction outputs. See discussion in the Template Distribution Protocol's CoinbaseOutputConstraints message for more details. |


| Field Name | Data Type | Description |
| ----------------------------------- | --------- | ----------------------------------------------------------------------------------------------- |
| coinbase_output_max_additional_size | U32 | The maximum additional serialized bytes which the pool will add in coinbase transaction outputs |
| coinbase_output_max_sigops | U16 | The maximum additional sigops which the pool will add in coinbase transaction outputs |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| coinbase_output_max_sigops | U16 | The maximum additional sigops which the pool will add in coinbase transaction outputs |
| coinbase_output_max_additional_sigops | U16 | The maximum additional sigops which the pool will add in coinbase transaction outputs |

@Sjors
Copy link
Contributor

Sjors commented Dec 16, 2024

I rebased Sjors/bitcoin#45, so you can test it against a modified SRI.

@Sjors
Copy link
Contributor

Sjors commented Dec 16, 2024

I added a commit that makes the Template Provider backwards compatible, it will just assume max additional sigops is 400 if the field is missing.

Suggested deployment strategy:

  1. Approve, but not merge, this spec change PR
  2. I'll make a new Template Provider release which includes Add max coinbase out sigops to CoinbaseOutputDataSize Sjors/bitcoin#45
  3. Update SRI to set the field
  4. Release new SRI version (maybe have instructions specify a minimum TP release)
  5. Merge this spec change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants