Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test JDC Pool Fallback #1343

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Test JDC Pool Fallback #1343

wants to merge 7 commits into from
+232 −453

Conversation

jbesraa
Copy link
Contributor

@jbesraa jbesraa commented Jan 9, 2025 •
edited
Loading

resolves #1207

blocked by #1319 & #1321

@jbesraa jbesraa mentioned this pull request Jan 9, 2025
Open
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branch2025-01-09-jdc-fallback-test
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		44.89
(-1.21%)		61.49
(73.00%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		72.61
(-4.54%)		102.28
(71.00%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		14.71
(+0.28%)		14.74
(99.78%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		251.55
(-4.13%)		287.11
(87.62%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		632.16
(+1.92%)		685.84
(92.17%)
client_sv2_open_channel📈 view plot
🚷 view threshold		167.04
(+0.66%)		179.49
(93.06%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		283.37
(-1.18%)		317.67
(89.20%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		379.83
(-0.69%)		399.10
(95.17%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		156.67
(-2.00%)		171.38
(91.42%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		502.06
(+6.51%)		561.15
(89.47%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		988.84
(-2.15%)		1,244.78
(79.44%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branch2025-01-09-jdc-fallback-test
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,595.30
(+0.53%)		6,992.43
(94.32%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,483.40
(+0.92%)		7,879.81
(94.97%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		8,069.00
(-0.35%)		9,550.90
(84.48%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		866.94
(-0.28%)		951.23
(91.14%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		661.53
(-2.37%)		730.39
(90.57%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		250.13
(-0.52%)		281.60
(88.82%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		157.87
(-0.21%)		170.07
(92.83%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,358.80
(+0.29%)		6,756.96
(94.11%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		321.16
(+10.56%)		388.46
(82.67%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		720.08
(-1.03%)		781.36
(92.16%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		615.72
(+3.63%)		648.21
(94.99%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		206.07
(-0.45%)		229.10
(89.95%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branch2025-01-09-jdc-fallback-test
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
1e3 x estimated cycles
(Result Δ%)		Upper Boundary
1e3 x estimated cycles
(Limit %)		InstructionsBenchmark Result
1e3 x instructions
(Result Δ%)		Upper Boundary
1e3 x instructions
(Limit %)		L1 AccessesBenchmark Result
1e3 x accesses
(Result Δ%)		Upper Boundary
1e3 x accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold		8.39
(-0.70%)		8.67
(96.80%)		📈 view plot
🚷 view threshold		3.69
(-0.92%)		3.86
(95.54%)		📈 view plot
🚷 view threshold		5.16
(-1.08%)		5.45
(94.65%)		📈 view plot
🚷 view threshold		11.00
(+28.70%)		16.47
(66.77%)		📈 view plot
🚷 view threshold		91.00
(-0.48%)		96.80
(94.01%)
get_submit📈 view plot
🚷 view threshold		95.19
(-0.18%)		95.62
(99.55%)		📈 view plot
🚷 view threshold		59.35
(-0.09%)		59.71
(99.39%)		📈 view plot
🚷 view threshold		85.22
(-0.10%)		85.82
(99.30%)		📈 view plot
🚷 view threshold		47.00
(+4.64%)		60.46
(77.73%)		📈 view plot
🚷 view threshold		278.00
(-0.98%)		291.88
(95.24%)
get_subscribe📈 view plot
🚷 view threshold		7.88
(-1.40%)		8.23
(95.75%)		📈 view plot
🚷 view threshold		2.77
(-1.63%)		2.94
(93.98%)		📈 view plot
🚷 view threshold		3.85
(-1.92%)		4.14
(92.83%)		📈 view plot
🚷 view threshold		16.00
(+26.73%)		20.95
(76.39%)		📈 view plot
🚷 view threshold		113.00
(-1.33%)		117.94
(95.81%)
serialize_authorize📈 view plot
🚷 view threshold		12.12
(-1.11%)		12.51
(96.91%)		📈 view plot
🚷 view threshold		5.27
(-0.58%)		5.43
(97.05%)		📈 view plot
🚷 view threshold		7.33
(-0.70%)		7.60
(96.36%)		📈 view plot
🚷 view threshold		14.00
(+35.55%)		19.02
(73.62%)		📈 view plot
🚷 view threshold		135.00
(-2.14%)		143.36
(94.17%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold		24.64
(-0.31%)		25.19
(97.82%)		📈 view plot
🚷 view threshold		9.84
(-0.18%)		10.01
(98.28%)		📈 view plot
🚷 view threshold		13.88
(-0.22%)		14.17
(97.89%)		📈 view plot
🚷 view threshold		38.00
(+5.60%)		45.99
(82.62%)		📈 view plot
🚷 view threshold		302.00
(-0.54%)		313.67
(96.28%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold		30.11
(-0.69%)		30.73
(97.99%)		📈 view plot
🚷 view threshold		12.02
(-0.28%)		12.19
(98.60%)		📈 view plot
🚷 view threshold		17.00
(-0.33%)		17.30
(98.29%)		📈 view plot
🚷 view threshold		60.00
(+7.37%)		67.53
(88.85%)		📈 view plot
🚷 view threshold		366.00
(-1.34%)		379.56
(96.43%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold		126.28
(-0.14%)		126.79
(99.60%)		📈 view plot
🚷 view threshold		73.20
(-0.06%)		73.53
(99.55%)		📈 view plot
🚷 view threshold		104.91
(-0.08%)		105.51
(99.43%)		📈 view plot
🚷 view threshold		116.00
(+8.51%)		126.06
(92.02%)		📈 view plot
🚷 view threshold		594.00
(-0.63%)		610.68
(97.27%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold		27.74
(-0.60%)		28.38
(97.73%)		📈 view plot
🚷 view threshold		9.58
(-0.47%)		9.76
(98.21%)		📈 view plot
🚷 view threshold		13.54
(-0.56%)		13.84
(97.82%)		📈 view plot
🚷 view threshold		69.00
(+6.74%)		78.13
(88.31%)		📈 view plot
🚷 view threshold		396.00
(-0.81%)		409.80
(96.63%)
serialize_deserialize_submit📈 view plot
🚷 view threshold		115.14
(-0.09%)		115.71
(99.51%)		📈 view plot
🚷 view threshold		68.06
(+0.01%)		68.42
(99.47%)		📈 view plot
🚷 view threshold		97.65
(+0.00%)		98.29
(99.35%)		📈 view plot
🚷 view threshold		69.00
(+4.93%)		87.84
(78.56%)		📈 view plot
🚷 view threshold		490.00
(-0.70%)		505.36
(96.96%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold		23.18
(-0.61%)		23.82
(97.32%)		📈 view plot
🚷 view threshold		8.14
(-0.50%)		8.32
(97.93%)		📈 view plot
🚷 view threshold		11.45
(-0.60%)		11.75
(97.48%)		📈 view plot
🚷 view threshold		43.00
(+9.90%)		51.32
(83.79%)		📈 view plot
🚷 view threshold		329.00
(-0.80%)		342.46
(96.07%)
serialize_submit📈 view plot
🚷 view threshold		99.57
(-0.21%)		100.09
(99.48%)		📈 view plot
🚷 view threshold		61.41
(-0.07%)		61.73
(99.47%)		📈 view plot
🚷 view threshold		88.08
(-0.08%)		88.65
(99.36%)		📈 view plot
🚷 view threshold		51.00
(+5.56%)		67.16
(75.93%)		📈 view plot
🚷 view threshold		321.00
(-1.35%)		337.65
(95.07%)
serialize_subscribe📈 view plot
🚷 view threshold		11.32
(-0.68%)		11.60
(97.62%)		📈 view plot
🚷 view threshold		4.12
(-1.01%)		4.28
(96.16%)		📈 view plot
🚷 view threshold		5.71
(-1.25%)		6.00
(95.16%)		📈 view plot
🚷 view threshold		17.00
(+20.96%)		24.45
(69.53%)		📈 view plot
🚷 view threshold		158.00
(-0.35%)		164.64
(95.97%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branch2025-01-09-jdc-fallback-test
Testbedsv2
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
1e3 x estimated cycles
(Result Δ%)		Upper Boundary
1e3 x estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		2.13
(+0.64%)		2.24
(95.09%)		📈 view plot
🚷 view threshold		473.00
(-0.14%)		490.49
(96.43%)		📈 view plot
🚷 view threshold		734.00
(-0.27%)		758.91
(96.72%)		📈 view plot
🚷 view threshold		6.00
(+16.72%)		11.61
(51.69%)		📈 view plot
🚷 view threshold		39.00
(+0.83%)		41.76
(93.39%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		8.32
(+1.14%)		8.40
(99.04%)		📈 view plot
🚷 view threshold		2,137.00📈 view plot
🚷 view threshold		3,152.00
(-0.21%)		3,167.46
(99.51%)		📈 view plot
🚷 view threshold		40.00
(+12.23%)		41.66
(96.02%)		📈 view plot
🚷 view threshold		142.00
(+1.62%)		144.63
(98.18%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		6.32
(+0.25%)		6.45
(97.97%)		📈 view plot
🚷 view threshold		1,750.00
(-0.04%)		1,767.49
(99.01%)		📈 view plot
🚷 view threshold		2,549.00
(-0.12%)		2,575.33
(98.98%)		📈 view plot
🚷 view threshold		19.00
(+10.65%)		24.04
(79.04%)		📈 view plot
🚷 view threshold		105.00
(+0.27%)		108.80
(96.51%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		14.78
(+0.38%)		14.94
(98.96%)		📈 view plot
🚷 view threshold		4,694.00
(-0.01%)		4,711.49
(99.63%)		📈 view plot
🚷 view threshold		6,747.00
(-0.13%)		6,786.72
(99.41%)		📈 view plot
🚷 view threshold		53.00
(+15.39%)		63.46
(83.52%)		📈 view plot
🚷 view threshold		222.00
(+0.37%)		226.13
(98.17%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		27.75
(+0.53%)		28.07
(98.89%)		📈 view plot
🚷 view threshold		10,645.00
(+0.35%)		10,700.14
(99.48%)		📈 view plot
🚷 view threshold		15,504.00
(+0.40%)		15,596.70
(99.41%)		📈 view plot
🚷 view threshold		91.00
(+8.43%)		99.35
(91.59%)		📈 view plot
🚷 view threshold		337.00
(+0.42%)		343.24
(98.18%)
client_sv2_open_channel📈 view plot
🚷 view threshold		4.41
(+0.19%)		4.59
(96.22%)		📈 view plot
🚷 view threshold		1,461.00
(-0.05%)		1,478.49
(98.82%)		📈 view plot
🚷 view threshold		2,158.00
(-0.12%)		2,184.65
(98.78%)		📈 view plot
🚷 view threshold		10.00
(+21.44%)		14.00
(71.44%)		📈 view plot
🚷 view threshold		63.00
(+0.10%)		67.88
(92.82%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		14.03
(+0.06%)		14.20
(98.77%)		📈 view plot
🚷 view threshold		5,064.00
(-0.01%)		5,081.49
(99.66%)		📈 view plot
🚷 view threshold		7,323.00
(-0.04%)		7,352.71
(99.60%)		📈 view plot
🚷 view threshold		39.00
(+6.10%)		48.34
(80.67%)		📈 view plot
🚷 view threshold		186.00
(-0.00%)		190.86
(97.45%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		22.79
(+0.33%)		23.08
(98.75%)		📈 view plot
🚷 view threshold		8,040.00
(+0.09%)		8,057.61
(99.78%)		📈 view plot
🚷 view threshold		11,691.00
(+0.05%)		11,713.57
(99.81%)		📈 view plot
🚷 view threshold		84.00
(+10.62%)		90.02
(93.31%)		📈 view plot
🚷 view threshold		305.00
(+0.27%)		312.88
(97.48%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		4.68
(-0.33%)		4.79
(97.58%)		📈 view plot
🚷 view threshold		1,502.00
(-0.04%)		1,519.49
(98.85%)		📈 view plot
🚷 view threshold		2,277.00
(-0.08%)		2,300.85
(98.96%)		📈 view plot
🚷 view threshold		11.00
(+16.36%)		15.26
(72.07%)		📈 view plot
🚷 view threshold		67.00
(-0.90%)		70.17
(95.48%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		16.21
(+0.35%)		16.33
(99.29%)		📈 view plot
🚷 view threshold		5,963.00
(-0.01%)		5,980.49
(99.71%)		📈 view plot
🚷 view threshold		8,654.00
(-0.11%)		8,692.67
(99.56%)		📈 view plot
🚷 view threshold		49.00
(+19.97%)		56.50
(86.73%)		📈 view plot
🚷 view threshold		209.00
(+0.35%)		212.20
(98.49%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		35.71
(+0.28%)		35.96
(99.29%)		📈 view plot
🚷 view threshold		14,888.00
(+0.13%)		14,920.01
(99.79%)		📈 view plot
🚷 view threshold		21,871.00
(+0.12%)		21,919.67
(99.78%)		📈 view plot
🚷 view threshold		107.00
(+12.86%)		120.21
(89.01%)		📈 view plot
🚷 view threshold		380.00
(+0.09%)		385.75
(98.51%)
🐰 View full continuous benchmarking report in Bencher

@codecov Codecov
Copy link

codecov bot commented Jan 9, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 19.11%. Comparing base (4ad657a) to head (6a9de6c).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1343      +/-   ##
==========================================
+ Coverage   16.90%   19.11%   +2.21%     
==========================================
  Files         164      166       +2     
  Lines       10956    11047      +91     
==========================================
+ Hits         1852     2112     +260     
+ Misses       9104     8935     -169
Flag Coverage Δ
binary_codec_sv2-coverage 0.00% <ø> (ø)
binary_serde_sv2-coverage 3.56% <ø> (ø)
binary_sv2-coverage 5.36% <ø> (ø)
bip32_derivation-coverage 0.00% <ø> (?)
buffer_sv2-coverage 25.02% <ø> (?)
codec_sv2-coverage 0.01% <ø> (ø)
common_messages_sv2-coverage 0.13% <ø> (ø)
const_sv2-coverage 0.00% <ø> (ø)
error_handling-coverage 0.00% <ø> (?)
framing_sv2-coverage 0.28% <ø> (ø)
jd_client-coverage 0.00% <ø> (ø)
jd_server-coverage 7.79% <ø> (ø)
job_declaration_sv2-coverage 0.00% <ø> (ø)
key-utils-coverage 2.39% <ø> (ø)
mining-coverage 2.45% <ø> (ø)
mining_device-coverage 0.00% <ø> (ø)
mining_proxy_sv2-coverage 0.70% <ø> (ø)
noise_sv2-coverage 4.46% <ø> (ø)
pool_sv2-coverage 2.05% <ø> (ø)
protocols 24.63% <ø> (ø)
roles 6.55% <ø> (ø)
roles_logic_sv2-coverage 7.95% <ø> (ø)
sv2_ffi-coverage 0.00% <ø> (ø)
template_distribution_sv2-coverage 0.00% <ø> (ø)
translator_sv2-coverage 9.60% <ø> (ø)
utils 25.13% <ø> (?)
v1-coverage 2.42% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Shourya742 and others added 5 commits January 13, 2025 14:27
@Shourya742 @jbesraa
fix sigterm issue with translator sv2
b8de13d
@Shourya742 @jbesraa
**Fix jdc sigterm signalling issue**
dd9b11b 
It changes the structuring of the start method, to avoid its blocking nature in case of
disconnection from the upstream. Currently, when we are sending the termination signal, during the
stance of disconnection from upstream, due of blocking nature of initialize_jd it halts the main
thread runtime from executing select block, which listens for any termination signal and channel
responses.

**Modifications**
1. Addition of new shutdown field, which can be used later in integration test to terminate the instances
2. Change argument type for methods initialize_jd and initialize_jd_solo, to make them movable.
3. Spawning of blocking process as a separate task
@jbesraa
Allow start_jdc to take multiple pool addresses
faae8c3
@jbesraa
Add sleep function
3c35cdd 
.. instead of repeated code
@jbesraa
Change JDC timeout config in tests
63e5b3d
@jbesraa jbesraa force-pushed the 2025-01-09-jdc-fallback-test branch from 65fcc64 to 5c367cd Compare January 14, 2025 11:27
@jbesraa jbesraa force-pushed the 2025-01-09-jdc-fallback-test branch from 5c367cd to b50e256 Compare January 14, 2025 11:30
@jbesraa jbesraa marked this pull request as ready for review January 14, 2025 11:31
jbesraa
jbesraa commented Jan 14, 2025
View reviewed changes
roles/tests-integration/tests/jdc_integration.rs
Comment on lines +27 to +38
Some(vec![InterceptMessage::new(
MessageDirection::ToDownstream,
MESSAGE_TYPE_MINING_SET_NEW_PREV_HASH,
PoolMessages::Mining(Mining::SubmitSharesError(SubmitSharesError {
channel_id: 0,
sequence_number: 0,
error_code: "invalid-nonce".to_string().into_bytes().try_into().unwrap(),
})),
MESSAGE_TYPE_SUBMIT_SHARES_ERROR,
false,
)]),
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Shourya742 @plebhash @GitGab19 I took a different approach here:
Instead of starting a miner and waiting for share submission and then success share submission message and altering it to error share submission, I just altered one of the previous messages(MINING_SET_NEW_PREV_HASH in this case). This is inspired by a discussion I had yesterday with @Shourya742 .
LMKWYT please.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CI migration: interop-jdc-change-upstream MG to Integration Test
2 participants
@jbesraa @Shourya742