Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ✨ remove file deletion requests expiration #311

Open
wants to merge 45 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
40164c9
feat: :construction: setup structure for file system benchmark scripts
TDemeco Dec 26, 2024
9894beb
feat: :sparkles: add generate file system benchmark proofs script
TDemeco Dec 27, 2024
2626579
feat: :sparkles: finish proof generator script for file system benchm…
TDemeco Dec 31, 2024
54d0ef8
Merge branch 'main' into feat/file-system-benchmark-script
TDemeco Dec 31, 2024
874b647
chore: :label: run typegen
TDemeco Dec 31, 2024
18afa70
feat: :test_tube: add benchmarking for msp respond storage requests
TDemeco Dec 31, 2024
7234d28
fix: :construction: fix invalid key proofs benchmarking
TDemeco Dec 31, 2024
20d2785
fix: :construction: add needed files to script to avoid failling proofs
TDemeco Jan 2, 2025
215138c
Merge branch 'main' into feat/file-system-benchmark-script
TDemeco Jan 2, 2025
10675e4
feat: :sparkles: finish benchmarking proofs script
TDemeco Jan 2, 2025
349365f
feat: :sparkles: add missing benchmark proofs + refactor tuple into s…
TDemeco Jan 3, 2025
29ced45
feat: :sparkles: finish benchmarking file system pallet
TDemeco Jan 6, 2025
1784ec1
fix: :adhesive_bandage: remove unused packages + run typegen
TDemeco Jan 6, 2025
38ce434
style: :rotating_light: run cargo fmt
TDemeco Jan 6, 2025
3f5af57
test: :white_check_mark: revert file key proof in insufficient BSP ca…
TDemeco Jan 6, 2025
d51f715
Merge branch 'main' into feat/file-system-benchmark-script
TDemeco Jan 6, 2025
a368044
style: :art: improve file system proof generating script's clarity
TDemeco Jan 6, 2025
c4ea8c1
Merge branch 'main' into feat/file-system-benchmark-script
TDemeco Jan 6, 2025
903a6f5
fix: :white_check_mark: fix merged file system benchmarks from main
TDemeco Jan 6, 2025
fb47d63
test: :white_check_mark: add unit tests for MSP used capacity update …
TDemeco Jan 7, 2025
0315b6d
chore: :label: run typegen
TDemeco Jan 7, 2025
a2cc5bf
feat: :sparkles: manage privileged providers list when deleting files
TDemeco Jan 7, 2025
178dad4
feat: :sparkles: remove expiration of pending file deletion requests
TDemeco Jan 7, 2025
40ced4a
fix: :white_check_mark: fix payment streams unit tests after change
TDemeco Jan 7, 2025
fa52b21
Merge branch 'feat/enforce-file-deletion-msps' into feat/remove-file-…
TDemeco Jan 7, 2025
367da03
feat: :sparkles: finish removing file deletion requests expirations +…
TDemeco Jan 7, 2025
a7643ad
style: :rotating_light: run cargo fmt
TDemeco Jan 7, 2025
7174efa
feat: :sparkles: add tracking of pending file deletion requests per MSP
TDemeco Jan 7, 2025
05ccf9c
Merge branch 'feat/enforce-file-deletion-msps' into feat/remove-file-…
TDemeco Jan 8, 2025
ef3be57
chore: :label: run typegen
TDemeco Jan 8, 2025
458ed2f
feat: :sparkles: benchmark on poll hook
TDemeco Jan 8, 2025
fe650eb
feat: :construction: add benchmarks for `on_idle` hook
TDemeco Jan 9, 2025
9c2c486
Merge branch 'main' into feat/file-system-benchmark-script
TDemeco Jan 9, 2025
4ae8278
chore: :label: run typegen
TDemeco Jan 9, 2025
41bd4ca
Merge branch 'feat/file-system-benchmark-script' into feat/enforce-fi…
TDemeco Jan 9, 2025
b4c77f8
Merge branch 'feat/enforce-file-deletion-msps' into feat/remove-file-…
TDemeco Jan 9, 2025
cd98685
chore: :label: run typegen
TDemeco Jan 9, 2025
b51c7d9
fix: :rotating_light: make fn public to the crate for benchmarking
TDemeco Jan 9, 2025
25a087b
Merge branch 'feat/file-system-benchmark-script' into feat/enforce-fi…
TDemeco Jan 9, 2025
f5bff0c
Merge branch 'feat/enforce-file-deletion-msps' into feat/remove-file-…
TDemeco Jan 9, 2025
01c6d9e
feat: :construction: set up `on_idle` hook to finish benchmarking aft…
TDemeco Jan 9, 2025
fb6af06
Merge branch 'feat/file-system-benchmark-script' into feat/enforce-fi…
TDemeco Jan 9, 2025
2916955
Merge branch 'feat/enforce-file-deletion-msps' into feat/remove-file-…
TDemeco Jan 9, 2025
be4cade
fix: :white_check_mark: change test after merge from main
TDemeco Jan 9, 2025
28de750
fix: :bug: change fn to pub(crate) since it got changed in the merge …
TDemeco Jan 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
255 changes: 129 additions & 126 deletions Cargo.lock

Large diffs are not rendered by default.

171 changes: 82 additions & 89 deletions api-augment/dist/interfaces/lookup.js

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion api-augment/dist/interfaces/lookup.js.map

Large diffs are not rendered by default.

4 changes: 0 additions & 4 deletions api-augment/dist/types/interfaces/augment-api-consts.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,6 @@ declare module "@polkadot/api-base/types/consts" {
* Time-to-live for a move bucket request, after which the request is considered expired.
**/
moveBucketRequestTtl: u32 & AugmentedConst<ApiType>;
/**
* Time-to-live for a pending file deletion request, after which a priority challenge is sent out to enforce the deletion.
**/
pendingFileDeletionRequestTtl: u32 & AugmentedConst<ApiType>;
/**
* Deposit held from the User when creating a new storage request
**/
Expand Down
20 changes: 0 additions & 20 deletions api-augment/dist/types/interfaces/augment-api-events.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -580,26 +580,6 @@ declare module "@polkadot/api-base/types/events" {
private: bool;
}
>;
/**
* Failed to decrease bucket size for expired file deletion request
**/
FailedToDecreaseBucketSize: AugmentedEvent<
ApiType,
[
user: AccountId32,
bucketId: H256,
fileKey: H256,
fileSize: u64,
error: SpRuntimeDispatchError
],
{
user: AccountId32;
bucketId: H256;
fileKey: H256;
fileSize: u64;
error: SpRuntimeDispatchError;
}
>;
/**
* Notifies that a priority challenge failed to be queued for pending file deletion.
**/
Expand Down
27 changes: 9 additions & 18 deletions api-augment/dist/types/interfaces/augment-api-query.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -305,17 +305,6 @@ declare module "@polkadot/api-base/types/storage" {
[H256, H256]
> &
QueryableStorageEntry<ApiType, [H256, H256]>;
/**
* A map of blocks to expired file deletion requests.
**/
fileDeletionRequestExpirations: AugmentedQuery<
ApiType,
(
arg: u32 | AnyNumber | Uint8Array
) => Observable<Vec<PalletFileSystemPendingFileDeletionRequest>>,
[u32]
> &
QueryableStorageEntry<ApiType, [u32]>;
/**
* Maximum number replication target allowed to be set for a storage request to be fulfilled.
**/
Expand All @@ -331,16 +320,18 @@ declare module "@polkadot/api-base/types/storage" {
> &
QueryableStorageEntry<ApiType, [u32]>;
/**
* A pointer to the earliest available block to insert a new file deletion request expiration.
* Mapping from MSPs to the amount of pending file deletion requests they have.
*
* This should always be greater or equal than current block + [`Config::PendingFileDeletionRequestTtl`].
* This is used to keep track of the amount of pending file deletion requests each MSP has, so that MSPs are removed
* from the privileged providers list if they have at least one, and are added back if they have none.
* This is to ensure that MSPs are correctly incentivised to submit the required proofs for file deletions.
**/
nextAvailableFileDeletionRequestExpirationBlock: AugmentedQuery<
mspsAmountOfPendingFileDeletionRequests: AugmentedQuery<
ApiType,
() => Observable<u32>,
[]
(arg: H256 | string | Uint8Array) => Observable<u32>,
[H256]
> &
QueryableStorageEntry<ApiType, []>;
QueryableStorageEntry<ApiType, [H256]>;
/**
* A pointer to the earliest available block to insert a new move bucket request expiration.
*
Expand Down Expand Up @@ -384,7 +375,7 @@ declare module "@polkadot/api-base/types/storage" {
/**
* Pending file deletion requests.
*
* A mapping from a user Account ID to a list of pending file deletion requests, holding a tuple of the file key, file size and Bucket ID.
* A mapping from a user Account ID to a list of pending file deletion requests (which have the file information).
**/
pendingFileDeletionRequests: AugmentedQuery<
ApiType,
Expand Down
18 changes: 18 additions & 0 deletions api-augment/dist/types/interfaces/augment-api-rpc.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1076,6 +1076,24 @@ declare module "@polkadot/rpc-core/types/jsonrpc" {
addToExcludeList: AugmentedRpc<
(file_key: H256 | string | Uint8Array) => Observable<ITuple<[]>>
>;
/**
* Generate a SCALE-encoded proof for a file key to allow a BSP to confirm storing it.
**/
generateFileKeyProofBspConfirm: AugmentedRpc<
(
bsp_id: H256 | string | Uint8Array,
file_key: H256 | string | Uint8Array
) => Observable<Bytes>
>;
/**
* Generate a SCALE-encoded proof for a file key to allow a MSP to accept storing it.
**/
generateFileKeyProofMspAccept: AugmentedRpc<
(
msp_id: H256 | string | Uint8Array,
file_key: H256 | string | Uint8Array
) => Observable<Bytes>
>;
/**
* Generate a SCALE-encoded proof for a group of file keys that might or might not be in the forest.
**/
Expand Down
29 changes: 13 additions & 16 deletions api-augment/dist/types/interfaces/augment-api-tx.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import type {
CumulusPrimitivesParachainInherentParachainInherentData,
PalletBalancesAdjustmentDirection,
PalletFileSystemBucketMoveRequestResponse,
PalletFileSystemFileKeyWithProof,
PalletFileSystemStorageRequestMspBucketResponse,
PalletNftsAttributeNamespace,
PalletNftsCancelAttributesApprovalWitness,
Expand All @@ -26,7 +27,6 @@ import type {
PalletNftsPreSignedMint,
PalletNftsPriceWithDirection,
PalletProofsDealerProof,
ShpFileKeyVerifierFileKeyProof,
SpRuntimeMultiSignature,
SpTrieStorageProofCompactProof,
SpWeightsWeightV2Weight,
Expand Down Expand Up @@ -524,21 +524,18 @@ declare module "@polkadot/api-base/types/submittable" {
| string
| Uint8Array,
fileKeysAndProofs:
| Vec<ITuple<[H256, ShpFileKeyVerifierFileKeyProof]>>
| [
H256 | string | Uint8Array,
(
| ShpFileKeyVerifierFileKeyProof
| {
fileMetadata?: any;
proof?: any;
}
| string
| Uint8Array
)
][]
) => SubmittableExtrinsic<ApiType>,
[SpTrieStorageProofCompactProof, Vec<ITuple<[H256, ShpFileKeyVerifierFileKeyProof]>>]
| Vec<PalletFileSystemFileKeyWithProof>
| (
| PalletFileSystemFileKeyWithProof
| {
fileKey?: any;
proof?: any;
}
| string
| Uint8Array
)[]
) => SubmittableExtrinsic<ApiType>,
[SpTrieStorageProofCompactProof, Vec<PalletFileSystemFileKeyWithProof>]
>;
/**
* Executed by a BSP to request to stop storing a file.
Expand Down
Loading
Loading