Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DAOS-16686 dfuse: Fix overlapping chunk reads #15680

Open
wants to merge 45 commits into
base: master
Choose a base branch
from
Open

DAOS-16686 dfuse: Fix overlapping chunk reads #15680

wants to merge 45 commits into from

Conversation

ashleypittman
Copy link
Contributor

Before requesting gatekeeper:

  • Two review approvals and any prior change requests have been resolved.
  • Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
  • Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
  • Commit messages follows the guidelines outlined here.
  • Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

  • You are the appropriate gatekeeper to be landing the patch.
  • The PR has 2 reviews by people familiar with the code, including appropriate owners.
  • Githooks were used. If not, request that user install them and check copyright dates.
  • Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
  • All builds have passed. Check non-required builds for any new compiler warnings.
  • Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
  • If applicable, the PR has addressed any potential version compatibility issues.
  • Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
  • Extra checks if forced landing is requested
    • Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
    • No new NLT or valgrind warnings. Check the classic view.
    • Quick-build or Quick-functional is not used.
  • Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

@ashleypittman
DAOS-15682 dfuse: Fail on concurrent read.
9aa13e0 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Try and fix issue.
1d4019f 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
First stab at a fix.
122faf0 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
270360f
@ashleypittman
Fix invalid free and leak.
f74f053 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Fix a logging line.
6ad750b 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Add some debugging.
578be24 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Track duplicate reads. This avoids a crash but there's still a memory…
e771912 
… leak

Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Fix logic.
6c407e7 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Rework to support blocking on network requests.
7f892f3 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Bump array size and add stats.
6e286d1 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Fix a segv in the stats.
b2a21c3 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Track EOF better in reads.
b685591 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
33409b3
@ashleypittman
Fixup after merge
4fcedda 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Move active read list to active.
018449e 
Test-tag: dfuse

Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
52e827c 
Test-tag: dfuse
@ashleypittman
Rebase and iterate on comments.
221c849 
Test-tag: dfuse
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
fix: remove an extra list operation.
f4956ac 
Test-tag: dfuse
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
33a2ee3
@ashleypittman
Back out test and stat changes.
0733c0b 
Test-tag: dfuse

Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
46a565b
@ashleypittman
Try and solve patchelf problem.
3a2bbd1 
Test-tag: DaosBuild
Skip-fault-injection-test: true

Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Change failure mode.
2308feb 
Skip-func-test-vm: true

Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
e3e9836 
Test-tag: dfuse

Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
7ebd792 
Test-tag: dfuse
@ashleypittman
add strace.
6fd17df 
Skip-func-test-vm: true

Test-tag: DaosBuild
Skip-fault-injection-test: true

Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Do not pre-fetch or keep cache for writeable files.
f8ac3f4 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Stat and re-fresh the files.
9453406 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Fix build not to fail.
ffe786c 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
ashleypittman and others added 15 commits November 21, 2024 15:23
@ashleypittman
Stat the file, not the link
1a8630c 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Disable matching reads and chunk code to debug failure.
a79ad9d 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Disalbe the file-size EOF optimisation.
6d64ede 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Re-enable feature.
0732488 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Back out build debug.
c4b0d02 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
f76c802
@ashleypittman
Back out matching read changes, they're in a different PR.
974c2ee 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true

Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Make number of slots vcariable.
db9213f 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Fix a compile warning.
342978e 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Handle truncated reads and avoid crash.
82854c0 
Skip-unit-tests: true
Skip-func-test-vm: true
Test-tag: DaosBuild
Skip-fault-injection-test: true
Signed-off-by: Ashley Pittman <[email protected]>
@jolivier23
Merge branch 'master' into amd/dfuse-concurrent-read
694bae0
@wangdi1
DAOS-16686 dfuse: avoid possible corruption
3f0e0f3 
Comments from Ashely

In chunk_cb(), there's no reference on cd held here after the
last call to DFUSE_REPLY../. so the list needs to be spliced
onto the stack before the list is iterated.

Required-githooks: true

Signed-off-by: Di Wang <[email protected]>
@wangdi1
Merge branch 'master' into amd/dfuse-concurrent-read
e8b1722
@ashleypittman
Fix list handling.
c7a8703 
Signed-off-by: Ashley Pittman <[email protected]>
@ashleypittman
Merge branch 'master' into amd/dfuse-concurrent-read
168053c 
Features: dfuse
@ashleypittman ashleypittman requested review from a team as code owners January 3, 2025 10:23
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 3, 2025

Ticket title is 'Concurrent reads hit the network even when caching enabled in dfuse'
Status is 'In Progress'
Labels: 'google-cloud-daos'
https://daosio.atlassian.net/browse/DAOS-16686

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jolivier23 jolivier23 jolivier23 approved these changes

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ashleypittman @jolivier23 @wangdi1