Update test callable discovery to work within the language service #2095

sezna · 2025-01-07T19:38:17Z

This PR rewrites #2059 to trigger test discovery from within the language server, as opposed to externally in an entirely separate context. Please read the description in that PR for more detail.

It also adds tests for LS state in both the Rust-based LS tests and the JS-based basics.js npm API tests.

…lex/track_test_changes_in_ls

minestarks

Not done looking at all the files, posting all my comments for today.

compiler/qsc/src/lib.rs

wasm/src/test_discovery.rs

wasm/src/language_service.rs

npm/qsharp/src/language-service/language-service.ts

npm/qsharp/src/compiler/compiler.ts

wasm/src/language_service.rs

minestarks

Finished looking. I really like the overall approach. Also I assume integration tests are coming up in a different PR.

compiler/qsc_passes/src/test_attribute.rs

compiler/qsc_passes/src/test_attribute/tests.rs

language_service/src/state.rs

language_service/src/state/tests.rs

vscode/src/language-service/activate.ts

vscode/src/language-service/testExplorer.ts

library/signed/src/Tests.qs

vscode/src/language-service/testExplorer.ts

library/signed/src/Tests.qs

language_service/src/protocol.rs

minestarks · 2025-01-09T00:46:57Z

@sezna hitting this odd bug where you can't navigate to the test case after it fails.

language_service/src/state.rs

swernli · 2025-01-13T18:42:45Z

I ran into an issue where if I open a notebook that includes Q# cells the test explorer loses its content and can't get it back until a full window reload:

Screen.Recording.2025-01-13.at.10.40.09.AM.mov

sezna · 2025-01-22T23:02:31Z

Just FYI @minestarks @swernli that test-disappearing bug is fixed and this is all green again.

billti · 2025-01-23T23:19:43Z

npm/qsharp/src/language-service/language-service.ts

@@ -1,6 +1,7 @@
 // Copyright (c) Microsoft Corporation.
 // Licensed under the MIT License.

+import { ITestDescriptor } from "../../lib/node/qsc_wasm.cjs";


This should be imported as part of the following import list.

billti · 2025-01-23T23:22:00Z

npm/qsharp/src/language-service/language-service.ts

+
+  async onTestCallables(callables: ITestDescriptor[]) {
+    try {
+      const event = new Event("testCallables") as LanguageServiceEvent & Event;


Should this and onDiagnostics (line 278 above) now be the specific event type? (e.g. LanguageServiceDiagnosticEvent above and the TestCallables version here)

billti · 2025-01-23T23:23:46Z

playground/src/editor.tsx

  QscEventTarget,
  VSDiagnostic,
  log,
  ProgramConfig,
  TargetProfile,
+  LanguageServiceDiagnosticEvent,


With this rename it'll likely be a breaking change for QCOM. We'll need to ensure we communicate this when they take a new drop.

billti · 2025-01-23T23:33:01Z

wasm/src/language_service.rs

@@ -58,7 +60,35 @@ impl LanguageService {
                )
                .expect("callback should succeed");
        };
-        let mut worker = self.0.create_update_worker(diagnostics_callback, host);
+
+        let test_callables_callback = test_callables_callback


Is the initial call done on line 40 for the diagnostics callback unnecessary then? (Mine thinks it might be)

billti · 2025-01-23T23:42:31Z

When a test case fails and I click on it to go to the test source, I get the below error.

Interestingly, when the test passes, it seems to work fine!

billti · 2025-01-23T23:44:47Z

The 'Run Tests' button is not running any tests. When I click Run on the items in the tree view it works, but this button doesn't.

sezna and others added 30 commits December 10, 2024 06:04

initial work on testing

b79c31e

Merge branch 'main' of github.com:microsoft/qsharp into alex/testHarness

1c09783

progress on test explorer

54eb209

test collection works

e6a0941

tests run

0495a6d

add todos

eae1b4d

switching to namespaces included with callable names

759036f

scoped test names

cec4bf9

deduplicate parent items

ad7d3e2

make running child items work

d9f288d

auto-refresh test cases

9ec9f41

wip -- checkpoint

a03ed14

wip

c11476a

Remove codelens stuff

b247c35

update libraries to use new testing harness

e75f65b

remove bad imports

131a341

document some functions

b52ad25

Add comment

180368e

Remove todos

e960916

Fix nested tests

7d2aadc

initial round of PR feedback

ac8bfd7

move discovery of test items into compiler layer

c22d14d

Use a pass to detect test attribute errors and report them nicely

5a28ef1

use getActiveProgram

16a2aed

remove unnecessary api in main.ts

214097c

Fmt

adbc4b2

fix lints

5b6a1f8

update tests

7cee532

filter out invalid test items

be64105

Merge branch 'main' of github.com:microsoft/qsharp into alex/testHarness

e34b7d2

sezna requested a review from ScottCarda-MS as a code owner January 7, 2025 20:37

sezna added 2 commits January 7, 2025 12:48

Merge branch 'main' of github.com:microsoft/qsharp into alex/testHarness

8255955

Merge branch 'alex/testHarness' of github.com:microsoft/qsharp into a…

007a76d

…lex/track_test_changes_in_ls

sezna changed the base branch from alex/testHarness to main January 7, 2025 23:41

sezna requested a review from DmitryVasilevsky as a code owner January 7, 2025 23:41

minestarks reviewed Jan 8, 2025

View reviewed changes

idavis mentioned this pull request Jan 8, 2025

Test Explorer in VS Code Extension and @Test() Attribute #2059

Closed

minestarks requested changes Jan 9, 2025

View reviewed changes

Fix output

1d5ef8e

github-advanced-security bot found potential problems Jan 9, 2025

View reviewed changes

language_service/src/state.rs Fixed Show fixed Hide fixed

mid PR review feedback

6ca81d0

sezna added 8 commits January 21, 2025 11:22

pull from main

f6ad5a3

Address large portion of pr feedback

4235d03

apply rest of PR feedback

cf99d0c

Clean up lints

b66fab4

final test updates; build.py passing

14043ad

update ts types

4c48051

fix test explorer

2471cc6

Remove empty else block

1bac925

Fix basics.js types

3339738

sezna requested a review from minestarks January 23, 2025 16:47

billti reviewed Jan 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update test callable discovery to work within the language service #2095

Update test callable discovery to work within the language service #2095

sezna commented Jan 7, 2025 •

edited

Loading

minestarks left a comment

minestarks left a comment

minestarks commented Jan 9, 2025

swernli commented Jan 13, 2025

sezna commented Jan 22, 2025

billti Jan 23, 2025

billti Jan 23, 2025

billti Jan 23, 2025

billti Jan 23, 2025

billti commented Jan 23, 2025

billti commented Jan 23, 2025

Update test callable discovery to work within the language service #2095

Are you sure you want to change the base?

Update test callable discovery to work within the language service #2095

Conversation

sezna commented Jan 7, 2025 • edited Loading

minestarks left a comment

Choose a reason for hiding this comment

minestarks left a comment

Choose a reason for hiding this comment

minestarks commented Jan 9, 2025

swernli commented Jan 13, 2025

sezna commented Jan 22, 2025

billti Jan 23, 2025

Choose a reason for hiding this comment

billti Jan 23, 2025

Choose a reason for hiding this comment

billti Jan 23, 2025

Choose a reason for hiding this comment

billti Jan 23, 2025

Choose a reason for hiding this comment

billti commented Jan 23, 2025

billti commented Jan 23, 2025

sezna commented Jan 7, 2025 •

edited

Loading