Skip to content

Commit

Permalink
Changed unique traces to exclude suppressed traces so they don't have…
Browse files Browse the repository at this point in the history
… to be filtered later.

Changed ClassicTheme to emit line break before trace colour block instead of inside.
Renamed CapabilitiesTest::testGigaSlow -> testVerySlow.
Updated Readme.
  • Loading branch information
Bilge committed Nov 6, 2024
1 parent 8d4a895 commit b64dcd0
Show file tree
Hide file tree
Showing 13 changed files with 51 additions and 52 deletions.
24 changes: 16 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Pip is a [PHPUnit][] extension that immediately prints exceptions and assertion
## Benefits

* Display the name of each test case as it is executed.
* Display the execution time of each test in tiered colour bands.
* Display the execution time of each test in configurable, tiered colour bands.
* Immediately print exceptions, assertion failures, warnings, notice and deprecation messages as they occur.
* Flawless test suite indicator: success dot turns to red exclamation mark if any prior tests failed. Useful for CI consoles without a scrollback buffer.

Expand Down Expand Up @@ -57,10 +57,10 @@ Pip's behaviour can be customized by adding `<parameter>` nodes as children of t
| Parameter name | Default value | Description |
|-----------------|---------------|------------------------------------------------------------------------|
| perf.slow | 200 (ms) | Sets the performance threshold for _slow_ (yellow) tests |
| perf.vslow | 1000 (ms) | Sets the performance threshold for _very slow_ (red) tests |
| perf.slow | 200 (ms) | Sets the performance threshold for _slow_ (yellow) tests. |
| perf.vslow | 1000 (ms) | Sets the performance threshold for _very slow_ (red) tests. |
| test.dp.args | true | True to show the arguments passed by the data provider, false to hide. |
| test.name.strip | '' | Strips the specified matching portion of the test name |
| test.name.strip | '' | Strips the specified matching portion of the test name. |
## Requirements
Expand All @@ -72,13 +72,21 @@ Pip's behaviour can be customized by adding `<parameter>` nodes as children of t
## Testing Pip
The printer's capabilities are exploited via `CapabilitiesTest`. However, this test file isn't run directly because many of these tests are designed to fail. Instead, we write tests that run PHPUnit internally, each of which invokes one of the capability test cases and verifies its output.
To run the full test suite, use the following command.
The real tests, also known as *functional tests*, are located in `test/functional`, written in PHPT format. PHPT is a [scarcely documented format](http://qa.php.net/phpt_details.php) designed to support [testing PHP itself](https://qa.php.net/write-test.php). An undocumented feature of PHPUnit is its limited support for a subset of the PHPT test specification, which we exploit to test PHPUnit itself with our printer implementation loaded.
```sh
composer test
```
Pip's capabilities are exploited via `CapabilitiesTest`. However, this test file isn't run directly because many of these tests are designed to fail. Instead, we write tests that run PHPUnit internally, each of which invokes one of the capability test cases and verifies its output. To run `CapabilitiesTest`, specify the following command
To run the tests, simply specify `vendor/bin/phpunit -c test` on the command line from the project directory. By default, we run all the functional PHPT tests. To run `CapabilitiesTest` instead, specify `vendor/bin/phpunit -c test test/CapabilitiesTest.php`.
```sh
composer test test/CapabilitiesTest.php
```
The real tests, also known as *functional tests*, are located in `test/functional`, written in PHPT format. PHPT is a [scarcely documented format](http://qa.php.net/phpt_details.php) designed to support [testing PHP itself](https://qa.php.net/write-test.php). An undocumented feature of PHPUnit is its limited support for a subset of the PHPT test specification, which we exploit to test PHPUnit itself with our printer implementation loaded.
### Writing a functional test
### Writing functional tests
To test the output of a particular capability we run `CapabilitiesTest` with the `--filter` option to target a specific test case. Each functional test contains the arguments passed to PHPUnit in the `--ARGS--` section of the file. These arguments can be pasted directly after the PHPUnit command to see the resulting output from that test case. We verify the output in the `--EXPECTF--` section of the file.
Expand Down
2 changes: 1 addition & 1 deletion src/PipExtension.php
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?php
declare(strict_types = 1);
declare(strict_types=1);

namespace ScriptFUSION\Pip;

Expand Down
10 changes: 5 additions & 5 deletions src/Printer.php
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,6 @@ public function __construct(private readonly PipConfig $config)
{
}

private function addUniqueTrace(Trace $trace): void
{
$this->uniqueTraces[$trace->getIssueId()] = $trace;
}

public function trace(Event $event): void
{
if ($event instanceof ExecutionStarted) {
Expand Down Expand Up @@ -155,4 +150,9 @@ public function trace(Event $event): void
echo PHP_EOL, PHP_EOL;
}
}

private function addUniqueTrace(Trace $trace): void
{
$trace->suppressed || $this->uniqueTraces[$trace->getId()] ??= $trace;
}
}
24 changes: 10 additions & 14 deletions src/Theme/ClassicTheme.php
Original file line number Diff line number Diff line change
Expand Up @@ -47,21 +47,17 @@ public function onTestFinished(TestResult $result): void
$throwable = $throwable->previous();
}

$firstIssue = true;
$result->uniqueTraces && print PHP_EOL;
foreach ($result->uniqueTraces as $trace) {
if (!$trace->suppressed) {
$issueStatusColour = self::getColour($trace->issueStatus);
printf(
Color::colorize("fg-$issueStatusColour", '%s%s: %s in %s on line %d%s'),
$firstIssue ? PHP_EOL : '',
$trace->issueStatus->name,
$trace->message,
$trace->file,
$trace->line,
PHP_EOL . PHP_EOL,
);
$firstIssue = false;
}
$issueStatusColour = self::getColour($trace->issueStatus);
printf(
Color::colorize("fg-$issueStatusColour", '%s: %s in %s on line %s%s%5$s'),
$trace->issueStatus->name,
$trace->message,
$trace->file,
$trace->line,
PHP_EOL,
);
}
}

Expand Down
11 changes: 3 additions & 8 deletions src/Trace.php
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,12 @@ public static function fromEvent(PhpWarningTriggered|PhpNoticeTriggered|PhpDepre
}

/**
* Key to identify identical issues, using the same rules as PHPUnit.
* Gets a unique identifier for the source of this trace, using the same rules as PHPUnit.
*
* @see \PHPUnit\TestRunner\TestResult\Collector::issueId
*/
public function getIssueId(): string
public function getId(): string
{
return sprintf(
'%s:%s:%s',
$this->file,
$this->line,
$this->message,
);
return "$this->file:$this->line:$this->message";
}
}
4 changes: 2 additions & 2 deletions test/CapabilitiesTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
use PHPUnit\Framework\TestCase;

/**
* Tests the capabilities of the PIP printer.
* Tests the capabilities of the Pip printer.
*/
final class CapabilitiesTest extends TestCase
{
Expand Down Expand Up @@ -155,7 +155,7 @@ public function testSlow(): void
self::assertTrue(true);
}

public function testGigaSlow(): void
public function testVerySlow(): void
{
sleep(1);

Expand Down
4 changes: 2 additions & 2 deletions test/functional/errors & exceptions/E_DEPRECATED.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Runtime: %s
Configuration: %s

100% D ScriptFUSIONTest\Pip\CapabilitiesTest::testDeprecation (%d ms)
[33;1m
Deprecated: Serializable@anonymous implements the Serializable interface, which is deprecated. Implement __serialize() and __unserialize() instead (or in addition, if support for old PHP versions is necessary) in %s%eCapabilitiesTest.php on line %d

[33;1mDeprecated: Serializable@anonymous implements the Serializable interface, which is deprecated. Implement __serialize() and __unserialize() instead (or in addition, if support for old PHP versions is necessary) in %s%eCapabilitiesTest.php on line %d



Expand Down
4 changes: 2 additions & 2 deletions test/functional/errors & exceptions/E_NOTICE.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Runtime: %s
Configuration: %s

100% N ScriptFUSIONTest\Pip\CapabilitiesTest::testNotice (%d ms)
[33;1m
Notice: Only variables should be assigned by reference in %s%eCapabilitiesTest.php on line %d

[33;1mNotice: Only variables should be assigned by reference in %s%eCapabilitiesTest.php on line %d



Expand Down
4 changes: 2 additions & 2 deletions test/functional/errors & exceptions/E_WARNING duplicates.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Runtime: %s
Configuration: %s

100% W ScriptFUSIONTest\Pip\CapabilitiesTest::testWarningDuplicates (%d ms)
[33;1m
Warning: foreach() argument must be of type array|object, int given in %s%eCapabilitiesTest.php on line %d

[33;1mWarning: foreach() argument must be of type array|object, int given in %s%eCapabilitiesTest.php on line %d

Warning: foreach() argument must be of type array|object, int given in %s%eCapabilitiesTest.php on line %d

Expand Down
4 changes: 2 additions & 2 deletions test/functional/errors & exceptions/E_WARNING.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Runtime: %s
Configuration: %s

100% W ScriptFUSIONTest\Pip\CapabilitiesTest::testWarning (%d ms)
[33;1m
Warning: foreach() argument must be of type array|object, int given in %s%eCapabilitiesTest.php on line %d

[33;1mWarning: foreach() argument must be of type array|object, int given in %s%eCapabilitiesTest.php on line %d



Expand Down
4 changes: 2 additions & 2 deletions test/functional/errors & exceptions/mixed severities.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ Runtime: %s
Configuration: %s

100% N ScriptFUSIONTest\Pip\CapabilitiesTest::testMixedSeverities (%d ms)
[33;1m
Notice: Only variables should be assigned by reference in %s%eCapabilitiesTest.php on line %d

[33;1mNotice: Only variables should be assigned by reference in %s%eCapabilitiesTest.php on line %d

Warning: foreach() argument must be of type array|object, int given in %s%eCapabilitiesTest.php on line %d

Expand Down
4 changes: 2 additions & 2 deletions test/functional/risky.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Runtime: %s
Configuration: %s

100% R ScriptFUSIONTest\Pip\CapabilitiesTest::testRisky (%d ms)
[33;1m
Risky: This test did not perform any assertions in %s%eCapabilitiesTest.php on line %d

[33;1mRisky: This test did not perform any assertions in %s%eCapabilitiesTest.php on line %d



Expand Down
4 changes: 2 additions & 2 deletions test/functional/speed vslow.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Tests that when a test takes more than 1s to complete, its timer output is shown in red.

--ARGS--
-c test --colors=always test/CapabilitiesTest.php --filter ::testGigaSlow$
-c test --colors=always test/CapabilitiesTest.php --filter ::testVerySlow$

--FILE_EXTERNAL--
../PHPUnit runner.php
Expand All @@ -13,7 +13,7 @@ PHPUnit %s
Runtime: %s
Configuration: %s

100% . [32;1mScriptFUSIONTest\Pip\CapabilitiesTest::testGigaSlow[0m [31m(%d ms)[0m
100% . [32;1mScriptFUSIONTest\Pip\CapabilitiesTest::testVerySlow[0m [31m(%d ms)[0m


Time: %s
Expand Down

0 comments on commit b64dcd0

Please sign in to comment.