feature: Add simple CLI #9

jjerphan · 2021-02-04T08:18:56Z

Hello there.

This is a simple proposition of a CLI, which makes this project even more usable.

$ py2puml -h  
usage: py2puml [-h] [-v] path module

Generate PlantUML diagrams to document your python code.

positional arguments:
  path           the path of the domain.
  module         the module of the domain.

optional arguments:
  -h, --help     show this help message and exit
  -v, --version  show program's version number and exit

For instance:

$ py2puml py2puml py2puml # in this project's source code directory
@startuml
# PlantUML statements of this project awesomeness
@enduml

jjerphan · 2021-02-04T08:41:32Z

With this, one can preview a UML diagram quickly:

$ py2puml py2puml py2puml | curl -X POST --data-binary @- http://your.plantuml.sever/svg/ --output - | display

lucsorel · 2021-02-05T16:40:55Z

hi @jjerphan,

Excellent idea, thanks! I have some minor remarks and requirements (unit-testing, mainly) that I would like to address. I haven't proposed some contribution guidelines yet in this project (I didn't expect them:) but I am glad it becomes necessary to provide some conventions. I hope to get back to you this weekend.

PS: we can keep on discussing in English (for other potential readers) or in French (for convenience), as you like 😃

jjerphan · 2021-02-05T16:47:24Z

Hi @lucsorel

Thanks for your message.

Excellent idea, thanks! I have some minor remarks and requirements (unit-testing, mainly) that I would like to address. I haven't proposed some contribution guidelines yet in this project (I didn't expect them:) but I am glad it becomes necessary to provide some conventions. I hope to get back to you this weekend.

I do agree. Let me know about the way you want it to be tested as there generally are several practices (for instance here are some good ones IMO).

PS: we can keep on discussing in English (for other potential readers) or in French (for convenience), as you like smiley

I would suggest keeping our 🥖-ness aside for public discussions, and thus use English to be more inclusive.

lucsorel · 2021-02-12T14:38:30Z

Forgive me for my lack of reactivity, weeks with very little spare time for this library. A few suggestions:

unit tests are already written using pytest. You could add some unit tests in tests/py2puml/test_cli.py (a valid case on one of the test/domain/*.py example, some others with missing arguments or invalid paths) which would launch some CLI commands with subprocess.run([...], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=True) (see https://docs.python.org/3/library/subprocess.html?highlight=subprocess%20run#subprocess.run, https://stackabuse.com/executing-shell-commands-with-python/#runningacommandwithsubprocess)
assertions about errors can be easily tested with pytest.raises (https://stackoverflow.com/a/44163840)
bump project version to 0.4.0 https://github.com/lucsorel/py2puml/blob/master/pyproject.toml#L3 and add the cli feature in the changelog (https://github.com/lucsorel/py2puml/blob/master/README.md#changelog)
please, add yourself to the contributors (https://github.com/lucsorel/py2puml/blob/master/README.md#contributions) if you want to 😃
add a documentation section about the cli in the main README.md. I have a question about the way to call the cli from a terminal: should it use the script defined in the pyproject.toml file or a more classic command like python -m py2puml.cli path module? I am not to familiar with that, I believe both approaches are possible, aren't they?
code styling (I need to add a CONTRIBUTING.md) to the project (I did not expect contributions but I am really glad this project helps people):
- favor single quote over double quote for strings (except for string including a single quote: "as in I'm happy")
- favor specific import (from pytest import raises > import pytest)

Le me know if you would like some help about the unit tests or other things. I hope I don't ask too much.

jjerphan · 2021-02-12T18:20:30Z

Hi @lucsorel

Forgive me for my lack of reactivity, weeks with very little spare time for this library.

It's fine. Please, don't feel in a hurry to respond. 🙂

I'll have a look at your suggestions.

Being done with this PR, I'd be interested to work on #3 and/or #7 (which are closely related) as I think that this would push the project forward: I think this project have the potential to help document a lot of python libraries!

jjerphan · 2021-02-12T19:27:56Z

As for your suggestion:

add a documentation section about the cli in the main README.md. I have a question about the way to call the cli from a terminal: should it use the script defined in the pyproject.toml file or a more classic command like python -m py2puml.cli path module? I am not to familiar with that, I believe both approaches are possible, aren't they?

I'll add a section for it. With what I've done there. If py2puml is installed, an eponymous command is available in the environment's shell directly (no need tu use python -m here).

python -m py2puml.cli does not work with this current setup, but it maybe be possible to adapt those changes to make the CLI usable via this call if needed. 🙂

jjerphan · 2021-02-12T20:13:38Z

I just have added a simple test for consistency and will add some more.

… a README typo

lucsorel · 2021-02-16T11:29:48Z

thank you for all the changes, I have just pushed some minor adjustments regarding the project version and CLI documentation: 37fb60a.

I am working on how to make the CLI work with the python -m py2puml.cli way too, so that the unit test can pass without the package being installed at the system level (although it is the way to install a CLI command). If you have an idea of how to make this work, feel free to let me know.

lucsorel · 2021-02-16T13:00:02Z

I modified the CLI script so that it can launched as python module as well. Can you tell me what you think about the 3 previous commits, please?

Moreover, I was wrong about the so that the unit test can pass without the package being installed at the system level: as poetry also installs the library in the virtual environment, the unit test you wrote actually passes in an development environment.

Feel free to add some unit tests about the -h and -v / --version if you want.

jjerphan · 2021-02-16T15:10:30Z

py2puml/cli.py

+# CLI end-point when py2puml is NOT installed at the system level
+if __name__ == '__main__':
+    if len(argv) == 2 and argv[1] in ['-v', '--version']:
+        print(f'{__description__} ({CLI_VERSION})')
+    elif len(argv) == 3:
+        print(''.join(py2puml(argv[1], argv[2])))
+    else:
+        raise ValueError(f'''
+The CLI can be called in 2 ways:
+- version: python -m py2puml.cli -v or python -m py2puml.cli --version
+- plantuml documentation (2 mandatory arguments): `python py2puml.cli domain/path domain.module`: {PATH_ARG_HELP}, {MODULE_ARG_HELP}
+''')


To me this is kind of convoluted and argparse usually takes care of CLI mis-usage, be it installed on the system or not. 🙂

It might be possible to make the CLI accessible with a similar and conciser syntax, for instance :

$ python -m py2puml

by modifying py2pulm.__init__ to include:

from py2puml.cli import run if __name__ == "__main__": run()

This is to test though.

yes, I didn't feel right implementing the CLI twice with sys.argv. Can you try modifying py2pulm.__init__ as you suggest and remove the convoluted code, please?

I'll do it.

py2puml/__init__.py

tests/py2puml/test_py2puml.py

jjerphan · 2021-02-16T15:19:30Z

I modified the CLI script so that it can launched as python module as well. Can you tell me what you think about the 3 previous commits, please?

Thanks for your useful additions! I left a few comments.

Moreover, I was wrong about the so that the unit test can pass without the package being installed at the system level: as poetry also installs the library in the virtual environment, the unit test you wrote actually passes in an development environment.

Yes, this is another (indirect) avantage of poetry, but we need to also have the test you just added. 🙂

Feel free to add some unit tests about the -h and -v / --version if you want.

Yes, I guess we could parametrised tests with both command/ entrypoints for conciseness and extendibility.

lucsorel · 2021-02-17T14:18:53Z

thank you for the parametrized test, it is an interesting feature of pytest that I never used before 😃

I did some minor refactoring (dedicated test files for __init__ and the cli, named imports instead of global imports, moved the doc-string of some test functions directly in their name - so that they are displayed in the pytest results).

If you are ok with the state of this PR, I am ready to squash and merge it & update the version in PyPI.

What do you think? Would you prefer adding some CLI unit tests about incomplete commands and the -h and -v options?

lucsorel · 2021-02-17T14:40:17Z

in the previous commit, I wanted to add in the README.md file the use case you suggested at the beginning of this PR (piping the puml model to a PlantUML server to generate/display an image) but I did not succeed in making it work 😞

Do you know how to fix the curl command?

jjerphan · 2021-02-17T14:41:01Z

If you are ok with the state of this PR, I am ready to squash and merge it & update the version in PyPI.

Yes, it looks like a good first version for the CLI.

What do you think? Would you prefer adding some CLI unit tests about incomplete commands and the -h and -v options?

I am not too concerned about the --help and the --verbose options: this is argparses role and this is mainly for human to have a readable output. I think that we can more tests on various entry-points later when improving the CLI.

lucsorel · 2021-02-17T14:46:40Z

I agree about not testing the help and verbose commands. What about the documentation about piping the CLI output to a PlantUML server?

jjerphan · 2021-02-17T14:51:36Z

What about the documentation about piping the CLI output to a PlantUML server?

You are faster than I am to reply 😁

in the previous, I wanted to add in the README.md file the use case you suggested at the beginning of this PR (piping the puml model to a PlantUML server to generate/display an image) but I did not succeed in making it work disappointed

Do you know how to fix the curl command?

In my case, I could previously use PlantUML's demo server with:

$ py2puml py2pulm py2pulm | curl -X POST --data-binary @- http://www.plantuml.com/plantuml/svg/ --output - | display

But with the latest changes for the setup with the toml file, I get errors which aren't related to requesting the server, e.g:

$ py2puml py2puml py2puml
Traceback (most recent call last):
  File "/home/jjerphan/.virtualenvs/py2puml/bin/py2puml", line 5, in <module>
    from py2puml.cli import run
  File "/home/jjerphan/.virtualenvs/py2puml/lib64/python3.9/site-packages/py2puml/__init__.py", line 21, in <module>
    with open(join(PARENT_DIR, 'pyproject.toml')) as pyproject:
FileNotFoundError: [Errno 2] No such file or directory: '/home/jjerphan/.virtualenvs/py2puml/lib/python3.9/site-packages/pyproject.toml'

We might want to adjust the setup to include project.toml in the distributions. This can be done using a MANIFEST.IN.

lucsorel · 2021-02-17T15:06:09Z

It may be a typo while you copy-pasted the command (py2pulm -> py2puml/domain and py2puml.domain). I did not reproduce the error you had. However, I got a different error:

poetry run py2puml py2puml/domain py2puml.domain | curl -X POST --data-binary @- http://www.plantuml.com/plantuml/svg/ --output - | display
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   896    0     0  100   896      0   2185 --:--:-- --:--:-- --:--:--  2185
display-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/560.

It looks like my linux distribution is missing come codec (searching the error message leads me to some issues with ImageMagick) 🤷

The following command yields an empty svg file:

poetry run py2puml py2puml/domain py2puml.domain | curl -X POST --data-binary @- http://www.plantuml.com/plantuml/svg/ --output stuff.svg

jjerphan · 2021-02-17T15:08:52Z

Try removing parts of the command just after the first pipe.

lucsorel · 2021-02-17T15:11:01Z

poetry run py2puml py2puml/domain py2puml.domain works fine: it outputs the expected plantuml code. Is that what you meant?

jjerphan · 2021-02-22T14:35:41Z

It looks like my linux distribution is missing come codec (searching the error message leads me to some issues with ImageMagick)

I had the same error message but this was due to this latter error rather than the rendering with PlantUML's server.

lucsorel · 2021-02-23T09:05:55Z

ok, thank you @jjerphan. So just to make sure: what do we need to do to finish this pull request, so that we can embark your contribution?

add a manifest.json so that the pyproject.toml file gets installed?
remove the code that retrieves the version number and the description from the toml file? I wanted the toml file to be the single source of truth for that (because it is used by pypi) but it may not be a good idea

What do you think?

jjerphan · 2021-02-23T13:12:15Z

Hum, it does seems that easy to embed the pyproject.toml file and manage to get the version from there in a source distribution using a MANIFEST.in. 🤔

Other projects include the version number in the source code directly and get the metadata published on PyPi using setup.py and use pyproject.toml solely for building purposes (e.g. scipy, scikit-learn).

I do understand that use a single source of truth (like pyproject.toml) would be better (I would also do so), but I do not know how easy it is to have it work fore most distributions of the project.

lucsorel · 2021-02-23T15:23:52Z

@jjerphan I have just pushed a commit which:

decouples the CLI from the pyproject.toml file
unit-tests that the version and description output by the CLI are consistent with the pyproject.toml file, as a workaround to maintain the consistency

What do you think?

Does this commit fix the poetry run py2puml py2puml/domain py2puml.domain | curl -X POST --data-binary @- https://www.plantuml.com/plantuml/png/ --output - | display command on your system? Unfortunately for me, it does not, I still have this issue:

poetry run python -m py2puml py2puml/domain py2puml.domain | curl -X POST --data-binary @- https://www.plantuml.com/plantuml/png/ --output - | display
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   896    0     0  100   896      0   1315 --:--:-- --:--:-- --:--:--  1315
display-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/560.

jjerphan · 2021-02-26T07:29:20Z

It runs (and has been) running with poetry and a PlantUML with a local host.

I.e, The following commands work:

poetry run python -m py2puml py2puml/domain py2puml.domain

# Priorly ran to setup a local PlantUML server:
# docker run -d -p 8080:8080 plantuml/plantuml-server:jetty 
poetry run python -m py2puml py2puml/domain py2puml.domain | curl -X POST --data-binary @- http://localhost:8080/svg/ --output - | display

the following ones do not:

poetry run python -m py2puml py2puml/domain py2puml.domain | curl -X POST --data-binary @- https://www.plantuml.com/plantuml/png/ --output - | display

It looks like the global PlantUML instance does not support direct API calls (I naively though it would, but it doesn't, which is understandable).

Personally, I would like the CLI to be working without using poetry. 🙂

py2puml/__init__.py

jjerphan · 2021-02-28T20:24:05Z

I think it would be interesting to add the docker command which runs a local plantuml server in the README.md's CLI documentation. What do you think?

Yes, let's add a few notes for this.

Regarding the issue with the pyproject.toml parsing, I think I will move it into a test script (which runs with poetry in a development or CI/CD environment) so that the production code (and the CLI in particular) does not do it anymore. That would solve this last issue, wouldn't it?

This won't solve the problem, I think. The problem is linked to the way pip (which I use) install sources distribution from the code.

Though I am not a maintainer, a CI might be interesting to test various configuration for the project generally and could be introduced in another PR. 🙂

lucsorel · 2021-03-03T14:09:49Z

hi @jjerphan

The last 2 commits:

the first one fixes the pip installation (which keeps the inner py2puml folder, thus removing the parent pyproject.toml file): the extraction of the version and description is performed in the tests/init.py file and used only by the unit tests. Thus it does not break the cli or the python API when the library is installed with pip
the 2nd one is the addition of some documentation about launching a plantuml server locally with docker

If you are ok with the state of this PR, I am ready to merge it and publish the new version in PyPI 😃

lucsorel · 2021-03-03T14:12:27Z

Though I am not a maintainer, a CI might be interesting to test various configuration for the project generally and could be introduced in another PR.

Forgive me, I did not reply to this good suggestion, but I approve it. Launching unit tests with github actions would be a first step

jjerphan · 2021-03-03T14:23:10Z

Hi @lucsorel

I got busy those last days, thanks a lot for inspecting and fixing this for me: it works on my end!
The documentation looks nice, and I guess we can finally merge this PR. 😁

Forgive me, I did not reply to this good suggestion, but I approve it. Launching unit tests with github actions would be a first step

It's completely fine. I might not have time in the coming days to work on it, so please feel free to do if you want to.

lucsorel · 2021-03-03T14:48:32Z

your contribution is alive in https://pypi.org/project/py2puml/.

Thank you so much for your contribution, dedication and patience 🙏

Add simple CLI

5bde636

lucsorel added the enhancement New feature or request label Feb 4, 2021

jjerphan added 5 commits February 12, 2021 20:46

Add simple consistency test for the CLI

74a99a8

Respect PEP8 using single quote

d5ca844

Add a documentation section for the CLI

d3563da

Update Changelog

017a60a

Update Contributors

534266b

bumped the project version to 0.4.0 in the pyproject.toml file, fixed…

37fb60a

… a README typo

lucsorel added 3 commits February 16, 2021 13:39

mutualized project description and supported CLI as a python module

fb9bd6c

added documentation about launching the CLI as a python module

2968071

added unit test for the CLI-as-a-module consistency

ece1042

jjerphan commented Feb 16, 2021

View reviewed changes

py2puml/__init__.py Outdated Show resolved Hide resolved

jjerphan commented Feb 16, 2021

View reviewed changes

tests/py2puml/test_py2puml.py Outdated Show resolved Hide resolved

jjerphan and others added 4 commits February 16, 2021 17:05

Make py2pulm executable as a package

2cb5603

PEP8 consistency

dfd8d11

Factorise tests using parametrisation

1a37ef8

dedicated unit tests for CLI and __init__, code style & format

63a981a

WIP document how to pipe the cli output to a puml server

aff5470

removed unused imports

fc786a3

lucsorel added 2 commits February 23, 2021 16:17

workaround to get the cli -h and -v working when installed system-wise

35de1e5

forgot to uncomment a line

eabbc41

jjerphan commented Feb 26, 2021

View reviewed changes

py2puml/__init__.py Outdated Show resolved Hide resolved

lucsorel added 2 commits March 3, 2021 14:57

moved the extraction of version and description to the tests folder

a0a39ca

documented launching a plantuml server with a docker image

e868705

tiny typo

66ef1ff

lucsorel merged commit 3174add into lucsorel:master Mar 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature: Add simple CLI #9

feature: Add simple CLI #9

jjerphan commented Feb 4, 2021

jjerphan commented Feb 4, 2021

lucsorel commented Feb 5, 2021

jjerphan commented Feb 5, 2021

lucsorel commented Feb 12, 2021

jjerphan commented Feb 12, 2021

jjerphan commented Feb 12, 2021 •

edited

Loading

jjerphan commented Feb 12, 2021

lucsorel commented Feb 16, 2021

lucsorel commented Feb 16, 2021 •

edited

Loading

jjerphan Feb 16, 2021

lucsorel Feb 16, 2021

jjerphan Feb 16, 2021

jjerphan commented Feb 16, 2021

lucsorel commented Feb 17, 2021

lucsorel commented Feb 17, 2021 •

edited

Loading

jjerphan commented Feb 17, 2021

lucsorel commented Feb 17, 2021

jjerphan commented Feb 17, 2021

lucsorel commented Feb 17, 2021 •

edited

Loading

jjerphan commented Feb 17, 2021

lucsorel commented Feb 17, 2021

jjerphan commented Feb 22, 2021

lucsorel commented Feb 23, 2021

jjerphan commented Feb 23, 2021 •

edited

Loading

lucsorel commented Feb 23, 2021

jjerphan commented Feb 26, 2021

jjerphan commented Feb 28, 2021

lucsorel commented Mar 3, 2021

lucsorel commented Mar 3, 2021

jjerphan commented Mar 3, 2021 •

edited

Loading

lucsorel commented Mar 3, 2021

feature: Add simple CLI #9

feature: Add simple CLI #9

Conversation

jjerphan commented Feb 4, 2021

jjerphan commented Feb 4, 2021

lucsorel commented Feb 5, 2021

jjerphan commented Feb 5, 2021

lucsorel commented Feb 12, 2021

jjerphan commented Feb 12, 2021

jjerphan commented Feb 12, 2021 • edited Loading

jjerphan commented Feb 12, 2021

lucsorel commented Feb 16, 2021

lucsorel commented Feb 16, 2021 • edited Loading

jjerphan Feb 16, 2021

Choose a reason for hiding this comment

lucsorel Feb 16, 2021

Choose a reason for hiding this comment

jjerphan Feb 16, 2021

Choose a reason for hiding this comment

jjerphan commented Feb 16, 2021

lucsorel commented Feb 17, 2021

lucsorel commented Feb 17, 2021 • edited Loading

jjerphan commented Feb 17, 2021

lucsorel commented Feb 17, 2021

jjerphan commented Feb 17, 2021

lucsorel commented Feb 17, 2021 • edited Loading

jjerphan commented Feb 17, 2021

lucsorel commented Feb 17, 2021

jjerphan commented Feb 22, 2021

lucsorel commented Feb 23, 2021

jjerphan commented Feb 23, 2021 • edited Loading

lucsorel commented Feb 23, 2021

jjerphan commented Feb 26, 2021

jjerphan commented Feb 28, 2021

lucsorel commented Mar 3, 2021

lucsorel commented Mar 3, 2021

jjerphan commented Mar 3, 2021 • edited Loading

lucsorel commented Mar 3, 2021

jjerphan commented Feb 12, 2021 •

edited

Loading

lucsorel commented Feb 16, 2021 •

edited

Loading

lucsorel commented Feb 17, 2021 •

edited

Loading

lucsorel commented Feb 17, 2021 •

edited

Loading

jjerphan commented Feb 23, 2021 •

edited

Loading

jjerphan commented Mar 3, 2021 •

edited

Loading