From 84e28c95ff4647b17f3d851b6d255251ebe42942 Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Sun, 13 Sep 2020 12:58:05 -0700 Subject: [PATCH 1/3] edit doc source files for grammar and understanding --- docs/autogen_config.py | 2 +- docs/source/execute_api.rst | 108 ++++++++++++++++++++++---------- docs/source/index.rst | 3 + docs/source/latex_citations.rst | 8 +-- docs/source/removing_cells.rst | 8 ++- docs/source/usage.rst | 19 +++--- 6 files changed, 98 insertions(+), 50 deletions(-) diff --git a/docs/autogen_config.py b/docs/autogen_config.py index 3db045b4c..aae9f3e3f 100644 --- a/docs/autogen_config.py +++ b/docs/autogen_config.py @@ -2,7 +2,7 @@ """ autogen_config.py -Create config_options.rst, a Sphinx documentation source file. +Creates config_options.rst, a Sphinx documentation source file. Documents the options that may be set in nbconvert's configuration file, jupyter_nbconvert_config.py. diff --git a/docs/source/execute_api.rst b/docs/source/execute_api.rst index d4fd246d7..d1349b0d7 100644 --- a/docs/source/execute_api.rst +++ b/docs/source/execute_api.rst @@ -9,72 +9,109 @@ nbconvert provides a convenient way to execute the input cells of an as a .ipynb file. In this section we show how to execute a ``.ipynb`` notebook -document saving the result in notebook format. If you need to export +document and save the result in notebook format. If you need to export notebooks to other formats, such as reStructured Text or Markdown (optionally executing them) see section :doc:`nbconvert_library`. -Executing notebooks can be very helpful, for example, to run all notebooks -in Python library in one step, or as a way to automate the data analysis in -projects involving more than one notebook. +Executing notebooks can be helpful functionality for some use cases. +For example, nbconvert can run all notebooks in a folder in one step. +Automating the data analysis in projects involving more than one notebook +is also possible. + +Notebook execution can be done either from the command line or programmatically +using the Python API interface. Executing notebooks from the command line ----------------------------------------- -The same functionality of executing notebooks is exposed through a -:doc:`command line interface ` or a Python API interface. -As an example, a notebook can be executed from the command line with:: +The same functionality of executing notebooks in Jupyter Notebook, JupyterLab, +or other front-end applications is available through a +:doc:`command line interface `. - jupyter nbconvert --to notebook --execute mynotebook.ipynb +To execute a notebook from the command line, enter the command below and +specify the command line options for the file to execute (``--execute``) +and the output (``--to``):: -Executing notebooks using the Python API interface --------------------------------------------------- -This section will illustrate the Python API interface. + jupyter nbconvert --to notebook --execute mynotebook.ipynb -Example -~~~~~~~ +To find available command line options, enter: -Let's start with a complete quick example, leaving detailed explanations -to the following sections. + jupyter nbconvert --help -**Import**: First we import nbconvert and the `ExecutePreprocessor` +Executing notebooks using the Python API interface +-------------------------------------------------- +Using nbconvert's Python API interface enables programmatic execution +of notebooks. This satisfies the use case where notebooks could +be executed from applications which import the nbconvert package. + +A complete example +~~~~~~~~~~~~~~~~~~ + +To better understand how to use nbconvert's Python API, let's start with a complete example. +This example illustrates the basics and leaves more in-depth explanations +to later sections. The basic steps include: + +- import +- load +- configure +- execute/run (preprocess) +- save + +Import +++++++ +First we import nbconvert and the `ExecutePreprocessor` class:: import nbformat from nbconvert.preprocessors import ExecutePreprocessor -**Load**: Assuming that ``notebook_filename`` contains the path of a notebook, -we can load it with:: +Load +++++ +Assuming that ``notebook_filename`` contains the path of a notebook, +use the following to open it:: with open(notebook_filename) as f: nb = nbformat.read(f, as_version=4) -**Configure**: Next, we configure the notebook execution mode:: +Configure +++++++++ + +Next, we configure the notebook execution mode:: ep = ExecutePreprocessor(timeout=600, kernel_name='python3') -We specified two (optional) arguments ``timeout`` and ``kernel_name``, which -define respectively the cell execution timeout and the execution kernel. +We specified two (optional) arguments ``timeout``, the cell execution +timeout, and ``kernel_name``, the execution kernel's name. The option to specify **kernel_name** is new in nbconvert 4.2. When not specified or when using nbconvert <4.2, the default Python kernel is chosen. -**Execute/Run (preprocess)**: To actually run the notebook we call the method +Execute/Run (preprocess) +++++++++++++++++++++++++ +To actually run the notebook we call the method :meth:`~ExecutePreprocessor.preprocess`:: ep.preprocess(nb, {'metadata': {'path': 'notebooks/'}}) -Hopefully, we will not get any errors during the notebook execution +Hopefully, notebook execution will not get any errors (see the last section for error handling). Note that ``path`` specifies in which folder to execute the notebook. -**Save**: Finally, save the resulting notebook with:: +Save +++++ +Finally, save the resulting notebook with:: with open('executed_notebook.ipynb', 'w', encoding='utf-8') as f: nbformat.write(nb, f) -That's all. Your executed notebook will be saved in the current folder +Your executed notebook will be saved in the current folder in the file ``executed_notebook.ipynb``. +Summary ++++++++ +The above example covers the fundamental steps of using nbconvert's Python API: +import, load, configure, execute/run, save + Execution arguments (traitlets) ------------------------------- @@ -112,8 +149,12 @@ maintain consistency: we can just run a notebook twice, specifying first Handling errors and exceptions ------------------------------ -In the previous sections we saw how to save an executed notebook, assuming -there are no execution errors. But, what if there are errors? +The previous sections covered how to save an executed notebook, assuming +no execution errors occur. But, what if there are errors? + +The options for handling errors are flexible and include execution until +the first error and save, handling errors during execution, and executing +and then saving all errors. Execution until first error ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -130,7 +171,8 @@ and includes a full stack-trace and error (which can help debugging). Handling errors ~~~~~~~~~~~~~~~ -A useful pattern to execute notebooks while handling errors is the following:: +A useful pattern to execute notebooks while handling errors uses a +try/except/finally block such as in the following:: from nbconvert.preprocessors import CellExecutionError @@ -146,7 +188,7 @@ A useful pattern to execute notebooks while handling errors is the following:: with open(notebook_filename_out, mode='w', encoding='utf-8') as f: nbformat.write(nb, f) -This will save the executed notebook regardless of execution errors. +This approach will save the executed notebook regardless of execution errors. In case of errors, however, an additional message is printed and the `CellExecutionError` is raised. The message directs the user to the saved notebook for further inspection. @@ -155,8 +197,8 @@ Execute and save all errors ~~~~~~~~~~~~~~~~~~~~~~~~~~~ As a last scenario, it is sometimes useful to execute notebooks which raise exceptions, for example to show an error condition. In this case, instead of -stopping the execution on the first error, we can keep executing the notebook -using the traitlet ``allow_errors`` (default is False). With +stopping the execution on the first error, we can keep executing the notebook. +To do this we set the traitlet ``allow_errors`` (default is False) to True. With ``allow_errors=True``, the notebook is executed until the end, regardless of any error encountered during the execution. The output notebook, will contain the stack-traces and error messages for **all** the cells raising exceptions. @@ -170,8 +212,8 @@ the state of all the widgets can be stored in the notebook's metadata. This allows rendering of the live widgets on for instance nbviewer, or when converting to html. -We can tell nbconvert to not store the state using the ``store_widget_state`` -argument:: +Setting the ``store_widget_state`` argument determines whether or not to +save the widget's state. We can tell nbconvert to not store the state using:: jupyter nbconvert --ExecutePreprocessor.store_widget_state=False --to notebook --execute mynotebook.ipynb diff --git a/docs/source/index.rst b/docs/source/index.rst index 2ae628adb..31773035f 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -41,6 +41,9 @@ batch of notebook files to another format. removing_cells execute_api +.. + Comment: config_options is created by running autogen_config.py + .. toctree:: :maxdepth: 2 :caption: Configuration diff --git a/docs/source/latex_citations.rst b/docs/source/latex_citations.rst index b04f7c91b..7be6e9621 100644 --- a/docs/source/latex_citations.rst +++ b/docs/source/latex_citations.rst @@ -1,12 +1,12 @@ LaTeX citations =============== -``nbconvert`` now has support for LaTeX citations. With this capability you +``nbconvert`` has support for LaTeX citations. With this capability you can: -* Manage citations using BibTeX. -* Cite those citations in Markdown cells using HTML data attributes. -* Have ``nbconvert`` generate proper LaTeX citations and run BibTeX. + * Manage citations using BibTeX. + * Cite those citations in Markdown cells using HTML data attributes. + * Have ``nbconvert`` generate proper LaTeX citations and run BibTeX. For an example of how this works, please see the `citations example`_ in the nbconvert-examples_ repository. diff --git a/docs/source/removing_cells.rst b/docs/source/removing_cells.rst index 69be245ee..e480ab713 100644 --- a/docs/source/removing_cells.rst +++ b/docs/source/removing_cells.rst @@ -6,8 +6,10 @@ Removing cells, inputs, or outputs When converting Notebooks into other formats, it is possible to remove parts of a cell, or entire cells, using preprocessors. The notebook will remain unchanged, but the outputs will have -certain pieces removed. Here are two primary ways to accomplish -this. +certain pieces removed. The two primary ways for removal are: + + - Using cell tags + - Using regular expressions Removing pieces of cells using cell tags ---------------------------------------- @@ -50,7 +52,7 @@ is a list of strings. For each cell, this preprocessor checks whether the cell contents match any of the strings provided in ``patterns``. If the contents match any of the patterns, the cell is removed from the notebook. -For example, execute the following command to convert a notebook to html +For example, executing the following command will convert a notebook to html and remove cells containing only whitespace: .. code-block:: bash diff --git a/docs/source/usage.rst b/docs/source/usage.rst index 301f8fbee..8b341cfa8 100644 --- a/docs/source/usage.rst +++ b/docs/source/usage.rst @@ -1,18 +1,18 @@ .. highlight:: none -Using as a command line tool -============================ +Using the command line tool +=========================== -The command-line syntax to run the ``nbconvert`` script is:: +``nbconvert`` can be run from the command line using:: $ jupyter nbconvert --to FORMAT notebook.ipynb -This will convert the Jupyter notebook file ``notebook.ipynb`` into the output +For example, nbconvert will convert the Jupyter notebook file ``notebook.ipynb`` into the output format given by the ``FORMAT`` string. Default output format - HTML ---------------------------- -The default output format is HTML, for which the ``--to`` argument may be +nbconvert's default output format is HTML, and the ``--to`` argument may be omitted:: $ jupyter nbconvert notebook.ipynb @@ -206,7 +206,7 @@ Markdown .. _convert_ascii: Ascii -~~~~~~~~ +~~~~~ * ``--to asciidoc`` Ascii output. @@ -301,18 +301,19 @@ the output may be sent to standard output with:: Converting multiple notebooks ----------------------------- -Multiple notebooks can be specified from the command line:: +Multiple notebooks can be specified from the command line for execution:: $ jupyter nbconvert notebook*.ipynb $ jupyter nbconvert notebook1.ipynb notebook2.ipynb -or via a list in a configuration file, say ``mycfg.py``, containing the text: +A list in a configuration file may also be used. For example, in a config file ``mycfg.py``, +a list of notebooks can be specified: .. code-block:: python c = get_config() c.NbConvertApp.notebooks = ["notebook1.ipynb", "notebook2.ipynb"] -and using the command:: +and then executed using the command:: $ jupyter nbconvert --config mycfg.py From e6e44dd180414ace9a90a238b6c65a1f362226a6 Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Sun, 13 Sep 2020 14:22:27 -0700 Subject: [PATCH 2/3] fix underline --- docs/source/execute_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/execute_api.rst b/docs/source/execute_api.rst index d1349b0d7..cab3b064d 100644 --- a/docs/source/execute_api.rst +++ b/docs/source/execute_api.rst @@ -73,7 +73,7 @@ use the following to open it:: nb = nbformat.read(f, as_version=4) Configure -++++++++ ++++++++++ Next, we configure the notebook execution mode:: From b1c0acef11feb3d60b7f1ce2d51679165dc5152b Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Sun, 13 Sep 2020 14:38:09 -0700 Subject: [PATCH 3/3] add preprocess info for subclasses --- docs/source/nbconvert_library.ipynb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/nbconvert_library.ipynb b/docs/source/nbconvert_library.ipynb index 77c3c372a..e48c27361 100644 --- a/docs/source/nbconvert_library.ipynb +++ b/docs/source/nbconvert_library.ipynb @@ -349,7 +349,7 @@ "source": [ "There are an endless number of transformations that you may want to apply to a notebook. In particularly complicated cases, you may want to actually create your own `Preprocessor`. Above, when we customized the list of preprocessors accepted by the `HTMLExporter`, we passed in a string -- this can be any valid module name. So, if you create your own preprocessor, you can include it in that same list and it will be used by the exporter.\n", "\n", - "To create your own preprocessor, you will need to subclass from `nbconvert.preprocessors.Preprocessor` and overwrite either the `preprocess` and/or `preprocess_cell` methods." + "To create your own preprocessor, you will need to subclass from `nbconvert.preprocessors.Preprocessor` and overwrite either the `preprocess` and/or `preprocess_cell` methods. As of version 6.0, subclasses likely will need to call `super().preprocess_cell()` or something functionally equivalent to `NotebookClient.async_execute_cell`. In general, `super().preprocess_cell()` is best to minimize the complexity and subtlety of the multi-inheritance and overrides at play." ] }, { @@ -508,7 +508,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.2" + "version": "3.8.5" } }, "nbformat": 4,