[Panelize][Added] copy_vias_on_mask option

- To workaround KiCad's bug 18991
- Also added warning about how to workaround it in plotter outputs

Closes #703 related to #704

CHANGELOG.md

@@ -19,6 +19,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Report: thickness units (#685)
 - PCB Print:
   - Worksheet: undocumented font face and color now can be used (See #695)
+- Panelize:
+  - `copy_vias_on_mask` option to workaround KiCad's bug 18991 (See #703 and
+    #704)
 
 ### Fixed
 - BoM: Sub-PCBs not applied (#697)

docs/samples/generic_plot.kibot.yaml

@@ -1094,7 +1094,8 @@ outputs:
       sketch_pads_on_fab_layers: false
       # [boolean=false] Don't fill objects, just draw the outline
       sketch_plot: false
-      # [boolean=true] Cover the vias
+      # [boolean=true] Cover the vias.
+      # Warning: KiCad 8 has a bug that ignores this option. Set it from KiCad GUI
       tent_vias: true
       # [boolean=false] Use uppercase names for the extensions
       uppercase_extensions: false
@@ -1322,7 +1323,8 @@ outputs:
       sketch_pads_on_fab_layers: false
       # [boolean=false] Subtract the solder mask from the silk screen
       subtract_mask_from_silk: false
-      # [boolean=true] Cover the vias
+      # [boolean=true] Cover the vias.
+      # Warning: KiCad 8 has a bug that ignores this option. Set it from KiCad GUI
       tent_vias: true
       # [boolean=false] Use uppercase names for the extensions
       uppercase_extensions: false
@@ -1405,7 +1407,8 @@ outputs:
       sketch_pads_on_fab_layers: false
       # [boolean=false] Don't fill objects, just draw the outline
       sketch_plot: false
-      # [boolean=true] Cover the vias
+      # [boolean=true] Cover the vias.
+      # Warning: KiCad 8 has a bug that ignores this option. Set it from KiCad GUI
       tent_vias: true
       # [boolean=false] Use uppercase names for the extensions
       uppercase_extensions: false
@@ -2501,6 +2504,14 @@ outputs:
             type: 'none'
             # [number|string=0] Vertical offset from panel edges
             voffset: 0
+      # [string='auto'] [auto,yes,no] Copy the GUI option to plot vias on the mask layers from the original PCB to
+      # the panel.
+      # This option is a workaround to KiCad 8 not allowing to choose to plot (or not to plot) vias
+      # on the mask layers using the Python API. So you have to set it in the GUI, but this option
+      # is lost during panelization.
+      # Setting this option to *auto* will copy the value for faulty KiCad 8 versions, but won't
+      # waste time for working KiCad versions
+      copy_vias_on_mask: 'auto'
       # [boolean=false] Use PcbDraw to create a preview of the panel
       create_preview: false
       # [string='deg'] [deg,°,rad] Angles used when omitted
@@ -3017,7 +3028,8 @@ outputs:
       sketch_pad_line_width: 0.1
       # [boolean=false] Draw only the outline of the pads on the *.Fab layers (KiCad 6+)
       sketch_pads_on_fab_layers: false
-      # [boolean=true] Cover the vias
+      # [boolean=true] Cover the vias.
+      # Warning: KiCad 8 has a bug that ignores this option. Set it from KiCad GUI
       tent_vias: true
       # [boolean=false] Use uppercase names for the extensions
       uppercase_extensions: false
@@ -3290,7 +3302,8 @@ outputs:
       sketch_pads_on_fab_layers: false
       # [boolean=false] Don't fill objects, just draw the outline
       sketch_plot: false
-      # [boolean=true] Cover the vias
+      # [boolean=true] Cover the vias.
+      # Warning: KiCad 8 has a bug that ignores this option. Set it from KiCad GUI
       tent_vias: true
       # [boolean=false] Use uppercase names for the extensions
       uppercase_extensions: false
@@ -3845,7 +3858,8 @@ outputs:
       # The value is how much zeros has the multiplier (1 mm = 10 power `svg_precision` units).
       # Note that for an A4 paper Firefox 91 and Chrome 105 can't handle more than 5
       svg_precision: 4
-      # [boolean=true] Cover the vias
+      # [boolean=true] Cover the vias.
+      # Warning: KiCad 8 has a bug that ignores this option. Set it from KiCad GUI
       tent_vias: true
       # [boolean=false] Use uppercase names for the extensions
       uppercase_extensions: false

docs/source/Changelog.rst

@@ -38,6 +38,11 @@ Added
    -  Worksheet: undocumented font face and color now can be used (See
       #695)
 
+-  Panelize:
+
+   -  ``copy_vias_on_mask`` option to workaround KiCad’s bug 18991 (See
+      #703 and #704)
+
 Fixed
 ~~~~~
 

docs/source/configuration/outputs/DXFOptions.rst

@@ -39,6 +39,11 @@ DXFOptions parameters
 -  ``sketch_pads_on_fab_layers`` :index:`: <pair: output - dxf - options; sketch_pads_on_fab_layers>` [:ref:`boolean <boolean>`] (default: ``false``) Draw only the outline of the pads on the \\*.Fab layers (KiCad 6+).
 -  ``sketch_plot`` :index:`: <pair: output - dxf - options; sketch_plot>` [:ref:`boolean <boolean>`] (default: ``false``) Don't fill objects, just draw the outline.
 -  ``tent_vias`` :index:`: <pair: output - dxf - options; tent_vias>` [:ref:`boolean <boolean>`] (default: ``true``) Cover the vias.
+
+.. warning::
+   KiCad 8 has a bug that ignores this option. Set it from KiCad GUI.
+..
+
 -  ``uppercase_extensions`` :index:`: <pair: output - dxf - options; uppercase_extensions>` [:ref:`boolean <boolean>`] (default: ``false``) Use uppercase names for the extensions.
 -  ``use_aux_axis_as_origin`` :index:`: <pair: output - dxf - options; use_aux_axis_as_origin>` [:ref:`boolean <boolean>`] (default: ``false``) Use the auxiliary axis as origin for coordinates.
 -  ``variant`` :index:`: <pair: output - dxf - options; variant>` [:ref:`string <string>`] (default: ``''``) Board variant to apply.

docs/source/configuration/outputs/GerberOptions.rst

@@ -41,6 +41,11 @@ GerberOptions parameters
    Note that this value is currently ignored by KiCad (6.0.9).
 -  ``sketch_pads_on_fab_layers`` :index:`: <pair: output - gerber - options; sketch_pads_on_fab_layers>` [:ref:`boolean <boolean>`] (default: ``false``) Draw only the outline of the pads on the \\*.Fab layers (KiCad 6+).
 -  ``tent_vias`` :index:`: <pair: output - gerber - options; tent_vias>` [:ref:`boolean <boolean>`] (default: ``true``) Cover the vias.
+
+.. warning::
+   KiCad 8 has a bug that ignores this option. Set it from KiCad GUI.
+..
+
 -  ``uppercase_extensions`` :index:`: <pair: output - gerber - options; uppercase_extensions>` [:ref:`boolean <boolean>`] (default: ``false``) Use uppercase names for the extensions.
 -  ``use_aux_axis_as_origin`` :index:`: <pair: output - gerber - options; use_aux_axis_as_origin>` [:ref:`boolean <boolean>`] (default: ``false``) Use the auxiliary axis as origin for coordinates.
 -  ``variant`` :index:`: <pair: output - gerber - options; variant>` [:ref:`string <string>`] (default: ``''``) Board variant to apply.

docs/source/configuration/outputs/HPGLOptions.rst

@@ -40,6 +40,11 @@ HPGLOptions parameters
 -  ``sketch_pads_on_fab_layers`` :index:`: <pair: output - hpgl - options; sketch_pads_on_fab_layers>` [:ref:`boolean <boolean>`] (default: ``false``) Draw only the outline of the pads on the \\*.Fab layers (KiCad 6+).
 -  ``sketch_plot`` :index:`: <pair: output - hpgl - options; sketch_plot>` [:ref:`boolean <boolean>`] (default: ``false``) Don't fill objects, just draw the outline.
 -  ``tent_vias`` :index:`: <pair: output - hpgl - options; tent_vias>` [:ref:`boolean <boolean>`] (default: ``true``) Cover the vias.
+
+.. warning::
+   KiCad 8 has a bug that ignores this option. Set it from KiCad GUI.
+..
+
 -  ``uppercase_extensions`` :index:`: <pair: output - hpgl - options; uppercase_extensions>` [:ref:`boolean <boolean>`] (default: ``false``) Use uppercase names for the extensions.
 -  ``variant`` :index:`: <pair: output - hpgl - options; variant>` [:ref:`string <string>`] (default: ``''``) Board variant to apply.
 

docs/source/configuration/outputs/PDFOptions.rst

@@ -38,6 +38,11 @@ PDFOptions parameters
    Note that this value is currently ignored by KiCad (6.0.9).
 -  ``sketch_pads_on_fab_layers`` :index:`: <pair: output - pdf - options; sketch_pads_on_fab_layers>` [:ref:`boolean <boolean>`] (default: ``false``) Draw only the outline of the pads on the \\*.Fab layers (KiCad 6+).
 -  ``tent_vias`` :index:`: <pair: output - pdf - options; tent_vias>` [:ref:`boolean <boolean>`] (default: ``true``) Cover the vias.
+
+.. warning::
+   KiCad 8 has a bug that ignores this option. Set it from KiCad GUI.
+..
+
 -  ``uppercase_extensions`` :index:`: <pair: output - pdf - options; uppercase_extensions>` [:ref:`boolean <boolean>`] (default: ``false``) Use uppercase names for the extensions.
 -  ``variant`` :index:`: <pair: output - pdf - options; variant>` [:ref:`string <string>`] (default: ``''``) Board variant to apply.
 

docs/source/configuration/outputs/PSOptions.rst

@@ -42,6 +42,11 @@ PSOptions parameters
 -  ``sketch_pads_on_fab_layers`` :index:`: <pair: output - ps - options; sketch_pads_on_fab_layers>` [:ref:`boolean <boolean>`] (default: ``false``) Draw only the outline of the pads on the \\*.Fab layers (KiCad 6+).
 -  ``sketch_plot`` :index:`: <pair: output - ps - options; sketch_plot>` [:ref:`boolean <boolean>`] (default: ``false``) Don't fill objects, just draw the outline.
 -  ``tent_vias`` :index:`: <pair: output - ps - options; tent_vias>` [:ref:`boolean <boolean>`] (default: ``true``) Cover the vias.
+
+.. warning::
+   KiCad 8 has a bug that ignores this option. Set it from KiCad GUI.
+..
+
 -  ``uppercase_extensions`` :index:`: <pair: output - ps - options; uppercase_extensions>` [:ref:`boolean <boolean>`] (default: ``false``) Use uppercase names for the extensions.
 -  ``variant`` :index:`: <pair: output - ps - options; variant>` [:ref:`string <string>`] (default: ``''``) Board variant to apply.
 -  ``width_adjust`` :index:`: <pair: output - ps - options; width_adjust>` [:ref:`number <number>`] (default: ``0``) This width factor is intended to compensate PS printers/plotters that do not strictly obey line width settings.

docs/source/configuration/outputs/PanelizeOptions.rst

@@ -9,6 +9,13 @@ PanelizeOptions parameters
    You can also include a preset using `:name`, i.e. `:vcuts`.
    Use a dict to specify the options using the KiBot YAML file.
 -  **output** :index:`: <pair: output - panelize - options; output>` [:ref:`string <string>`] (default: ``'%f-%i%I%v.%x'``) Filename for the output (%i=panel, %x=kicad_pcb). Affected by global options.
+-  ``copy_vias_on_mask`` :index:`: <pair: output - panelize - options; copy_vias_on_mask>` [:ref:`string <string>`] (default: ``'auto'``) (choices: "auto", "yes", "no") Copy the GUI option to plot vias on the mask layers from the original PCB to
+   the panel. |br|
+   This option is a workaround to KiCad 8 not allowing to choose to plot (or not to plot) vias
+   on the mask layers using the Python API. So you have to set it in the GUI, but this option
+   is lost during panelization. |br|
+   Setting this option to *auto* will copy the value for faulty KiCad 8 versions, but won't
+   waste time for working KiCad versions.
 -  ``create_preview`` :index:`: <pair: output - panelize - options; create_preview>` [:ref:`boolean <boolean>`] (default: ``false``) Use PcbDraw to create a preview of the panel.
 -  ``default_angles`` :index:`: <pair: output - panelize - options; default_angles>` [:ref:`string <string>`] (default: ``'deg'``) (choices: "deg", "°", "rad") Angles used when omitted.
 -  ``dnf_filter`` :index:`: <pair: output - panelize - options; dnf_filter>` [:ref:`string <string>` | :ref:`list(string) <list(string)>`] (default: ``'_null'``) Name of the filter to mark components as not fitted.

docs/source/configuration/outputs/SVGOptions.rst

@@ -52,6 +52,11 @@ SVGOptions parameters
    The value is how much zeros has the multiplier (1 mm = 10 power `svg_precision` units).
    Note that for an A4 paper Firefox 91 and Chrome 105 can't handle more than 5.
 -  ``tent_vias`` :index:`: <pair: output - svg - options; tent_vias>` [:ref:`boolean <boolean>`] (default: ``true``) Cover the vias.
+
+.. warning::
+   KiCad 8 has a bug that ignores this option. Set it from KiCad GUI.
+..
+
 -  ``uppercase_extensions`` :index:`: <pair: output - svg - options; uppercase_extensions>` [:ref:`boolean <boolean>`] (default: ``false``) Use uppercase names for the extensions.
 -  ``use_aux_axis_as_origin`` :index:`: <pair: output - svg - options; use_aux_axis_as_origin>` [:ref:`boolean <boolean>`] (default: ``false``) Use the auxiliary axis as origin for coordinates.
 -  ``variant`` :index:`: <pair: output - svg - options; variant>` [:ref:`string <string>`] (default: ``''``) Board variant to apply.

docs/source/configuration/outputs/pdf.rst

@@ -79,6 +79,11 @@ Parameters:
    Note that this value is currently ignored by KiCad (6.0.9).
 -  ``sketch_pads_on_fab_layers`` :index:`: <pair: output - pdf; sketch_pads_on_fab_layers>` [:ref:`boolean <boolean>`] (default: ``false``) Draw only the outline of the pads on the \\*.Fab layers (KiCad 6+).
 -  ``tent_vias`` :index:`: <pair: output - pdf; tent_vias>` [:ref:`boolean <boolean>`] (default: ``true``) Cover the vias.
+
+.. warning::
+   KiCad 8 has a bug that ignores this option. Set it from KiCad GUI.
+..
+
 -  ``uppercase_extensions`` :index:`: <pair: output - pdf; uppercase_extensions>` [:ref:`boolean <boolean>`] (default: ``false``) Use uppercase names for the extensions.
 -  ``variant`` :index:`: <pair: output - pdf; variant>` [:ref:`string <string>`] (default: ``''``) Board variant to apply.
 

kibot/config_reader.py

@@ -1322,7 +1322,7 @@ def print_example_options(f, cls, name, indent, po, is_list=False):
             f.write(ind_str+'# `{}` is an alias for `{}`\n'.format(k, alias))
             continue
         if help:
-            help_lines = help.split('\n')
+            help_lines = help.replace('|br|', '').split('\n')
             for hl in help_lines:
                 # Dots at the beginning are replaced by spaces.
                 # Used to keep indentation.

kibot/out_any_layer.py

@@ -67,7 +67,8 @@ def __init__(self):
                 For this reason you must avoid generating two variants at the same directory when one of
                 them uses the default KiCad name """
             self.tent_vias = True
-            """ Cover the vias """
+            """ Cover the vias.
+                Warning: KiCad 8 has a bug that ignores this option. Set it from KiCad GUI """
             self.uppercase_extensions = False
             """ Use uppercase names for the extensions """
             self.inner_extension_pattern = ''

kibot/out_panelize.py

@@ -664,6 +664,14 @@ def __init__(self):
             """ [deg,°,rad] Angles used when omitted """
             self.create_preview = False
             """ Use PcbDraw to create a preview of the panel """
+            self.copy_vias_on_mask = 'auto'
+            """ [auto,yes,no] Copy the GUI option to plot vias on the mask layers from the original PCB to
+                the panel. |br|
+                This option is a workaround to KiCad 8 not allowing to choose to plot (or not to plot) vias
+                on the mask layers using the Python API. So you have to set it in the GUI, but this option
+                is lost during panelization. |br|
+                Setting this option to *auto* will copy the value for faulty KiCad 8 versions, but won't
+                waste time for working KiCad versions """
         super().__init__()
         self._expand_id = 'panel'
         self._expand_ext = 'kicad_pcb'
@@ -766,6 +774,38 @@ def create_config(self, cfg):
         logger.debugl(1, js)
         return GS.tmp_file(content=js, suffix='.json', prefix='panel_cfg', what='panel config', a_logger=logger)
 
+    def do_copy_vias_on_mask(self, old, new):
+        """ KiCad 8 has a bug that prevents choosing if vias are or aren't plotted in the solder mask layer.
+            Only the setting in the PCB file is used, the API is ignored.
+            KiKit resets this option, most probably unintentionally.
+            So here we read the old PCB and we see if the new one needs to be updated. """
+        logger.debug('Copying viasonmask option')
+        # Read the old setting
+        logger.debug('- Reading the old PCB')
+        with open(old) as f:
+            old_pcb = f.read()
+        pattern = re.compile(r'\(viasonmask (.*)\)')
+        match = pattern.search(old_pcb)
+        if not match:
+            logger.debug("- The old PCB doesn't have the option!")
+            return
+        old_value = match.group(1)
+        logger.debug('- Reading the new PCB')
+        with open(new) as f:
+            new_pcb = f.read()
+        match = pattern.search(new_pcb)
+        if not match:
+            logger.debug("- The new PCB doesn't have the option!")
+            return
+        new_value = match.group(1)
+        if old_value == new_value:
+            logger.debug(f'- No need to change (both {new_value})')
+            return
+        new_pcb = new_pcb.replace('(viasonmask '+new_value, '(viasonmask '+old_value)
+        logger.debug(f'- Saving the panel PCB with `{old_value}`')
+        with open(new, 'w') as f:
+            f.write(new_pcb)
+
     def create_preview_file(self, name):
         if not self.create_preview or not os.path.isfile(name):
             return
@@ -816,6 +856,8 @@ def run(self, output):
             self.create_preview_file(output)
             remove_tmps = True
         finally:
+            if (self.copy_vias_on_mask == 'auto' and GS.ki8) or self.copy_vias_on_mask == 'yes':
+                self.do_copy_vias_on_mask(fname, output)
             if GS.debug_enabled and not remove_tmps:
                 if self._files_to_remove:
                     logger.warning(W_KEEPTMP+'Keeping temporal files: '+str(self._files_to_remove))

tests/GUI/outputs

@@ -7765,6 +7765,13 @@
           ],
           null
         ],
+        [
+          "copy_vias_on_mask",
+          [
+            "DataTypeChoice"
+          ],
+          null
+        ],
         [
           "variant",
           [

tests/GUI/stats

@@ -1,8 +1,8 @@
-48 outputs types with a total of 1873 different parameters
-Single type parameters: 1699 (91 %)
+48 outputs types with a total of 1874 different parameters
+Single type parameters: 1700 (91 %)
 Multi type parameters: 174 (9 %)
 Average parameters: 39
-Maximum number of parameters: 213
+Maximum number of parameters: 214
 Minimum number of parameters: 14
 Histogram:
   0-  9: 
@@ -75,7 +75,7 @@ Outputs sorted by parameters:
 - blender_export: 75
 - pcb_print: 89
 - bom: 145
-- panelize: 213
+- panelize: 214
 Average depth: 2.5416666666666665
 Maximum depth: 5
 Minimum depth: 2
@@ -141,7 +141,7 @@ Outputs sorted by depth:
 -            Boolean:  469
 -             Number:  266
 - ListStringSingular:  222
--             Choice:   94
+-             Choice:   95
 -               Dict:   79
 -   ListDictOrString:   21
 -           ListDict:   20
@@ -157,7 +157,7 @@ Used as single data type:
 -            Boolean:  413 
 - ListStringSingular:  219 
 -             Number:  172 
--             Choice:   91 
+-             Choice:   92 
 -               Dict:   70 
 -           ListDict:   20 
 -         ListString:    9 ['blender_export.options.pcb3d.highlight', 'bom.options.group_fields', 'bom.options.group_fields_fallbacks', 'bom.options.no_conflict', 'kibom.options.conf.group_fields', 'pcbdraw.options.libs', 'pcbdraw.options.highlight', 'render_3d.options.highlight', 'vrml.options.highlight']
@@ -390,11 +390,11 @@ Used as single data type:
 -              Number,String:   15 
 ================================================================================
 ================================================================================
-86 totals types with a total of 2241 different parameters
-Single type parameters: 2043 (91 %)
+86 totals types with a total of 2242 different parameters
+Single type parameters: 2044 (91 %)
 Multi type parameters: 198 (9 %)
 Average parameters: 26
-Maximum number of parameters: 213
+Maximum number of parameters: 214
 Minimum number of parameters: 1
 Histogram:
   0-  9: ***********************
@@ -505,7 +505,7 @@ Totals sorted by parameters:
 - blender_export: 75
 - pcb_print: 89
 - bom: 145
-- panelize: 213
+- panelize: 214
 Average depth: 2.186046511627907
 Maximum depth: 5
 Minimum depth: 1
@@ -609,7 +609,7 @@ Totals sorted by depth:
 -            Boolean:  567
 -             Number:  314
 - ListStringSingular:  251
--             Choice:  126
+-             Choice:  127
 -               Dict:   90
 -           ListDict:   30
 -   ListDictOrString:   24
@@ -625,7 +625,7 @@ Used as single data type:
 -            Boolean:  503 
 - ListStringSingular:  248 
 -             Number:  205 
--             Choice:  123 
+-             Choice:  124 
 -               Dict:   73 
 -           ListDict:   30 
 -         ListString:   12