.. _pyqgis_testing: *********************************** Writing code in the PyQGIS Cookbook *********************************** .. contents:: :local: If you are planning to add or update some chapters of the :ref:`PyQGIS-Developer-Cookbook`, then you should follow some rules to enable automatic testing of the code snippets. Testing is really important because it allows automatic checking of the code. Code snippets with errors or code that uses outdated methods will fail and the notification will help you fix the problems. For testing, we use the `Sphinx doctest extension `_. Refer to the extension documentation for more detailed information. How to write testable code snippets =================================== Writing testable code snippets is not so different from the *old* method. Basically, you need to use a different Sphinx `directive`. Doctest sphinx directives ------------------------- Instead of embedding the code in a ``.. code-block:: python`` directive (which would highlight the code syntax automatically), you now need to embed it in a ``.. testcode::``. That is, instead of this:: .. code-block:: python crs = QgsCoordinateReferenceSystem(4326, QgsCoordinateReferenceSystem.PostgisCrsId) assert crs.isValid() You now use this:: .. testcode:: crs = QgsCoordinateReferenceSystem(4326, QgsCoordinateReferenceSystem.PostgisCrsId) assert crs.isValid() After you wrote the example code, you should add some *assertion* that will evaluate the code and that will be run automatically. In the above example, you are creating a crs and with ``assert crs.isValid()`` you **test** if it is valid. If the code has a wrong python syntax or the ``crs.isValid()`` returns ``False``, this code snippet will fail during testing. To successfully run the tests on snippets, you must import all the classes and declare any variables used in the code snippets. You can include those in the code snippet itself (visible in the HTML pages) or you can add them to a ``.. testsetup::`` directive (hidden in the HTML pages). The ``.. testsetup::`` needs to be placed before the ``.. testcode::``:: .. testsetup:: from qgis.core import QgsCoordinateReferenceSystem .. testcode:: crs = QgsCoordinateReferenceSystem(4326, QgsCoordinateReferenceSystem.PostgisCrsId) assert crs.isValid() If the code snippet doesn't create objects (and therefore you cannot use something like ``assert object.isValid()``), you can test the code using the ``print()`` method, then add the expected results within a ``.. testoutput::`` directive to compare the expected output:: .. testcode:: print("QGIS CRS ID:", crs.srsid()) print("PostGIS SRID:", crs.postgisSrid()) .. testoutput:: QGIS CRS ID: 3452 PostGIS SRID: 4326 By default, the content of ``.. testoutput::`` is shown in the HTML output. To hide it from the HTML use the `:hide:` option:: .. testoutput:: :hide: QGIS CRS ID: 3452 PostGIS SRID: 4326 .. note:: If the code snippet contains any print statements, you MUST add a ``testoutput`` with the expected outputs; otherwise the test will fail. Grouping tests ---------------------------- For each rst document, the code snippets are tested sequentially, which means you can use one ``.. testsetup::`` for all the following code snippets and that later snippets will have access to variables declared in earlier ones in the document. Alternatively, you can use groups to break down the examples on the same page in different tests. You add the code snippet to groups by adding one or more group names (separated by commas) in the respective directive:: .. testcode:: crs_crsfromID [, morenames] crs = QgsCoordinateReferenceSystem(4326, QgsCoordinateReferenceSystem.PostgisCrsId) assert crs.isValid() The ``doctest`` will pick each group snippets and run them independently. .. note:: Use group names that make sense with the related content. Use something similar to _, for example: crs_intro, crs_fromwkt. In case of failures, this will help identifying where the failures occur. If you don't declare any group, the code snippet will be added to a group named ``default``. If instead, you use ``*`` as a group name, the snippet will be used in all testing groups, something normally usefull to use in the test setup:: .. testsetup:: * from qgis.core import QgsCoordinateReferenceSystem How to test snippets on your local machine ========================================== .. note:: Instructions are valid for Linux system. To test Python code snippets, you need a *QGIS* installation. For this, there are many options. You can: * Use your system *QGIS* installation with *Sphinx* from a Python virtual environment:: make -f venv.mk doctest * Use a manually built installation of *QGIS*. You'd need to: #. Create a custom ``Makefile`` extension on top of the :file:`venv.mk` file, for example a :file:`user.mk` file with the following content:: # Root installation folder QGIS_PREFIX_PATH = /home/user/apps/qgis-master include venv.mk Or :: # build output folder QGIS_PREFIX_PATH = /home/user/dev/QGIS-build-master/output include venv.mk #. Then, use it to run target ``doctest``:: make -f user.mk doctest * Run target ``doctest`` inside the official *QGIS* docker image:: make -f docker.mk doctest You have to install `Docker `_ first because this uses a docker image with QGIS in it.