22.8. Writing new Processing algorithms as Python scripts

There are two options for writing Processing algorithms using Python.

Within QGIS, you can use Create new script in the Scripts menu at the top of the Processing Toolbox to open the Processing Script Editor where you can write your code. To simplify the task, you can start with a script template by using Create new script from template from the same menu. This opens a template that extends QgsProcessingAlgorithm.

If you save the script in the scripts folder (the default location) with a .py extension, the algorithm will become available in the Processing Toolbox.

22.8.1. Extending QgsProcessingAlgorithm

The following code

  1. takes a vector layer as input

  2. counts the number of features

  3. does a buffer operation

  4. creates a raster layer from the result of the buffer operation

  5. returns the buffer layer, raster layer and number of features

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
from qgis.PyQt.QtCore import QCoreApplication
from qgis.core import (QgsProcessing,
                       QgsProcessingAlgorithm,
                       QgsProcessingException,
                       QgsProcessingOutputNumber,
                       QgsProcessingParameterDistance,
                       QgsProcessingParameterFeatureSource,
                       QgsProcessingParameterVectorDestination,
                       QgsProcessingParameterRasterDestination)
from qgis import processing


class ExampleProcessingAlgorithm(QgsProcessingAlgorithm):
    """
    This is an example algorithm that takes a vector layer,
    creates some new layers and returns some results.
    """

    def tr(self, string):
        """
        Returns a translatable string with the self.tr() function.
        """
        return QCoreApplication.translate('Processing', string)

    def createInstance(self):
        # Must return a new copy of your algorithm.
        return ExampleProcessingAlgorithm()

    def name(self):
        """
        Returns the unique algorithm name.
        """
        return 'bufferrasterextend'

    def displayName(self):
        """
        Returns the translated algorithm name.
        """
        return self.tr('Buffer and export to raster (extend)')

    def group(self):
        """
        Returns the name of the group this algorithm belongs to.
        """
        return self.tr('Example scripts')

    def groupId(self):
        """
        Returns the unique ID of the group this algorithm belongs
        to.
        """
        return 'examplescripts'

    def shortHelpString(self):
        """
        Returns a localised short help string for the algorithm.
        """
        return self.tr('Example algorithm short description')

    def initAlgorithm(self, config=None):
        """
        Here we define the inputs and outputs of the algorithm.
        """
        # 'INPUT' is the recommended name for the main input
        # parameter.
        self.addParameter(
            QgsProcessingParameterFeatureSource(
                'INPUT',
                self.tr('Input vector layer'),
                types=[QgsProcessing.TypeVectorAnyGeometry]
            )
        )
        self.addParameter(
            QgsProcessingParameterVectorDestination(
                'BUFFER_OUTPUT',
                self.tr('Buffer output'),
            )
        )
        # 'OUTPUT' is the recommended name for the main output
        # parameter.
        self.addParameter(
            QgsProcessingParameterRasterDestination(
                'OUTPUT',
                self.tr('Raster output')
            )
        )
        self.addParameter(
            QgsProcessingParameterDistance(
                'BUFFERDIST',
                self.tr('BUFFERDIST'),
                defaultValue = 1.0,
                # Make distance units match the INPUT layer units:
                parentParameterName='INPUT'
            )
        )
        self.addParameter(
            QgsProcessingParameterDistance(
                'CELLSIZE',
                self.tr('CELLSIZE'),
                defaultValue = 10.0,
                parentParameterName='INPUT'
            )
        )
        self.addOutput(
            QgsProcessingOutputNumber(
                'NUMBEROFFEATURES',
                self.tr('Number of features processed')
            )
        )

    def processAlgorithm(self, parameters, context, feedback):
        """
        Here is where the processing itself takes place.
        """
        # First, we get the count of features from the INPUT layer.
        # This layer is defined as a QgsProcessingParameterFeatureSource
        # parameter, so it is retrieved by calling
        # self.parameterAsSource.
        input_featuresource = self.parameterAsSource(parameters,
                                                     'INPUT',
                                                     context)
        numfeatures = input_featuresource.featureCount()

        # Retrieve the buffer distance and raster cell size numeric
        # values. Since these are numeric values, they are retrieved
        # using self.parameterAsDouble.
        bufferdist = self.parameterAsDouble(parameters, 'BUFFERDIST',
                                            context)
        rastercellsize = self.parameterAsDouble(parameters, 'CELLSIZE',
                                                context)
        if feedback.isCanceled():
            return {}
        buffer_result = processing.run(
            'native:buffer',
            {
                # Here we pass on the original parameter values of INPUT
                # and BUFFER_OUTPUT to the buffer algorithm.
                'INPUT': parameters['INPUT'],
                'OUTPUT': parameters['BUFFER_OUTPUT'],
                'DISTANCE': bufferdist,
                'SEGMENTS': 10,
                'DISSOLVE': True,
                'END_CAP_STYLE': 0,
                'JOIN_STYLE': 0,
                'MITER_LIMIT': 10
            },
            # Because the buffer algorithm is being run as a step in
            # another larger algorithm, the is_child_algorithm option
            # should be set to True
            is_child_algorithm=True,
            #
            # It's important to pass on the context and feedback objects to
            # child algorithms, so that they can properly give feedback to
            # users and handle cancelation requests.
            context=context,
            feedback=feedback)

        # Check for cancelation
        if feedback.isCanceled():
            return {}

        # Run the separate rasterization algorithm using the buffer result
        # as an input.
        rasterized_result = processing.run(
            'qgis:rasterize',
            {
                # Here we pass the 'OUTPUT' value from the buffer's result
                # dictionary off to the rasterize child algorithm.
                'LAYER': buffer_result['OUTPUT'],
                'EXTENT': buffer_result['OUTPUT'],
                'MAP_UNITS_PER_PIXEL': rastercellsize,
                # Use the original parameter value.
                'OUTPUT': parameters['OUTPUT']
            },
            is_child_algorithm=True,
            context=context,
            feedback=feedback)

        if feedback.isCanceled():
            return {}

        # Return the results
        return {'OUTPUT': rasterized_result['OUTPUT'],
                'BUFFER_OUTPUT': buffer_result['OUTPUT'],
                'NUMBEROFFEATURES': numfeatures}

Processing algorithm standard functions:

  • createInstance (mandatory)

    Must return a new copy of your algorithm. If you change the name of the class, make sure you also update the value returned here to match!

  • name (mandatory)

    Returns the unique algorithm name, used for identifying the algorithm.

  • displayName (mandatory)

    Returns the translated algorithm name.

  • group

    Returns the name of the group this algorithm belongs to.

  • groupId

    Returns the unique ID of the group this algorithm belongs to.

  • shortHelpString

    Returns a localised short help string for the algorithm.

  • initAlgorithm (mandatory)

    Here we define the inputs and outputs of the algorithm.

    INPUT and OUTPUT are recommended names for the main input and main output parameters, respectively.

    If a parameter depends on another parameter, parentParameterName is used to specify this relationship (could be the field / band of a layer or the distance units of a layer).

  • processAlgorithm (mandatory)

    This is where the processing takes place.

    Parameters are retrieved using special purpose functions, for instance parameterAsSource and parameterAsDouble.

    processing.run can be used to run other processing algorithms from a processing algorithm. The first parameter is the name of the algorithm, the second is a dictionary of the parameters to the algorithm. is_child_algorithm is normally set to True when running an algorithm from within another algorithm. context and feedback inform the algorithm about the environment to run in and the channel for communicating with the user (catching cancel request, reporting progress, providing textual feedback). When using the (parent) algorithm’s parameters as parameters to “child” algorithms, the original parameter values should be used (e.g. parameters['OUTPUT']).

    It is good practice to check the feedback object for cancelation as much as is sensibly possible! Doing so allows for responsive cancelation, instead of forcing users to wait for unwanted processing to occur.

    The algorithm should return values for all the output parameters it has defined as a dictionary. In this case, that’s the buffer and rasterized output layers, and the count of features processed. The dictionary keys must match the original parameter/output names.

22.8.2. The @alg decorator

Using the @alg decorator, you can create your own algorithms by writing the Python code and adding a few extra lines to supply additional information needed to make it a proper Processing algorithm. This simplifies the creation of algorithms and the specification of inputs and outputs.

One important limitation with the decorator approach is that algorithms created in this way will always be added to a user’s Processing Scripts provider – it is not possible to add these algorithms to a custom provider, e.g. for use in plugins.

The following code uses the @alg decorator to

  1. use a vector layer as input

  2. count the number of features

  3. do a buffer operation

  4. create a raster layer from the result of the buffer operation

  5. returns the buffer layer, raster layer and number of features

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
from qgis import processing
from qgis.processing import alg
from qgis.core import QgsProject

@alg(name='bufferrasteralg', label='Buffer and export to raster (alg)',
     group='examplescripts', group_label='Example scripts')
# 'INPUT' is the recommended name for the main input parameter
@alg.input(type=alg.SOURCE, name='INPUT', label='Input vector layer')
# 'OUTPUT' is the recommended name for the main output parameter
@alg.input(type=alg.RASTER_LAYER_DEST, name='OUTPUT',
           label='Raster output')
@alg.input(type=alg.VECTOR_LAYER_DEST, name='BUFFER_OUTPUT',
           label='Buffer output')
@alg.input(type=alg.DISTANCE, name='BUFFERDIST', label='BUFFER DISTANCE',
           default=1.0)
@alg.input(type=alg.DISTANCE, name='CELLSIZE', label='RASTER CELL SIZE',
           default=10.0)
@alg.output(type=alg.NUMBER, name='NUMBEROFFEATURES',
            label='Number of features processed')

def bufferrasteralg(instance, parameters, context, feedback, inputs):
    """
    Description of the algorithm.
    (If there is no comment here, you will get an error)
    """
    input_featuresource = instance.parameterAsSource(parameters,
                                                     'INPUT', context)
    numfeatures = input_featuresource.featureCount()
    bufferdist = instance.parameterAsDouble(parameters, 'BUFFERDIST',
                                            context)
    rastercellsize = instance.parameterAsDouble(parameters, 'CELLSIZE',
                                                context)
    if feedback.isCanceled():
        return {}
    buffer_result = processing.run('native:buffer',
                               {'INPUT': parameters['INPUT'],
                                'OUTPUT': parameters['BUFFER_OUTPUT'],
                                'DISTANCE': bufferdist,
                                'SEGMENTS': 10,
                                'DISSOLVE': True,
                                'END_CAP_STYLE': 0,
                                'JOIN_STYLE': 0,
                                'MITER_LIMIT': 10
                                },
                               is_child_algorithm=True,
                               context=context,
                               feedback=feedback)
    if feedback.isCanceled():
        return {}
    rasterized_result = processing.run('qgis:rasterize',
                               {'LAYER': buffer_result['OUTPUT'],
                                'EXTENT': buffer_result['OUTPUT'],
                                'MAP_UNITS_PER_PIXEL': rastercellsize,
                                'OUTPUT': parameters['OUTPUT']
                               },
                               is_child_algorithm=True, context=context,
                               feedback=feedback)
    if feedback.isCanceled():
        return {}
    return {'OUTPUT': rasterized_result['OUTPUT'],
            'BUFFER_OUTPUT': buffer_result['OUTPUT'],
            'NUMBEROFFEATURES': numfeatures}

As you can see, it involves two algorithms (‘native:buffer’ and ‘qgis:rasterize’). The last one (‘qgis:rasterize’) creates a raster layer from the buffer layer that was generated by the first one (‘native:buffer’).

The part of the code where this processing takes place is not difficult to understand if you have read the previous chapter. The first lines, however, need some additional explanation. They provide the information that is needed to turn your code into an algorithm that can be run from any of the GUI components, like the toolbox or the graphical modeler.

These lines are all calls to the @alg decorator functions that help simplify the coding of the algorithm.

  • The @alg decorator is used to define the name and location of the algorithm in the Toolbox.

  • The @alg.input decorator is used to define the inputs of the algorithm.

  • The @alg.output decorator is used to define the outputs of the algorithm.

22.8.3. Input and output types for Processing Algorithms

Here is the list of input and output types that are supported in Processing with their corresponding alg decorator constants (algfactory.py contains the complete list of alg constants). Sorted on class name.

22.8.3.1. Input types

Class

Alg constant

Description

QgsProcessingParameterAuthConfig

alg.AUTH_CFG

Allows users to select from available authentication configurations or create new authentication configurations

QgsProcessingParameterBand

alg.BAND

A band of a raster layer

QgsProcessingParameterBoolean

alg.BOOL

A boolean value

QgsProcessingParameterColor

alg.COLOR

A color

QgsProcessingParameterCrs

alg.CRS

A Coordinate Reference System

QgsProcessingParameterDistance

alg.DISTANCE

A double numeric parameter for distance values

QgsProcessingParameterEnum

alg.ENUM

An enumeration, allowing for selection from a set of predefined values

QgsProcessingParameterExpression

alg.EXPRESSION

An expression

QgsProcessingParameterExtent

alg.EXTENT

A spatial extent defined by xmin, xmax, ymin, ymax

QgsProcessingParameterField

alg.FIELD

A field in the attribute table of a vector layer

QgsProcessingParameterFile

alg.FILE

A filename of an existing file

QgsProcessingParameterFileDestination

alg.FILE_DEST

A filename for a newly created output file

QgsProcessingParameterFolderDestination

alg.FOLDER_DEST

A folder

QgsProcessingParameterNumber

alg.INT

An integer

QgsProcessingParameterLayout

alg.LAYOUT

A layout

QgsProcessingParameterLayoutItem

alg.LAYOUT_ITEM

A layout item

QgsProcessingParameterMapLayer

alg.MAPLAYER

A map layer

QgsProcessingParameterMatrix

alg.MATRIX

A matrix

QgsProcessingParameterMeshLayer

alg.MESH_LAYER

A mesh layer

QgsProcessingParameterMultipleLayers

alg.MULTILAYER

A set of layers

QgsProcessingParameterNumber

alg.NUMBER

A numerical value

QgsProcessingParameterPoint

alg.POINT

A point

QgsProcessingParameterRange

alg.RANGE

A number range

QgsProcessingParameterRasterLayer

alg.RASTER_LAYER

A raster layer

QgsProcessingParameterRasterDestination

alg.RASTER_LAYER_DEST

A raster layer

QgsProcessingParameterScale

alg.SCALE

A map scale

QgsProcessingParameterFeatureSink

alg.SINK

A feature sink

QgsProcessingParameterFeatureSource

alg.SOURCE

A feature source

QgsProcessingParameterScale

A map scale

QgsProcessingParameterString

alg.STRING

A text string

QgsProcessingParameterVectorLayer

alg.VECTOR_LAYER

A vector layer

QgsProcessingParameterVectorDestination

alg.VECTOR_LAYER_DEST

A vector layer

22.8.3.2. Output types

Class

Alg constant

Description

QgsProcessingOutputBoolean

alg.BOOL

A boolean value

QgsProcessingOutputNumber

alg.DISTANCE

A double numeric parameter for distance values

QgsProcessingOutputFile

alg.FILE

A filename of an existing file

QgsProcessingOutputFolder

alg.FOLDER

A folder

QgsProcessingOutputHtml

alg.HTML

HTML

QgsProcessingOutputNumber

alg.INT

A integer

QgsProcessingOutputLayerDefinition

alg.LAYERDEF

A layer definition

QgsProcessingOutputMapLayer

alg.MAPLAYER

A map layer

QgsProcessingOutputMultipleLayers

alg.MULTILAYER

A set of layers

QgsProcessingOutputNumber

alg.NUMBER

A numerical value

QgsProcessingOutputRasterLayer

alg.RASTER_LAYER

A raster layer

QgsProcessingOutputString

alg.STRING

A text string

QgsProcessingOutputVectorLayer

alg.VECTOR_LAYER

A vector layer

22.8.4. Handing algorithm output

When you declare an output representing a layer (raster or vector), the algorithm will try to add it to QGIS once it is finished.

  • Raster layer output: QgsProcessingParameterRasterDestination / alg.RASTER_LAYER_DEST.

  • Vector layer output: QgsProcessingParameterVectorDestination / alg.VECTOR_LAYER_DEST.

So even if the processing.run() method does not add the layers it creates to the user’s current project, the two output layers (buffer and raster buffer) will be loaded, since they are saved to the destinations entered by the user (or to temporary destinations if the user does not specify destinations).

If a layer is created as output of an algorithm, it should be declared as such. Otherwise, you will not be able to properly use the algorithm in the modeler, since what is declared will not match what the algorithm really creates.

You can return strings, numbers and more by specifying them in the result dictionary (as demonstrated for “NUMBEROFFEATURES”), but they should always be explicitly defined as outputs from your algorithm. We encourage algorithms to output as many useful values as possible, since these can be valuable for use in later algorithms when your algorithm is used as part of a model.

22.8.5. Communicating with the user

If your algorithm takes a long time to process, it is a good idea to inform the user about the progress. You can use feedback (QgsProcessingFeedback) for this.

The progress text and progressbar can be updated using two methods: setProgressText(text) and setProgress(percent).

You can provide more information by using pushCommandInfo(text), pushDebugInfo(text), pushInfo(text) and reportError(text).

If your script has a problem, the correct way of handling it is to raise a QgsProcessingException. You can pass a message as an argument to the constructor of the exception. Processing will take care of handling it and communicating with the user, depending on where the algorithm is being executed from (toolbox, modeler, Python console, …)

22.8.6. Documenting your scripts

You can document your scripts by overloading the helpString() and helpUrl() methods of QgsProcessingAlgorithm.

22.8.7. Flags

You can override the flags method of QgsProcessingAlgorithm to tell QGIS more about your algorithm. You can for instance tell QGIS that the script shall be hidden from the modeler, that it can be canceled, that it is not thread safe, and more.

Tip

By default, Processing runs algorithms in a separate thread in order to keep QGIS responsive while the processing task runs. If your algorithm is regularly crashing, you are probably using API calls which are not safe to do in a background thread. Try returning the QgsProcessingAlgorithm.FlagNoThreading flag from your algorithm’s flags() method to force Processing to run your algorithm in the main thread instead.

22.8.8. Best practices for writing script algorithms

Here’s a quick summary of ideas to consider when creating your script algorithms and, especially, if you want to share them with other QGIS users. Following these simple rules will ensure consistency across the different Processing elements such as the toolbox, the modeler or the batch processing interface.

  • Do not load resulting layers. Let Processing handle your results and load your layers if needed.

  • Always declare the outputs your algorithm creates.

  • Do not show message boxes or use any GUI element from the script. If you want to communicate with the user, use the methods of the feedback object (QgsProcessingFeedback) or throw a QgsProcessingException.

There are already many processing algorithms available in QGIS. You can find code on https://github.com/qgis/QGIS/blob/release-3_10/python/plugins/processing/algs/qgis.