[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-kimageshop
Subject:    Re: Scripting Initial Report
From:       Raghavendra Kamath <raghu () raghukamath ! com>
Date:       2017-02-15 14:58:55
Message-ID: 2338753.S5ZGaFR3nd () raghu
[Download RAW message or body]

On Wednesday, February 15, 2017 7:04:33 PM IST Boudewijn Rempt wrote:
> I'm pretty sure that we're on the right track now, and getting close
> to a first usable version, so I spent today finishing up my research 
> report on scripting. Permanent copy is here: https://phabricator.kde.org/T1624

Thank you very much for working on this Boud.

> 
> = Scripting in Krita =
> 
> Since early 2016, I've been working on the scripting functionality for Krita. I \
> have tested a number of approaches. This report will describe the various ways we \
> have implemented scripting before in Krita, the approaches I have tried, the final \
> approach chosen. A number of notes on future work are added. 
> == Earlier Scripting Plugins in Krita ==
> 
> === Krita 1.4: kiskjsembed ===
> 
> The Krita 1.4 scripting plugin can be found in \
> calligra-history/krita/plugins/kiskjsembed. It was started in 2004. This plugin was \
> based on the kjs javascript interpreter \
> (https://api.kde.org/frameworks/kjs/html/index.html, \
> https://en.wikipedia.org/wiki/KJS_(software) ). The kiskjsembed plugin offered a \
> javascript api with access to both functions and objects. When the plugin was \
> abandoned, only a KisPaintDevice wrapper without functionality and a KisMainWindow \
> wrapper that could raise, lower and close a window were implemented.  
> The functions were implemented as a static objects:
> 
> * MainWindow
> - MainWindow.raise() : this function raises the main window
> - MainWindow.lower() : this function lowers the main window
> - MainWindow.close() : this function closes the main window
> 
> The objects could be created from Javascript:
> 
> Dynamic objects are objects that can be created.
> 
> * PaintDevice
> 
> 
> That was all.    
> 
> === Krita 1.5: Kross ===
> 
> Starting with Krita 1.5, the kiskjsembed plugin was replaced with the 'scripting' \
> plugin. The Krita 1.5 scripting plugin can be found in \
> calligra-history/krita/plugins/viewplugins/scripting. The plugin was retired \
> because of lack of maintenance with the Calligra 2.5 release.  
> Based on the kross framework (https://api.kde.org/frameworks/kross/html/index.html, \
> https://techbase.kde.org/Development/Tutorials/Kross/Introduction) which itself is \
> now on life support, the scripting module provided one generic api definition that \
> could be used from any language that was bound to the kross library. Kross also \
> provided a generic way to create e.g. dialogs. Kross makes heavily use of QObject \
> introspection, dynamically building classes from QObject-based wrapper classes. 
> The scripting plugin was loaded for every view/window. Scripts were available from \
> a "Scripts Manager" docker. It was not possible to write actual plugins in Krita \
> that would get loaded on startup. 
> The scripting api was implemented in the kritacore wrapper library. The wrapper \
> library implemented the following classes: 
> * Brush : a brush object
> * Color : represents a color
> * ConstIterator : an iterator for walking over the pixels of an image or layer
> * ConstPaintDevice : a layer within the image where you are able to perform paint \
>                 operations on
> * Filter : access to the filter plugins in Krita; not the base class for new \
>                 filters.
> * Histogram : gives access to the histogram of a paintdevice object
> * Image : an image object with some methods to manipulate the image, get some \
>                 information and create new layers.
> * Iterator: an iterator that allows changing pixels. Pixel values where accessed by \
>                 calling setPixel on the iterator.
> * Module: based on KoScriptingModule, provides static access to things like the \
>                 active layer. 
> * Monitor : was meant to check whether iterators need to be invalidated, seems \
>                 unused.
> * PaintDevice : represents a layer that can be painted on
> * Painter: abstracts KisPainter and makes it possible to perform high-level \
>                 painting on a paintdevice
> * PaintLayer : represents a layer that can be painted on (there seems to be some \
>                 confusion with PaintDevice, dox-wise)
> * Pattern : wraps a pattern object. When passed to Painter::setPattern, the Painter \
> class retrieves the actual pattern object from the wrapper. The script itself \
>                 cannot get at the pattern, or its image.
> * Progress: makes it possible to display a progressbar in Krita
> * Wavelet: a fast wavelet object
> 
> 
> Apart from these classes there were two helper classes. For the first there is no \
> usage example. 
> * KisScriptDecoration::KisCanvasDecoration : to implement canvas decorations like \
>                 assistants, grids, rulers, guides. 
> * KisScriptDockfactory: called to create new dockwidgets
> 
> A sample script:
> 
> ~~~
> require "Krita"
> 
> # fetch the image.
> image = Krita.image()
> 
> countStar = 60
> maxStarSize = 4
> 
> # we like to manipulate the active painting layer.
> layer = image.createPaintLayer("Sky", 255).paintDevice()
> 
> # get the height and the width the image has.
> width = image.width()
> height = image.height()
> 
> # start drawing
> layer.beginPainting("sky")
> 
> painter = layer.createPainter()
> 
> painter.setStrokeStyle(1)
> painter.setFillStyle(1)
> painter.setPaintOp("paintbrush")
> 
> painter.setPaintColor( Krita.createRGBColor(24,24,24) )
> painter.paintRect(0.0,0.0, width, height, 0.5)
> 
> painter.setPaintColor( Krita.createRGBColor(255,255,255) )
> for i in 1..countStar
> size = rand() * maxStarSize
> size = 1 if(size < 1)
> painter.setBrush(Krita.createCircleBrush(size,size, size / 2 +1, size / 2 +1) )
> painter.paintAt(rand() * width, rand() * height, 0.5)
> end
> 
> # painting is done now.
> layer.endPainting()
> ~~~
> 
> Which scripts were loaded by the scripting docker was determined by a single file, \
> scripts.rc that contained a short title, comment, name, type of interpreter and the \
> single file that contained the script. Scripts could not consist of more than one \
> file, as far as I can tell. 
> === PyKrita ===
> 
> In the summer of 2016, another scripting plugin was started. This was based on \
> Kate's "pate" plugin. This plugin makes it possible to load plugins written in a \
> scripting language on startup. These plugins integrate with the application's main \
> instance. Based on Pate, the plugin uses sip, PyQt, Python2 or Python3, a wrapper \
> library and hand-written SIP wrapper files. The plugin has never been released, but \
> has been used as a basis for further research. 
> == Other Applications ==
> 
> 
> === Photoshop ===
> 
> Photoshop offers one single API in three languages: javascript, vbscript, \
> applescript. There is, of course, no way to figure out how the scripting facility \
> is bound to the application's functionality. 
> http://www.adobe.com/devnet/photoshop/scripting.html
> 
> When designing various API's for Krita's scripting module, the Photoshop API, which \
> offers a hierarchy of objects acessible starting with the Application object, was \
> used as an example. Scripts are available in the File/Scripts menu and it is \
> possible to create scripts that are run on startup. 
> The object model starts with the Application object, which gives access to the \
> Document object, which gives access to the LayerSet object, which contains the \
> layers -- and so on. 
> 
> === GIMP ===
> 
> GIMP's scripting facility is based on the libgimp library. Gimp can be scripted in \
> scheme and python: https://docs.gimp.org/en/gimp-scripting.html . The scripting \
> system makes all functions available through the procedural database. This is \
> exposed through the Procedure Browser, where for every function the script writer \
> can see documentation, information about parameters, return values and additional \
> information. 
> The PDB is generated at build time and doesn't seem to be created dynamically as \
> plugins are loaded, but plugins can register and offer functionality for use in \
> scripts.  
> == Approaches to Scripting in Krita ==
> 
> 
> === QtScript ===
> 
> QtScript is Qt's scripting module: http://doc.qt.io/qt-5/qtscript-index.html . In \
> Qt5, this module has been deprecated and cannot be used for new development. This \
> is a pity because it is very complete, includes bindings to Qt and has even a \
> complete ide-like script editor and debugger. It is well suited to extending an \
> application with scripting, if javascript is acceptable. 
> === QtQuick/QML ===
> 
> QtScript has been replaced by QJSEngine and QML. This is another javascript based \
> environment, using a different javascript engine than QtScript. As an experiment, a \
> Krita plugin that uses QJSEngine and QQmlEngine was written: 
> http://valdyas.org/~boud/jsdocker.tgz
> 
> Since there are no bindings for Qt, the GUI for plugins written with this engine \
> should be written in QtQuick2, which is a declarative user-interface language. \
> Exposing Krita objects works through QObject introspection. 
> In the end, the facilities for writing user interface controls that fit with the \
> rest of Krita, debugging and exposing Krita functionality to the script writer were \
> insufficient. 
> 
> === Python ===
> 
> In the VFX industry, Python is the standard scripting language. Applications like \
> Mari are written almost entirely in Python (and glsl), Maya and other applications \
> offer Python as a scripting language. Since Qt is  also virtually the standard \
> toolkit in this field, the combination of Python and Qt would make Krita fit right \
> in with the other players in the field. 
> The most used binding to Qt is PySide, which was created by Nokia because the \
> long-standing PyQt bindings are only available under a dual GPL and commercial \
> license. PySide is LGPL. However, maintenance of PySide has halted since Nokia \
> divested itself of Qt. There is a wrapper, Qt.py that makes it possible to use the \
> exact same syntax, no matter the underlying wrapper: \
> https://github.com/mottosso/Qt.py/blob/master/Qt.py. 
> There are two approaches to exposing application functionality to Python, \
> intersecting with two technical options: 
> * wrap the core libraries (kritaimage, kritaui, etc) directly
> * create a wrapper library. the wrapper library can either be generated from a \
> definition file, or hand-written. Gimp's libgimp is an example of the former, as is \
> the Scribus' scripter-ng wrapper library. The kross-based krita scripting plugin \
> was hand-written. 
> And either way, one can:
> 
> * use qobject introspection
> * define the interface using a interface definition language. For PyQt that's sip. \
> The sip files can be either written manually or generated. Automatically generating \
> sip files means parsing C++ header files and then creating correct sip files/ 
> ==== QObject Introspection ====
> 
> ===== Mikro ======
> 
> Inspired by Kross, Mikro or Mini Kross \
> (https://wiki.scribus.net/canvas/ScripterNG), is a qobject-introspecter that \
> automatically builds Python classes from QObject wrapper classes. As found in \
> Scribus, the class didn't work. It turned out to be possible to make it mostly \
> work, with Python 3 and some hacks. 
> The fixed class in in krita/plugins/extensions/pykrita/plugins/krita/mikro.py.
> 
> The result however was very fragile and very slow. It also turned out to be very \
> hard to create signal/slot connections. The kind of Python that mikro generates \
> looks "weird". The wrapper library needs to provide Q_PROPERTY definitions for \
> getters and setters, Q_SLOTS for all callable methods and Q_SIGNALS for signals. \
> Signals do not work with this solution; we were not able to figure out why. 
> Mikro would be packaged with krita.
> 
> ===== PythonQt ======
> 
> PythonQt (http://pythonqt.sourceforge.net/) is a dynamic Python binding for the Qt \
> framework. It offers an easy way to embed the Python scripting language into your \
> C++ Qt applications. The focus of PythonQt is on embedding Python into an existing \
> C++ application, not on writing the whole application completely in Python. \
> PythonQt is a stable library that was developed to make the Image Processing and \
> Visualization platform MeVisLab scriptable from Python. 
> We investigated PythonQt:
> 
> https://phabricator.kde.org/T3588
> 
> Since PythonQt is another QObject-introspection based solution, we would either \
> need to create a QObject-based wrapper library, or make all relevant classes in \
> Krita itself QObject-based. PythonQt is not generally available in Linux \
> distributions, which makes it an objectionable dependency. 
> ==== Wrapper Library ====
> 
> ===== Generating SIP files at build time =====
> 
> Shaheed Haque and Stephen Kelly were working in 2016 on a way to generate the sip \
> files for a KDE frameworks library. This is done using the extra-cmake-modules \
> framework, which uses clang to parse all relevant header files and then generates \
> the sip files. In order to "help" the clang-based parser/generator, specification \
> files need to be created with details on, for instance, which methods need to be \
> skipped. The format for these files is fairly opaque. 
> The work is in:
> 
> https://phabricator.kde.org/source/krita/browse/rempt%252FT1625-python-scripting-ecm/
>  
> The big disadvantage of this approach is that everyone needs to have clang and \
> python-clang installed in order to generate the bindings; the bindings would be \
> regenerated every time Krita is built. We felt that this would make it too easy for \
> Krita developers to not install the required dependencies, leading to them working \
> on Krita and breaking the bindings without noticing. 
> 
> ===== Hand-written SIP files =====
> 
> SIP (https://riverbankcomputing.com/software/sip/intro) makes it possible to write \
> wrappers for existing C++ headers. These wrappers can be annotated and contain \
> custom code to make the C++ and Python memory and execution model work well \
> together. The technology is already quite old: the first book in the topic appeared \
> in 2001 (http://valdyas.org/python/book.html).  
> In the end, hand-writing and maintain SIP files for a single wrapper library turns \
> out to be easier than maintain the specification file that the extra-cmake-modules \
> based automatic sip wrapper generator needs. Automatic generation would be better \
> if we would wrapp all of Krita; but we decided not to do that. 
> ==== Wrap Everything ====
> 
> An initial investigation into wrapping all Krita's libraries proved that this would \
> be unworkable: 
> * After about twenty years of development, the API is very inconsistent 
> * There are many C++ constructs used that are very hard to wrap
> * Automatic wrapping at runtime would be the only feasible way to wrap the about \
> 2600 header files in Krita's 20 or so libraries. 
> But most importantly, Krita's libraries provide an internal API. Krita's plugins \
> are all in-tree. Wrapping this API and making it available to script authors fixes \
> the API once and for all time, depriving ourselves of the flexibility we need when \
> developing Krita. 
> === Conclusion ===
> 
> We decided to go with a specialized hand-written wrapper library, libkis. We \
> generated the initial C++ files from an API definition file, and then tweaked the \
> result to be usable both with Mikro and SIP. The we used Shaheed and Stephens SIP \
> generator to generate the initial set of SIP files, and tweaked those. 
> The final API has been closely modeled on Photoshop's Javascript API \
> (http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/photoshop/pdfs/photoshop-cc-scripting-guide-2015.pdf).
>  
> From that point on, the libkis wrapper library and the sip files have been \
> hand-developed in tandem. It turns out that some Krita classes needed direct \
> wrapping: one current example is KisCubicCurve. We do expect that some more classes \
> will be wrapped like this, with the resource classes for patterns, brushes and so \
> on being good candidates. Those have had a stable API for a long time, and spending \
> time to manually write a C++ wrapper class for each resource type could be seen as \
> a waste of time. 
> Currently, it is possible to run ad-hoc scripts in the scripter mini-IDE, mostly \
> written by Eliakin Almeida, add dock widgets to the user interface, add scripts to \
> the tools/scripts menu and, with hackery, to other menus. Plugin scripts, that are \
> loaded on startup, can be enabled and disabled in the Settings window. 
> In addition, there is a separate executable, kritarunner, that can run scripts from \
> the command-line. This might be integrated as just another commandline option in \
> the main krita executable (because you cannot have two executables in a Linux \
> appimage). 
> === API ===
> 
> The current scripting module provides the following objects:
> 
> * Action : encapsulates a QAction
> * Canvas : provides access to the canvas, for rotation, zoom etc.
> * Channel : represents a single channel in a Node
> * DockWidgetFactoryBase : baseclass for scripts that want to add a docker to every \
>                 window
> * DockWidget : base class for dockers implemented as a script.
> * Document : represents the combination of a KisDocument and KisImage (those are \
> split in the internal API for internal reasons, the image contains the image data, \
>                 the document the filename).
> * Filter : encapsulates a Krita filter. Cannot be used to implement new filters, as \
>                 in the kross-based scripting plugin
> * Generator : encapsulates a Krita generator. Same restrictions as for Filter hold
> * InfoObject : a key/value store
> * KisCubicCurve : represents the values for a cubic curve
> * Krita : the root class, accessible as Krita.instance() or under the Application \
>                 and Scripter aliases
> * Node : a node, that is a layer or mask
> * Notifier : emits signals when the application state changes.
> * Resource : a generic wrapper for resources like patterns, gradients or brush \
>                 tips.
> * Selection : represents a selection of pixels
> * ViewExtension : represents a plugin that can be added to the Krita window. Can be \
>                 re-implemented by a plugin script.
> * View : a view on an image in a window
> * Window : a single main window
> 
> Additional useful classes would be 'Histogram', 'Transformation' and others.
> 
> A sample script looks like this:
> 
> ~~~
> #
> # Tests the PyKrita API
> #
> 
> import sys
> from PyQt5.QtGui import *
> from PyQt5.QtWidgets import *
> from krita import *
> 
> def __main__(args):
> print("Arguments:", args)
> Application.setBatchmode(True)
> print("Batchmode: ", Application.batchmode())
> print("Profiles:", Application.profiles("GRAYA", "U16"));
> document = Application.openDocument(args[0])
> print("Opened", document.fileName(), "WxH", document.width(), document.height(), \
> "resolution", document.xRes(), document.yRes(), "in ppi", document.resolution()) \
> node = document.rootNode() print("Root", node.name(), "opacity", node.opacity())
> for child in node.childNodes():
> print("\tChild", child.name(), "opacity", node.opacity(), node.blendingMode())
> #r = child.save(child.name() + ".png", document.xRes(), document.yRes());
> #print("Saving result:", r)
> for channel in child.channels():
> print("Channel", channel.name(), "contents:", \
> len(channel.pixelData(node.bounds()))) 
> document.close()
> 
> document = Application.createDocument(100, 100, "test", "GRAYA", "U16", "")
> document.setBatchmode(True)
> document.saveAs("test.kra")
> ~~~
> 
As a python noob and a non programmer this surely sounds familiar and easy to learn \
and master.  I hope in coming years we would see nice library of plugins by the \
comunity that add and enhance features that krita currently has.

> === Maintenance ===
> 
> The first scripting plugin for Krita was never really finished; the second one \
> bit-rotted because nobody used it and because it was too easy to break it while \
> developing if one hadn't installed the optional kross runtimes. 
> This time, we propose to make sip, python and pyqt mandatory dependencies, on Linux \
> at least, where most developers are. The libkis wrapper library is already always \
> built. 
> == Future Work ==
> 
> === libkis is a generic wrapper ===
> 
> Libkis is a generic, QObject-based wrapper library. That means that if anyone steps \
> up and wants to provide another scripting language, libkis can be re-used easily. \
> Libkis is always built, even if the dependencies for the pykrita scripting module \
> are missing. 
> === libkis is an API for C++ plugins ===
> 
> Libkis is also poised to become a proper C++ API for external plugins. It still \
> isn't intended to use libkis to to implement tools, filters, generators and brush \
> engines as plugins. For those types of plugins, use of Krita's internal API is \
> still needed. But most of the plugins in krita's plugins/extensions group of \
> plugins could be rewritten against a single well-document libkis API without loss \
> of performance. 
> === Exposing plugin functionality dynamically ===
> 
> Right now, a script writer has access to all the KisAction objects and can toggle \
> them; that will show the dialogs associated with those actions and act on the image \
> and layer that is currently active in the user interface. It would be good to have \
> some generic solution where plugins can not only add actions to the gui, but also \
> methods to the scripting interface. 
> Something like gimp's pdb, without the generation step. It should be possible for \
> plugins, and maybe some internal functionality, to register a method or a runner \
> object in a central registry that describes the method, the parameters and the \
> return values, which can then dynamically be added to the Krita object and exposed \
> to the scripting environment. 
> === Exposing Krita widgets ===
> 
> Many script and script-b
> 
> 

Thanky ou once again
-- 
Raghavendra Kamath
Illustrator
raghukamath.com


[prev in list] [next in list] [prev in thread] [next in thread]