[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-kimageshop
Subject:    Re: Scripting Initial Report
From:       Raghavendra Kamath <raghu () raghukamath ! com>
Date:       2017-02-15 14:58:55
Message-ID: 2338753.S5ZGaFR3nd () raghu
[Download RAW message or body]

On Wednesday, February 15, 2017 7:04:33 PM IST Boudewijn Rempt wrote:
> I'm pretty sure that we're on the right track now, and getting close
> to a first usable version, so I spent today finishing up my research 
> report on scripting. Permanent copy is here: https://phabricator.kde.org/T1624

Thank you very much for working on this Boud.

> 
> = Scripting in Krita =
> 
> Since early 2016, I've been working on the scripting functionality for Krita. I have tested a \
> number of approaches. This report will describe the various ways we have implemented \
> scripting before in Krita, the approaches I have tried, the final approach chosen. A number \
> of notes on future work are added. 
> == Earlier Scripting Plugins in Krita ==
> 
> === Krita 1.4: kiskjsembed ===
> 
> The Krita 1.4 scripting plugin can be found in calligra-history/krita/plugins/kiskjsembed. It \
> was started in 2004. This plugin was based on the kjs javascript interpreter \
> (https://api.kde.org/frameworks/kjs/html/index.html, \
> https://en.wikipedia.org/wiki/KJS_(software) ). The kiskjsembed plugin offered a javascript \
> api with access to both functions and objects. When the plugin was abandoned, only a \
> KisPaintDevice wrapper without functionality and a KisMainWindow wrapper that could raise, \
> lower and close a window were implemented.  
> The functions were implemented as a static objects:
> 
> * MainWindow
> - MainWindow.raise() : this function raises the main window
> - MainWindow.lower() : this function lowers the main window
> - MainWindow.close() : this function closes the main window
> 
> The objects could be created from Javascript:
> 
> Dynamic objects are objects that can be created.
> 
> * PaintDevice
> 
> 
> That was all.    
> 
> === Krita 1.5: Kross ===
> 
> Starting with Krita 1.5, the kiskjsembed plugin was replaced with the 'scripting' plugin. The \
> Krita 1.5 scripting plugin can be found in \
> calligra-history/krita/plugins/viewplugins/scripting. The plugin was retired because of lack \
> of maintenance with the Calligra 2.5 release.  
> Based on the kross framework (https://api.kde.org/frameworks/kross/html/index.html, \
> https://techbase.kde.org/Development/Tutorials/Kross/Introduction) which itself is now on \
> life support, the scripting module provided one generic api definition that could be used \
> from any language that was bound to the kross library. Kross also provided a generic way to \
> create e.g. dialogs. Kross makes heavily use of QObject introspection, dynamically building \
> classes from QObject-based wrapper classes. 
> The scripting plugin was loaded for every view/window. Scripts were available from a "Scripts \
> Manager" docker. It was not possible to write actual plugins in Krita that would get loaded \
> on startup. 
> The scripting api was implemented in the kritacore wrapper library. The wrapper library \
> implemented the following classes: 
> * Brush : a brush object
> * Color : represents a color
> * ConstIterator : an iterator for walking over the pixels of an image or layer
> * ConstPaintDevice : a layer within the image where you are able to perform paint operations \
>                 on
> * Filter : access to the filter plugins in Krita; not the base class for new filters.
> * Histogram : gives access to the histogram of a paintdevice object
> * Image : an image object with some methods to manipulate the image, get some information and \
>                 create new layers.
> * Iterator: an iterator that allows changing pixels. Pixel values where accessed by calling \
>                 setPixel on the iterator.
> * Module: based on KoScriptingModule, provides static access to things like the active layer. \
>                 
> * Monitor : was meant to check whether iterators need to be invalidated, seems unused.
> * PaintDevice : represents a layer that can be painted on
> * Painter: abstracts KisPainter and makes it possible to perform high-level painting on a \
>                 paintdevice
> * PaintLayer : represents a layer that can be painted on (there seems to be some confusion \
>                 with PaintDevice, dox-wise)
> * Pattern : wraps a pattern object. When passed to Painter::setPattern, the Painter class \
> retrieves the actual pattern object from the wrapper. The script itself cannot get at the \
>                 pattern, or its image.
> * Progress: makes it possible to display a progressbar in Krita
> * Wavelet: a fast wavelet object
> 
> 
> Apart from these classes there were two helper classes. For the first there is no usage \
> example. 
> * KisScriptDecoration::KisCanvasDecoration : to implement canvas decorations like assistants, \
>                 grids, rulers, guides. 
> * KisScriptDockfactory: called to create new dockwidgets
> 
> A sample script:
> 
> ~~~
> require "Krita"
> 
> # fetch the image.
> image = Krita.image()
> 
> countStar = 60
> maxStarSize = 4
> 
> # we like to manipulate the active painting layer.
> layer = image.createPaintLayer("Sky", 255).paintDevice()
> 
> # get the height and the width the image has.
> width = image.width()
> height = image.height()
> 
> # start drawing
> layer.beginPainting("sky")
> 
> painter = layer.createPainter()
> 
> painter.setStrokeStyle(1)
> painter.setFillStyle(1)
> painter.setPaintOp("paintbrush")
> 
> painter.setPaintColor( Krita.createRGBColor(24,24,24) )
> painter.paintRect(0.0,0.0, width, height, 0.5)
> 
> painter.setPaintColor( Krita.createRGBColor(255,255,255) )
> for i in 1..countStar
> size = rand() * maxStarSize
> size = 1 if(size < 1)
> painter.setBrush(Krita.createCircleBrush(size,size, size / 2 +1, size / 2 +1) )
> painter.paintAt(rand() * width, rand() * height, 0.5)
> end
> 
> # painting is done now.
> layer.endPainting()
> ~~~
> 
> Which scripts were loaded by the scripting docker was determined by a single file, scripts.rc \
> that contained a short title, comment, name, type of interpreter and the single file that \
> contained the script. Scripts could not consist of more than one file, as far as I can tell. 
> === PyKrita ===
> 
> In the summer of 2016, another scripting plugin was started. This was based on Kate's "pate" \
> plugin. This plugin makes it possible to load plugins written in a scripting language on \
> startup. These plugins integrate with the application's main instance. Based on Pate, the \
> plugin uses sip, PyQt, Python2 or Python3, a wrapper library and hand-written SIP wrapper \
> files. The plugin has never been released, but has been used as a basis for further research. \
>  == Other Applications ==
> 
> 
> === Photoshop ===
> 
> Photoshop offers one single API in three languages: javascript, vbscript, applescript. There \
> is, of course, no way to figure out how the scripting facility is bound to the application's \
> functionality. 
> http://www.adobe.com/devnet/photoshop/scripting.html
> 
> When designing various API's for Krita's scripting module, the Photoshop API, which offers a \
> hierarchy of objects acessible starting with the Application object, was used as an example. \
> Scripts are available in the File/Scripts menu and it is possible to create scripts that are \
> run on startup. 
> The object model starts with the Application object, which gives access to the Document \
> object, which gives access to the LayerSet object, which contains the layers -- and so on. 
> 
> === GIMP ===
> 
> GIMP's scripting facility is based on the libgimp library. Gimp can be scripted in scheme and \
> python: https://docs.gimp.org/en/gimp-scripting.html . The scripting system makes all \
> functions available through the procedural database. This is exposed through the Procedure \
> Browser, where for every function the script writer can see documentation, information about \
> parameters, return values and additional information. 
> The PDB is generated at build time and doesn't seem to be created dynamically as plugins are \
> loaded, but plugins can register and offer functionality for use in scripts.  
> == Approaches to Scripting in Krita ==
> 
> 
> === QtScript ===
> 
> QtScript is Qt's scripting module: http://doc.qt.io/qt-5/qtscript-index.html . In Qt5, this \
> module has been deprecated and cannot be used for new development. This is a pity because it \
> is very complete, includes bindings to Qt and has even a complete ide-like script editor and \
> debugger. It is well suited to extending an application with scripting, if javascript is \
> acceptable. 
> === QtQuick/QML ===
> 
> QtScript has been replaced by QJSEngine and QML. This is another javascript based \
> environment, using a different javascript engine than QtScript. As an experiment, a Krita \
> plugin that uses QJSEngine and QQmlEngine was written: 
> http://valdyas.org/~boud/jsdocker.tgz
> 
> Since there are no bindings for Qt, the GUI for plugins written with this engine should be \
> written in QtQuick2, which is a declarative user-interface language. Exposing Krita objects \
> works through QObject introspection. 
> In the end, the facilities for writing user interface controls that fit with the rest of \
> Krita, debugging and exposing Krita functionality to the script writer were insufficient. 
> 
> === Python ===
> 
> In the VFX industry, Python is the standard scripting language. Applications like Mari are \
> written almost entirely in Python (and glsl), Maya and other applications offer Python as a \
> scripting language. Since Qt is  also virtually the standard toolkit in this field, the \
> combination of Python and Qt would make Krita fit right in with the other players in the \
> field. 
> The most used binding to Qt is PySide, which was created by Nokia because the long-standing \
> PyQt bindings are only available under a dual GPL and commercial license. PySide is LGPL. \
> However, maintenance of PySide has halted since Nokia divested itself of Qt. There is a \
> wrapper, Qt.py that makes it possible to use the exact same syntax, no matter the underlying \
> wrapper: https://github.com/mottosso/Qt.py/blob/master/Qt.py. 
> There are two approaches to exposing application functionality to Python, intersecting with \
> two technical options: 
> * wrap the core libraries (kritaimage, kritaui, etc) directly
> * create a wrapper library. the wrapper library can either be generated from a definition \
> file, or hand-written. Gimp's libgimp is an example of the former, as is the Scribus' \
> scripter-ng wrapper library. The kross-based krita scripting plugin was hand-written. 
> And either way, one can:
> 
> * use qobject introspection
> * define the interface using a interface definition language. For PyQt that's sip. The sip \
> files can be either written manually or generated. Automatically generating sip files means \
> parsing C++ header files and then creating correct sip files/ 
> ==== QObject Introspection ====
> 
> ===== Mikro ======
> 
> Inspired by Kross, Mikro or Mini Kross (https://wiki.scribus.net/canvas/ScripterNG), is a \
> qobject-introspecter that automatically builds Python classes from QObject wrapper classes. \
> As found in Scribus, the class didn't work. It turned out to be possible to make it mostly \
> work, with Python 3 and some hacks. 
> The fixed class in in krita/plugins/extensions/pykrita/plugins/krita/mikro.py.
> 
> The result however was very fragile and very slow. It also turned out to be very hard to \
> create signal/slot connections. The kind of Python that mikro generates looks "weird". The \
> wrapper library needs to provide Q_PROPERTY definitions for getters and setters, Q_SLOTS for \
> all callable methods and Q_SIGNALS for signals. Signals do not work with this solution; we \
> were not able to figure out why. 
> Mikro would be packaged with krita.
> 
> ===== PythonQt ======
> 
> PythonQt (http://pythonqt.sourceforge.net/) is a dynamic Python binding for the Qt framework. \
> It offers an easy way to embed the Python scripting language into your C++ Qt applications. \
> The focus of PythonQt is on embedding Python into an existing C++ application, not on writing \
> the whole application completely in Python. PythonQt is a stable library that was developed \
> to make the Image Processing and Visualization platform MeVisLab scriptable from Python. 
> We investigated PythonQt:
> 
> https://phabricator.kde.org/T3588
> 
> Since PythonQt is another QObject-introspection based solution, we would either need to \
> create a QObject-based wrapper library, or make all relevant classes in Krita itself \
> QObject-based. PythonQt is not generally available in Linux distributions, which makes it an \
> objectionable dependency. 
> ==== Wrapper Library ====
> 
> ===== Generating SIP files at build time =====
> 
> Shaheed Haque and Stephen Kelly were working in 2016 on a way to generate the sip files for a \
> KDE frameworks library. This is done using the extra-cmake-modules framework, which uses \
> clang to parse all relevant header files and then generates the sip files. In order to "help" \
> the clang-based parser/generator, specification files need to be created with details on, for \
> instance, which methods need to be skipped. The format for these files is fairly opaque. 
> The work is in:
> 
> https://phabricator.kde.org/source/krita/browse/rempt%252FT1625-python-scripting-ecm/
> 
> The big disadvantage of this approach is that everyone needs to have clang and python-clang \
> installed in order to generate the bindings; the bindings would be regenerated every time \
> Krita is built. We felt that this would make it too easy for Krita developers to not install \
> the required dependencies, leading to them working on Krita and breaking the bindings without \
> noticing. 
> 
> ===== Hand-written SIP files =====
> 
> SIP (https://riverbankcomputing.com/software/sip/intro) makes it possible to write wrappers \
> for existing C++ headers. These wrappers can be annotated and contain custom code to make the \
> C++ and Python memory and execution model work well together. The technology is already quite \
> old: the first book in the topic appeared in 2001 (http://valdyas.org/python/book.html).  
> In the end, hand-writing and maintain SIP files for a single wrapper library turns out to be \
> easier than maintain the specification file that the extra-cmake-modules based automatic sip \
> wrapper generator needs. Automatic generation would be better if we would wrapp all of Krita; \
> but we decided not to do that. 
> ==== Wrap Everything ====
> 
> An initial investigation into wrapping all Krita's libraries proved that this would be \
> unworkable: 
> * After about twenty years of development, the API is very inconsistent 
> * There are many C++ constructs used that are very hard to wrap
> * Automatic wrapping at runtime would be the only feasible way to wrap the about 2600 header \
> files in Krita's 20 or so libraries. 
> But most importantly, Krita's libraries provide an internal API. Krita's plugins are all \
> in-tree. Wrapping this API and making it available to script authors fixes the API once and \
> for all time, depriving ourselves of the flexibility we need when developing Krita. 
> === Conclusion ===
> 
> We decided to go with a specialized hand-written wrapper library, libkis. We generated the \
> initial C++ files from an API definition file, and then tweaked the result to be usable both \
> with Mikro and SIP. The we used Shaheed and Stephens SIP generator to generate the initial \
> set of SIP files, and tweaked those. 
> The final API has been closely modeled on Photoshop's Javascript API \
> (http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/photoshop/pdfs/photoshop-cc-scripting-guide-2015.pdf).
>  
> From that point on, the libkis wrapper library and the sip files have been hand-developed in \
> tandem. It turns out that some Krita classes needed direct wrapping: one current example is \
> KisCubicCurve. We do expect that some more classes will be wrapped like this, with the \
> resource classes for patterns, brushes and so on being good candidates. Those have had a \
> stable API for a long time, and spending time to manually write a C++ wrapper class for each \
> resource type could be seen as a waste of time. 
> Currently, it is possible to run ad-hoc scripts in the scripter mini-IDE, mostly written by \
> Eliakin Almeida, add dock widgets to the user interface, add scripts to the tools/scripts \
> menu and, with hackery, to other menus. Plugin scripts, that are loaded on startup, can be \
> enabled and disabled in the Settings window. 
> In addition, there is a separate executable, kritarunner, that can run scripts from the \
> command-line. This might be integrated as just another commandline option in the main krita \
> executable (because you cannot have two executables in a Linux appimage). 
> === API ===
> 
> The current scripting module provides the following objects:
> 
> * Action : encapsulates a QAction
> * Canvas : provides access to the canvas, for rotation, zoom etc.
> * Channel : represents a single channel in a Node
> * DockWidgetFactoryBase : baseclass for scripts that want to add a docker to every window
> * DockWidget : base class for dockers implemented as a script.
> * Document : represents the combination of a KisDocument and KisImage (those are split in the \
> internal API for internal reasons, the image contains the image data, the document the \
>                 filename).
> * Filter : encapsulates a Krita filter. Cannot be used to implement new filters, as in the \
>                 kross-based scripting plugin
> * Generator : encapsulates a Krita generator. Same restrictions as for Filter hold
> * InfoObject : a key/value store
> * KisCubicCurve : represents the values for a cubic curve
> * Krita : the root class, accessible as Krita.instance() or under the Application and \
>                 Scripter aliases
> * Node : a node, that is a layer or mask
> * Notifier : emits signals when the application state changes.
> * Resource : a generic wrapper for resources like patterns, gradients or brush tips.
> * Selection : represents a selection of pixels
> * ViewExtension : represents a plugin that can be added to the Krita window. Can be \
>                 re-implemented by a plugin script.
> * View : a view on an image in a window
> * Window : a single main window
> 
> Additional useful classes would be 'Histogram', 'Transformation' and others.
> 
> A sample script looks like this:
> 
> ~~~
> #
> # Tests the PyKrita API
> #
> 
> import sys
> from PyQt5.QtGui import *
> from PyQt5.QtWidgets import *
> from krita import *
> 
> def __main__(args):
> print("Arguments:", args)
> Application.setBatchmode(True)
> print("Batchmode: ", Application.batchmode())
> print("Profiles:", Application.profiles("GRAYA", "U16"));
> document = Application.openDocument(args[0])
> print("Opened", document.fileName(), "WxH", document.width(), document.height(), \
> "resolution", document.xRes(), document.yRes(), "in ppi", document.resolution()) node = \
> document.rootNode() print("Root", node.name(), "opacity", node.opacity())
> for child in node.childNodes():
> print("\tChild", child.name(), "opacity", node.opacity(), node.blendingMode())
> #r = child.save(child.name() + ".png", document.xRes(), document.yRes());
> #print("Saving result:", r)
> for channel in child.channels():
> print("Channel", channel.name(), "contents:", len(channel.pixelData(node.bounds())))
> 
> document.close()
> 
> document = Application.createDocument(100, 100, "test", "GRAYA", "U16", "")
> document.setBatchmode(True)
> document.saveAs("test.kra")
> ~~~
> 
As a python noob and a non programmer this surely sounds familiar and easy to learn and master. \
 I hope in coming years we would see nice library of plugins by the comunity that add and \
enhance features that krita currently has.

> === Maintenance ===
> 
> The first scripting plugin for Krita was never really finished; the second one bit-rotted \
> because nobody used it and because it was too easy to break it while developing if one hadn't \
> installed the optional kross runtimes. 
> This time, we propose to make sip, python and pyqt mandatory dependencies, on Linux at least, \
> where most developers are. The libkis wrapper library is already always built. 
> == Future Work ==
> 
> === libkis is a generic wrapper ===
> 
> Libkis is a generic, QObject-based wrapper library. That means that if anyone steps up and \
> wants to provide another scripting language, libkis can be re-used easily. Libkis is always \
> built, even if the dependencies for the pykrita scripting module are missing. 
> === libkis is an API for C++ plugins ===
> 
> Libkis is also poised to become a proper C++ API for external plugins. It still isn't \
> intended to use libkis to to implement tools, filters, generators and brush engines as \
> plugins. For those types of plugins, use of Krita's internal API is still needed. But most of \
> the plugins in krita's plugins/extensions group of plugins could be rewritten against a \
> single well-document libkis API without loss of performance. 
> === Exposing plugin functionality dynamically ===
> 
> Right now, a script writer has access to all the KisAction objects and can toggle them; that \
> will show the dialogs associated with those actions and act on the image and layer that is \
> currently active in the user interface. It would be good to have some generic solution where \
> plugins can not only add actions to the gui, but also methods to the scripting interface. 
> Something like gimp's pdb, without the generation step. It should be possible for plugins, \
> and maybe some internal functionality, to register a method or a runner object in a central \
> registry that describes the method, the parameters and the return values, which can then \
> dynamically be added to the Krita object and exposed to the scripting environment. 
> === Exposing Krita widgets ===
> 
> Many script and script-b
> 
> 

Thanky ou once again
-- 
Raghavendra Kamath
Illustrator
raghukamath.com


[prev in list] [next in list] [prev in thread] [next in thread]