[prev in list] [next in list] [prev in thread] [next in thread]
List: qubes-devel
Subject: [qubes-devel] [GSoC] Qubes-MIME-Handlers Weekly Progress Report #8
From: Andrew Morgan <andrew () amorgan ! xyz>
Date: 2017-08-08 10:38:00
Message-ID: omc4a6$4qj$1 () blaine ! gmane ! org
[Download RAW message or body]
[Attachment #2 (multipart/mixed)]
[Attachment #4 (multipart/mixed)]
Another report with some long-awaited news, enjoy!
Blog post with screenshots:
https://blog.amorgan.xyz/gsoc-weekly-progress-report-8.html
Otherwise text-only is below (with screenshots attached):
---
Hey there, welcome to another report! Since the last one, you may
remember that I had finished setting up the build environment for
Nautilus inside of GNOME Builder and was just getting started on
properly doing the patch for supporting file trust attributes. Well, I'm
happy to report that the patch is now finished! Screenshots and details
below.
## Working Demo
As screen recording in dom0 still isn't really doable yet, you'll have
to settle for some screenshots of it in action :)
*Double-clicking a trusted file opens normally...*
*Everyone knows gedit is the ultimate text editor.*
*Whereas an untrusted file summons a DispVM!*
*Virus free, yes-sir-ee!*
This short demo makes use of a majority of all the different components
that have been worked on throughout the project. The now-patched
Nautilus makes a call to the patched NautilusPython which asks our
`qvm_trust` python extension to open a file, which it gives a response
based on what our cli tool, `qvm-file-trust`, says about it, and if it
is determined an untrusted file, `open-file-trust-based` will open it
securely in a DisposableVM. The contents of the file were never parsed
on the local machine!
The low-level part of this process was already working, but it took the
patch to Nautilus allowing us to set off the chain from the GUI to
really bring it all together. Let's talk about how that was done and how
it works.
## Nautilus and NautilusPython
By default, Nautilus does not support extensions written in Python, but
rather only those written in C. To get around this, a _Nautilus C
extension called NautilusPython_ was created. This extension has the
ability to provide a bridging interface from Nautilus' extension
infastructure to Python, handling any calls Python programs make to
Nautilus and calling them from C. Arguments and return variables are all
passed along seamlessly.
This is a really nice feature, as writing extensions in Python are about
90% less work than writing them in C, due to the large amount of
boiler-plate associated with setting up and tearing down an extension.
If you recall, the point of patching Nautilus was to allow extensions to
be able to be notified of when a file is being opened, and block that
request if necessary. I didn't want to subject myself or anyone else to
only being able to write an extension that made use of this new
functionality in C, thus patching NautilusPython as well as Nautilus
itself to add our new `file_open` function was necessary. So how was
each one done then?
Patching NautilusPython to add a new method was relatively simple, you
only need look at the existing methods that an extension can subclass,
find one that is functionally similar your own (I used
`update_file_info` as my guide), then insert your own method
(`file_open` in this case) at all the relevant points as well as create
the appropriate function bodies and set argument/return variables
appropriately.
So overall NautilusPython wasn't too bad really. The real time sink was
Nautilus itself.
## Patching Nautilus
First of all, I almost immediately ditched GNOME Builder after my last
blog post. It's not that it was a bad IDE, far from it actually, I quite
liked the interface and the various tools it contained. The ability to
download a project from git, package and run it inside a Flatpak, and
then send patches upstream all from within the editor was quite
appealing! The first major showstopper however, was that the version of
Nautilus that Qubes makes use of (v3.22.3), doesn't support Flatpak.
While building and running from GNOME Builder without Flatpak seemed
possible, no matter what I tried I could not get extensions to function
properly with the GNOME Builder version.
After a lot of fiddling about, I settled on what I probably should have
been using since the very beginning:
[qubes-builder](https://github.com/qubesos/qubes-builder)!
Not only does it build the correct version (and supports extensions),
but I knew that whatever I did with this build system would be exactly
what users would be using when they built Qubes' templates or got them
from the build servers. So, unfortunately no Flatpak'd builds, but I do
have pre-packaged RPMs with debug-symbols! That's probably even better,
right?
I'll post them in a separate blog post with instructions on how to set
everything up soon.
qubes-builder worked very well and was quite easy to drill into and mess
about with, as it is mostly just a collection of various scripts. I
eventually built a script that ran in dom0 that would build Nautilus in
my development VM, transfer it to my testing VM, delete the old version
and install the new version.
The only downside to this process was that building Nautilus took quite
a while each time (2-3m) which made quick changes quite an agonizing
process. I believe this was due to both having to rebuild _every_ file
each time (as they were packaged in a .tar.xz before being built,
meaning all files were considered to be modified) as well as a call to
`dnf` to fetch any updates was made, _every time_. I found out the
latter when working without internet and realizing my could would not
build! Each of these things probably could have been worked out or
changed if I had felt the need to, but I just stuck with the loading
times and usually had internet, so it was not too terrible.
Was _was_ terrible and time-consuming were a couple of very strange
issues I had during the creation of the patch.
The first was the dreaded _provider issue_. You see, every time Nautilus
calls a method that is meant to be subclassed by a Nautilus extension,
you must _provide a provider_. This can be an `INFO` provider, `MENU`
provider, and so on. Each method only accepts one of each type,
typically corresponding to the category of method you were calling. For
the `file_open` method I decided to stick it under the `INFO` provider
category, as that had to do with information about files, which I felt
'when they were opened' best fell into.
Well, these providers don't just pop out of nowhere. You have to create
them! You can do that by calling the intuitively-named
`nautilus_module_get_extensions_for_type` method, providing it the
`NAUTILUS_TYPE_INFO_PROVIDER` enum. It is then _supposed_ to return a
provider with a type that matches the one you gave it, but no matter
what I did it would always just return a `NULL` pointer! I spent _days_
on this problem, trying loads of different things. Every other call of
`nautilus_module_get_extensions_for_type` had no special setup or
initialization, leading me confused why my own calling of it brought
such different results.
After picking through many lines of code, talking with GNOME people over
IRC who were currently GUADEC'ing, and trying to understand the whole
GNOME development ecosystem, I eventually traced it all back to
NautilusPython not finding an extension that made use of a subclassed
method from the `INFO` method type. But wait, we _do_ have an extension
that does that with `qvm_trust`, so why doesn't it work??
Well I hit my head after this one folks. The version of NautilusPython
that Nautilus was loading in my test VM was the original, from-repo
version, not our modified version. **D'oh!**
"Well that's just a simple fix, I just have to build it and import it
into Nautilus in the testing VM!", I assumed, before realizing that
while my modified NautilusPython built, Nautilus _would not accept it_.
Nautilus would just _hang_ with little to no output.
After a few emails, Marek pointed me in the correct direction with the
`nautilus-3.0` branch of NautilusPython, which, while 6 years since the
last commit, worked with the latest version. Alright no problem. Rebase
my commits on that branch, rebuild, transfer the newly-built
shared-library over and bam, Nautilus runs again! And we have a provider
that's not NULL, hooray!
And then, ladies and gentlemen, problem #2 reared _its_ ugly head.
Here's the code to get a list (`GList*`) of providers to use with a
Nautilus extension:
providers = nautilus_module_get_extensions_for_type
(NAUTILUS_TYPE_INFO_PROVIDER);
While we were getting a _non-null_ provider value, the value we received
was _different_ from the one returned by the method we called. "Wat?"
you say, and I too said the same. For some reason we were returning a
64-bit `GList` pointer from `nautilus_module_get_extensions_for_type`,
but storing a 32-bit `GList` pointer in `providers`.
I tried all sorts of weird tricks to get this fixed. Nobody I talked to
seemed to have a solution, and after wrestling with it for a few days I
begrudingly settled on this very elegant workaround:
// TODO: Providers ends up as 0x55f65ac0
// when method is returning 0x555555f65ac0, very odd...
providers = (GList*) (nautilus_module_get_extensions_for_type
(NAUTILUS_TYPE_INFO_PROVIDER) + 0x555500000000);
Yes I know this is terrible. It did work though, and I needed to stop
hitting this strange bug and get on with patching, so that's where it
currently sits. I'm still actively looking for a proper solution. If
anyone knows why a 64bit pointer might get truncated to a 32bit pointer
after a function call in C please do let me know! You can find the full
code
[here](https://github.com/anoadragon453/nautilus/blob/master/src/nautilus-mime-actions.c#L2496)
and the `type` method
[here](https://github.com/anoadragon453/nautilus/blob/master/src/nautilus-module.c#L269).
After this was out of the way, and a handful of more methods were
implemented, we finally had a working solution between the `qvm_trust`
extension and our patched Nautilus! It was beautiful, really.
## Conclusion and going forward
My [email](https://amorgan.xyz/upstream-request.html) to the [Nautilus
mailing list](https://mail.gnome.org/archives/nautilus-list/) is still
awaiting moderation. GUADEC has now concluded, and with no update on the
situation it seems I need to do some more active poking of people to get
moderation approval. We'll see where that lands, the code still needs to
be cleaned up a bit before sending it upstream anyways, so I'm not too
worried.
At this point in the project, we essentially have the following tasks left:
* Patch KDE Dolphin to provide the same functionality as Nautilus now does
* Finish up some small tasks with the daemon like logging and some more
interfacing with the cli tool
* Code cleanup, linting, tests
* Integrate into main QubesOS repos
I'm going to prioritize the latter 3 bullet points at this point to
ensure we get a really solid core implementation down of the project. I
don't want to spend another couple weeks patching Dolphin only to run
out of time for a general cleanup and other polish. I will try to pour
through the Dolphin code and note down everything that one would need to
do with Dolphin to make it work with file trust attributes, so that in
the future either myself or someone else can give it a shot.
It may end up that Dolphin already has most of the functionality and the
patch will be very small, we'll see!
Anyways, for now I'm going to start on finishing the remaining small
tasks, and will post any questions regarding packaging and/or cleanup to
the qubes-devel mailing list.
As always, you can find the code
[here](https://github.com/anoadragon453/qubes-mime-types).
And as of now, you can also find the patched code for
[Nautilus](https://github.com/anoadragon453/nautilus) and
[NautilusPython](https://github.com/anoadragon453/nautilus-python/tree/nautilus-3.0)
on Github as well. Make sure that when you're inspecting the
NautilusPython code, you're looking at the `nautilus-3.0` branch. That's
where all the important commits are!
That's all for now, thanks for reading!
--
You received this message because you are subscribed to the Google Groups \
"qubes-devel" group. To unsubscribe from this group and stop receiving emails from \
it, send an email to qubes-devel+unsubscribe@googlegroups.com. To post to this group, \
send email to qubes-devel@googlegroups.com. To view this discussion on the web visit \
https://groups.google.com/d/msgid/qubes-devel/omc4a6%244qj%241%40blaine.gmane.org. \
For more options, visit https://groups.google.com/d/optout.
["open-trusted.png" (image/png)]
["opened-gedit.png" (image/png)]
["open-untrusted.png" (image/png)]
["opened-dispvm.png" (image/png)]
["signature.asc" (application/pgp-signature)]
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic