[prev in list] [next in list] [prev in thread] [next in thread] 

List:       qubes-devel
Subject:    [qubes-devel] Deterministic builds for Qubes OS -- the shortcut?
From:       Joanna Rutkowska <joanna () invisiblethingslab ! com>
Date:       2015-12-22 15:26:18
Message-ID: 20151222152618.GA15086 () work-mutt
[Download RAW message or body]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

I've recently had a chance to discuss with Marek and Wojtek, who recently took
part in the Reproducible Builds Summit in Athens [1], the current state of the
these efforts. TL;DR, the ETA for having Debian build deterministically is ~1.5
year, which I take to mean it won't happen before 2018. Then, at least 0.5 year
to get Qubes (i.e. the ISO) to build deterministically, which means we won't
have it before the end of 2018, optimistically counting ;)

So, I thought, why not take the following shortcut:

Rather than trying to make *everything* build deterministically from scratch
(which is what current efforts try to do, AFAIU) why not prepare an environment
(as in: a VM image running under Qubes) that would be optimized for ensuring
that whatever we build using qubes-builder (yes, let's focus on Qubes, for now,
ok?) it always produces the same binaries?

Admittedly the image itself would not build deterministically, true, but we
would publish it (i.e. the binary) for everybody to download and inspect.
Assuming people find it acceptable to trust this image (after spending many
nights analysing it with a disassembler, of course ;) then all future builds,
including future ISOs, images, and packages, should build deterministically. I
think this would be a big win for everybody, especially that I think we might
get this working within single weeks...

So, how such an environment should look like? I think (perhaps naively) that the
following adjustments should be just enough:

1. Patch any date/time-returning syscalls (or lib calls?) in the VM so that each
call returns time incremented by, say, +1 sec (we really don't want to go with
the resolution down to anything too small, so that the build process do not miss
the time differences due to rounding errors). AFAIU there is already a tool for
that, although, I've been told, it returns always 0, which clearly is not gonna
work with most build systems. Also, there is apparently a tool that returns
incremental time readings but on nanosec resolution, which, again, seems too
little for most build processes to notice.

2. We need to ensure the whole build process is single-threaded (think: -j1).

3. Potentially patch /dev/(u)random to return some predictable values, similar
to how we need to patch the time syscalls (think: return sha1(seed++)?)

I think we need to ensure these patches (and time/seed variables initialization)
are only for the processes started by the qubes-builder, so that any potential
interaction with the VM by the user via e.g. Terminal do not interfere here
(alternatively do not allow starting of any processes other than qubes-builder,
and use only qvm-run/qrexec for this).

And I think this should be pretty much that :) Again, we need to start with this
image being (nondeterministically) build by one of us, but then everything
should build deterministically. Perhaps including the next iteration of the very
image itself...

Marek and Wojtek tried to convince this would not work, because... if things
were that simple, others would already done that by now. That's a legit
argument, I agree, but because none of them were able to provide an actual
_technical_ argument why this would not work, I think it's worth for somebody to
just try that.  Unsurprisingly I won't be able to do that myself anytime in the
coming weeks, but hope others might be interested...?

Thanks,
joanna.

[1] https://reproducible-builds.org/events/athens2015/
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCAAGBQJWeWuaAAoJEDOT2L8N3GcY4gYQAIkb59xUdiechRe6b0j3CVov
xdD7st61wzL0FTvJXMLxbgmvpDGBxO9BG5mb5HkuB1dJEyllcKeV4VUYBw8u+2te
kIn5t9o8+gYK/Z2gokMSH442p01J56mSPLtn2X1KM7/x6OpkNKlmokFGeLK42V8i
Yhx2VvwRLSALi0o3w67Rzxh17ApIZjzxHAYBXzS8HkMj1giGdtnEbluD2+awh/zc
ySVefV8BYAvEW6LSu3kUriIOyvTwadg0MKtwfig9ufzrfMv3+fyzJoQYiHTN8d/C
yBuz2keCuB3HhasevtICErpIcQPhaGpRzQlJd4Jm7vxkCJG5uQaK0mPNafc7WqYY
ZXin/GXsKCBuNYhyx39RkJ5GVocBlXzKvDtMm4JOTiCxzPEUHq/LkXAqKG/U+H0Y
pRWHA3cvskm6jn3G2x+y3I3aRYkg4ShTZ7iw0nQ1WndcfiMgCXrdrbEJ2vd8MXpS
IRAzF9PBwGd17yrq1bttkkQdanW+YjDVy+l8lsoYJbWX0cHZwJuLrDac7RgvDspb
HbbSuj49FHAhYdfzN7oPfs+aGoykuZ3u7rKeXBCnPjxQjRvePXHGbAgd5/tyrRJi
f6uNMxVLRYNIDzn71mapyLJPSaJmLFhDSfnGFaBXUyStJfTnf/Bxtt7LaeDl97qX
//vcjwkRX2uAEh890fiY
=DQNG
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups \
"qubes-devel" group. To unsubscribe from this group and stop receiving emails from \
it, send an email to qubes-devel+unsubscribe@googlegroups.com. To post to this group, \
send email to qubes-devel@googlegroups.com. To view this discussion on the web visit \
https://groups.google.com/d/msgid/qubes-devel/20151222152618.GA15086%40work-mutt. For \
more options, visit https://groups.google.com/d/optout.


[prev in list] [next in list] [prev in thread] [next in thread]