[prev in list] [next in list] [prev in thread] [next in thread] 

List:       git
Subject:    RE: Consist timestamps within a checkout/clone
From:       <rsbecker () nexbridge ! com>
Date:       2022-10-31 22:42:02
Message-ID: 005e01d8ed7a$020589a0$06109ce0$ () nexbridge ! com
[Download RAW message or body]

On October 31, 2022 6:31 PM, Mark Hills wrote:
> On Mon, 31 Oct 2022, Taylor Blau wrote:
> > On Mon, Oct 31, 2022 at 09:21:20PM +0100, Ævar Arnfjörð Bjarmason wrote:
> > > I think you're almost certainly running into the parallel checkout,
> > > which is new in that revision range. Try tweaking checkout.workers
> > > and checkout.thresholdForParallelism (see "man git-config").
> > > 
> > > I can't say without looking at the code/Makefile (and even then, I
> > > don't have time to dig here:), but if I had to bet I'd say that your
> > > dependencies have probably always been broken with these checked-in
> > > files, but they happend to work out if they were checked out in
> > > sorted order.
> > > 
> > > And now with the parallel checkout they're not guaranteed to do
> > > that, as some workers will "race ahead" and finish in an unpredictable order.
> > 
> > Doesn't checkout.thresholdForParallelism only matter when
> > checkout.workers != 1?
> > 
> > So what you wrote seems like a reasonable explanation, but only if the
> > original reporter set checkout.workers to imply the non-sequential
> > behavior in the first place.
> > 
> > That said...
> > 
> > - I also don't know off-hand of a place where we've defined the order
> > where Git will checkout files in the working copy. So depending on
> > that behavior isn't a safe thing to do.
> > 
> > - Committing build artifacts into your repository is generally
> > discouraged.
> 
> If it's undefined and never implemented this is reasonable.
> 
> But "generally" is a caveat, so while I agree with the statement it also implies
> there's valid cases outside of that. Ones which used to work, too.
> 
> Here are some useful cases I have seen for the combination of build rule +
> checked in file:
> 
> - part of a build requires licensed software that's not always available
> 
> - part of the build requires large memory that other builders generally do
> not have available
> 
> - part of the build process uses a different platform or some other system
> requirement
> 
> - to fetch data eg. from a URL, with a record of the URL/automation but
> also a copy of the file as a record and for offline use
> 
> So it's useful, to retain repeatable automation but not always build from square
> one.
> 
> Generally discouraged to check in build results yes, but I've found it very \
> practical. 
> > So while I'd guess that setting `checkout.workers` back to "1" (if it
> > wasn't already) will probably restore the existing behavior, counting
> > on that behavior in the first place is wrong.
> 
> I think perhaps the tail is wagging the dog here, though.
> 
> It's 'wrong' because it doesn't work; but I haven't seen anything to make me think
> this is fundamentally or theoretically flawed.
> 
> If we had a transactional file system we'd reasonably expect a checkout to be an
> atomic operation -- same timestamp on the files created in that step. A
> discrepancy in timestamps would be considered incorrect; it would imply an 'order'
> to the checkout which, as you say, is order-less.
> 
> Sowhat could be the bad outcomes if Git created files stamped with the point in
> time of the "git checkout"?

Timestamps are written based on when git modifies the file in the working directory. \
This actually ensures that automation does work. If intermediate contents are checked \
into repositories (I have people who do this for very justifiable regulatory \
reasons), the build has to make sure that there are appropriate separations of \
timestamps (a.k.a. 1 second) at a minimum on UNIX-ish systems. On some other boxes \
that do not even have timestamps for files (you know who you are) this is moot.

However, there is a use case for maintaining timestamps - specifically for debuggers \
that check timestamps of source files. It is a big pain to make this work in git - \
but I script around this by setting the timestamps of files to the commit time when \
doing release builds, and allowing users to set the timestamp to the same for \
debugging. It helps but should not change the semantics of dev builds.

-Randall


[prev in list] [next in list] [prev in thread] [next in thread]