[prev in list] [next in list] [prev in thread] [next in thread]
List: solr-dev
Subject: Re: Gradle HELP
From: Dawid Weiss <dawid.weiss () gmail ! com>
Date: 2021-08-20 19:56:28
Message-ID: CAM21Rt_u6Zx2V2md=p_BHWBQjUXjQ4LrnoN8JKVWLRKP27+D0Q () mail ! gmail ! com
[Download RAW message or body]
I'm actually looking into switching to gradle 7... The only
problematic part I see so far is jvm registries. Don't know how to
solve it yet. They've changed a lot and I need to figure out whether
it's possible to somehow keep the existing behavior or if we need to
adopt to the new conventions (I'd rather stick to what we're used to).
Dawid
On Fri, Aug 20, 2021 at 6:13 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> In the midst of all this hackery I have been using Gradle 6.9 and it
> seems to have support for Java 15 - should we switch? Might be easier
> than going to 7
>
> On Thu, Aug 19, 2021 at 4:26 PM Michael Sokolov <msokolov@gmail.com> wrote:
> >
> > ah nvm I see it is the wrong version of junit - I had 4.12 and need to
> > get to 4.13
> >
> > On Thu, Aug 19, 2021 at 4:21 PM Michael Sokolov <msokolov@gmail.com> wrote:
> > >
> > > Thanks for your sympathy :) I am told the situation here should
> > > improve soon... After working around the security policy I am able to
> > > run some tests, but I still have some dependency issues around junit
> > > methods -- I think maybe it's some issue w/hamcrest? What I see is
> > > most methods get resolved correctly, but eg assertThrows does not.
> > >
> > > I see we have this tricksy logic in test-framework's build.gradle:
> > >
> > > - api ("com.carrotsearch.randomizedtesting:randomizedtesting-runner",
> > > {
> > > - exclude group: "junit"
> > > - })
> > > - api ("junit:junit", {
> > > - exclude group: "org.hamcrest"
> > > - })
> > > - api ('org.hamcrest:hamcrest')
> > >
> > > and I wonder why we need these exclusions?
> > >
> > >
> > > On Thu, Aug 19, 2021 at 8:44 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> > > >
> > > > Ok, clear. I understand the reasons these things are shielded from the
> > > > internet. I'm sure gradle folks have thought about such scenarios
> > > > though - gradle is used across enterprises with similar setups? I
> > > > really know nothing about it but I feel pretty confident artifact
> > > > resolution can be redirected to only known locations. Sorry you have
> > > > to hack around!
> > > >
> > > > Dawid
> > > >
> > > > On Thu, Aug 19, 2021 at 2:00 PM Michael Sokolov <msokolov@gmail.com> wrote:
> > > > >
> > > > > > This seems a tad insane?
> > > > >
> > > > > Agree, and yes it's an Amazon thing. Basically our build servers are
> > > > > not connected to the internet, and we use an internal repository and
> > > > > corresponding gradle plugin for resolving dependencies that is
> > > > > somewhat idiosyncratic (not Maven or anything else familiar, but acts
> > > > > kind of like Maven repo). Uwe, I am basically doing what you say,
> > > > > re-using as much as I can from our excellent Lucene build system, but
> > > > > wrapping alone is insufficient: I have to make invasive changes here
> > > > > and there. Believe me this will be a big improvement from the hacked
> > > > > ant build we have been using.
> > > > >
> > > > > On Thu, Aug 19, 2021 at 5:00 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > would you give us some hints what you are trying to do? Is this related \
> > > > > > to the Lucene build system and are you preparing a PR or is this some \
> > > > > > Amazon-only rewrite? In the latter case, can't you not wrap the execution \
> > > > > > of gradlew in your "internal" build system? You can also configure gradle \
> > > > > > to fetch dependencies from other repositories than maven central.
> > > > > > Uwe
> > > > > >
> > > > > > -----
> > > > > > Uwe Schindler
> > > > > > Achterdiek 19, D-28357 Bremen
> > > > > > https://www.thetaphi.de
> > > > > > eMail: uwe@thetaphi.de
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Michael Sokolov <msokolov@gmail.com>
> > > > > > > Sent: Thursday, August 19, 2021 5:18 AM
> > > > > > > To: Lucene Dev <dev@lucene.apache.org>
> > > > > > > Subject: Re: Gradle HELP
> > > > > > >
> > > > > > > OK I see it's because this build system puts the output in a different
> > > > > > > folder that is not covered by the security policy. phew.
> > > > > > >
> > > > > > > On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Update: running tests with -Dtests.useSecurityManager=false got over
> > > > > > > > that hump. Not sure why the security manager would be behaving
> > > > > > > > differently in this setup...
> > > > > > > >
> > > > > > > > On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > ah interesting. I may have broken it in the midst of debugging
> > > > > > > > > something else. I did manage to get most of the module jars built, \
> > > > > > > > > and I'm moving on to trying to get tests to run. I'm running into \
> > > > > > > > > what looks like more dependency/classpath resolution problems, but \
> > > > > > > > > with our module-module dependencies. If anyone has a clue what \
> > > > > > > > > might be going on or can suggest some futher debugging tips, I'd be \
> > > > > > > > > very appreciative. Here are the gory details...
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > For example, tests in o.a.l.codecs are failing because:
> > > > > > > > >
> > > > > > > > > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat \
> > > > > > > > > > classMethod FAILED
> > > > > > > > > java.lang.ExceptionInInitializerError
> > > > > > > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > > > > > > at
> > > > > > > org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup
> > > > > > > AndRestoreClassEnv.java:133)
> > > > > > > > > ...
> > > > > > > > > java.lang.IllegalArgumentException: An SPI class of type
> > > > > > > > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > > > > > > > > You need to add the corresponding JAR file supporting this SPI to \
> > > > > > > > > your classpath. The current classpath supports the following \
> > > > > > > > > names: [SimpleText]
> > > > > > > > >
> > > > > > > > > and I print out the classpath used to run the test by:
> > > > > > > > >
> > > > > > > > > tasks.withType(Test) {
> > > > > > > > > doFirst {
> > > > > > > > > println "Test $allJvmArgs"
> > > > > > > > > println "Test classpath " + classpath.getFiles().toString()
> > > > > > > > > }
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > and the classpath includes
> > > > > > > > >
> > > > > > > > > ...build/lib/lucene-core-9.0.jar
> > > > > > > > >
> > > > > > > > > and that jar includes
> > > > > > > > >
> > > > > > > > > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > > > > > > > > META-INF/services/org.apache.lucene.codecs.Codec
> > > > > > > > >
> > > > > > > > > which has the expected content -- so I am kind of confused. Are \
> > > > > > > > > these Codec files supposed to be merged together? Or is this a \
> > > > > > > > > conflict resolved by classpath-order?
> > > > > > > > >
> > > > > > > > > and there are other dependency issues, but I think it would just be
> > > > > > > > > more confusing to list them all here.
> > > > > > > > >
> > > > > > > > > PS: full listing below:
> > > > > > > > >
> > > > > > > > > Test classpath
> > > > > > > [/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > > > > ses/java/test,
> > > > > > > > >
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > > > > urces/test,
> > > > > > > > >
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > > > > ses/java/main,
> > > > > > > > >
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > > > > urces/main,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-
> > > > > > > > >
> > > > > > > framework-9.0_az.jar,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-
> > > > > > > codecs-9.0_az.jar,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-
> > > > > > > > >
> > > > > > > 9.0_az.jar,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > > > > 2/brazil/junit/4.12/junit-4.12.jar,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > > > > 2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > > 2.1/com.carrotsearch.randomizedtesting/randomizedtesting-
> > > > > > > runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesti
> > > > > > > ng-runner-2.7.2.jar,
> > > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > > 2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-
> > > > > > > 4.12.jar,
> > > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > > 2.1/org.hamcrest/hamcrest-
> > > > > > > core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-
> > > > > > > 1.3.jar,
> > > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > > 2.1/org.hamcrest/java-
> > > > > > > hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-
> > > > > > > hamcrest-2.0.0.0.jar]
> > > > > > > > >
> > > > > > > > > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > > > > > > > >
> > > > > > > > > > I believe it is one of the built in gradle tasks available to us.
> > > > > > > > > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > > > > > > > > >
> > > > > > > > > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov
> > > > > > > <msokolov@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > I am trying to get Lucene to build in a very "special" build \
> > > > > > > > > > > system based on Gradle, and I am stumbling across all kinds of \
> > > > > > > > > > > things I don't really understand (which is most of everything \
> > > > > > > > > > > about gradle at this point). Cookie cutter cargo cultism is \
> > > > > > > > > > > getting me pretty far, but one thing I don't understand: we \
> > > > > > > > > > > have this in the precommit task:
> > > > > > > > > > > // Root-level validation tasks.
> > > > > > > > > > > dependsOn ":verifyLocks"
> > > > > > > > > > > dependsOn ":versionsPropsAreSorted"
> > > > > > > > > > > dependsOn ":checkWorkingCopyClean"
> > > > > > > > > > >
> > > > > > > > > > > What is the "verifyLocks" task? I can't seem to find it defined \
> > > > > > > > > > > anywhere
> > > > > > > > > > > -Mike
> > > > > > > > > > >
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > \
> > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic