[prev in list] [next in list] [prev in thread] [next in thread] 

List:       jakarta-commons-dev
Subject:    [jira] [Comment Edited] (MATH-1405) Kolmogorov-Smirnov fixTies can set minDelta too small for jiggle
From:       "Daniil Finkel (JIRA)" <jira () apache ! org>
Date:       2017-02-28 21:58:45
Message-ID: JIRA.13047102.1488317910000.23091.1488319125578 () Atlassian ! JIRA
[Download RAW message or body]


    [ https://issues.apache.org/jira/browse/MATH-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888931#comment-15888931 \
] 

Daniil Finkel edited comment on MATH-1405 at 2/28/17 9:58 PM:
--------------------------------------------------------------

A temporary solution I've found is, in fixTies(), pulling the RealDistribution Object \
creation inside the do-while loop, and doubling the minDelta with every iteration \
until it's large enough:

{code}
do {
            LOGGER.warn("using minDelta: {}", minDelta);
            // Add jitter using a fixed seed (so same arguments always give same \
results),  // low-initialization-overhead generator
            final RealDistribution dist = new UniformRealDistribution(new \
JDKRandomGenerator(100), -minDelta, minDelta);

            jitter(x, dist);
            jitter(y, dist);
            ties = hasTies(x, y);
            ct++;
            minDelta *= 2;
        } while (ties && ct < 1000);
{code}


was (Author: dfinkel):
A temporary solution I've found is, in fixTies(), pulling the RealDistribution Object \
creation inside the do-while loop, and doubling the minDelta with every iteration \
until it's large enough:

`do {
            LOGGER.warn("using minDelta: {}", minDelta);
            // Add jitter using a fixed seed (so same arguments always give same \
results),  // low-initialization-overhead generator
            final RealDistribution dist = new UniformRealDistribution(new \
JDKRandomGenerator(100), -minDelta, minDelta);

            jitter(x, dist);
            jitter(y, dist);
            ties = hasTies(x, y);
            ct++;
            minDelta *= 2;
        } while (ties && ct < 1000);
`

> Kolmogorov-Smirnov fixTies can set minDelta too small for jiggler to have \
>                 significant effect
> --------------------------------------------------------------------------------------------
>  
> Key: MATH-1405
> URL: https://issues.apache.org/jira/browse/MATH-1405
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.6.1
> Reporter: Daniil Finkel
> 
> For samples that do not exceed LARGE_SAMPLE_PRODUCT with their product and \
> relatively large values, a minDelta can be calculated in fixTies() that is too \
> small to have any effect on the "tied" values. This results in a MathInternalError, \
> as the jiggling with the ineffective minDelta fails to fix the ties. The following \
> arrays exhibit this behavior when run with kolmogorovSmirnovTest(x, y) in 3.6.1 x = \
> [1.3750969645841487, 1.0845460746754014, 1.3693352427126644, 1.329688765445783, \
> 1.3392109491039106, 1.3532766470312723, 1.3187287426697727, 1.386273031970554, \
> 1.3416950149276097, 1.0510872606482404, 1.3532766470312723, 1.3075923871137798, \
> 1.3862730319705543, 1.3814421433922548, 1.0527927570919202, 1.3847314864464313, \
> 1.319362658529506, 1.3579238253227275, 1.2455452272301641, 1.329688765445783, \
> 1.3827781646781876, 1.0755168081687903, 1.2566273460024566, 1.3099622795250825, \
> 1.357440924560318, 1.3519397370266515, 1.0927347979524134, 1.3566357346921618, \
> 1.238800036669969, 1.2931730628634528, 1.048463407884969, 1.3779471642491719, \
> 1.2978533797116658, 1.376230881554943, 1.166901202345226, 1.3690425182006263, \
> 1.166901202345226, 1.2953476417603207, 1.0827945761165951, 1.2942406680885112, \
> 1.224414840377028, 1.3910905417259205, 1.303231085263425, 1.348635183816037, \
> 1.3750969645841487, 1.049648651501274, 1.3119534979602083, 1.0446033225080773, \
> 1.0494686631294756, 1.3862026705844126, 1.2719496963348844, 1.3489938748102903, \
> 1.3780468374004164, 1.3884878389662338, 1.3352682241994538, 1.3348722240568909, \
> 1.3921944407986777, 1.0476833161122294, 1.0845460746754008, 1.344165352323966, \
> 1.298548179079665, 1.1979240079667628, 1.3539078973394736, 1.3187287426697725, \
> 1.082794576116595, 1.3779471642491719, 1.3771347858434184, 1.3921944407986777, \
> 1.193793081523992, 1.362050393265006, 1.076638744462226, 1.3551174562135766, \
> 1.3393693468578751, 1.2470361076952952, 1.3696023478216113, 1.3750969645841487, \
> 1.2964734722088322, 1.2953476417603207, 1.2470361076952952, 1.382661263313539, \
> 1.3862026705844126, 1.3771240109822156, 1.25443884328785, 1.3136690818105938, \
> 1.3853832858443051, 1.3486351838160378, 1.348026557887345, 1.0604869883721861, \
> 1.3352682241994536, 1.3480480718535308, 1.3363233390543028, 1.154658436584056, \
> 1.3921944407986775, 1.1979240079667626, 1.3620503932650059, 1.0881358731694244, \
> 1.369042518200626, 1.3532766470312723, 1.2890012831575908, 1.3735565244300663] and
> y = [1.1262991662205104, 1.3136690818105938, 1.0446033225080773, \
> 1.3551174562135764, 1.3032310852634252, 1.3806258468851462, 1.2270612333345983, \
> 1.2719496963348844, 1.3601566259413194, 1.3756888280688913, 1.3475322202511097, \
> 1.1937930815239919, 1.0510872606482404, 1.3441653523239654, 1.359738761905118, \
> 1.3382152957887032, 1.0766387444622263, 1.1937930815239919, 1.0820779503060238, \
> 1.1448104521200428, 1.3853832858443051, 1.28757746537949, 1.298548179079665, \
> 1.067255392172351, 1.3168701741293156, 1.3910905417259205, 1.2908594990421354, \
> 1.3750969645841487, 1.329688765445783, 1.386649365275275, 1.285486511663053, \
> 1.2566273460024566, 1.323664826995234, 1.3862730319705538, 1.049346328049449] which \
> produce minDelta = 1.11022302462516E-016



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[prev in list] [next in list] [prev in thread] [next in thread]