[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-dev
Subject:    [jira] [Resolved] (HADOOP-16798) job commit failure in S3A MR magic committer test
From:       "Steve Loughran (Jira)" <jira () apache ! org>
Date:       2020-06-30 9:55:00
Message-ID: JIRA.13278752.1578673687000.362511.1593510900542 () Atlassian ! JIRA
[Download RAW message or body]


     [ https://issues.apache.org/jira/browse/HADOOP-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel \
]

Steve Loughran resolved HADOOP-16798.
-------------------------------------
    Fix Version/s: 3.3.1
       Resolution: Fixed

> job commit failure in S3A MR magic committer test
> -------------------------------------------------
> 
> Key: HADOOP-16798
> URL: https://issues.apache.org/jira/browse/HADOOP-16798
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Fix For: 3.3.1
> 
> Attachments: stdout
> 
> 
> failure in 
> {code}
> ITestS3ACommitterMRJob.test_200_execute:304->Assert.fail:88 Job \
> job_1578669113137_0003 failed in state FAILED with cause Job commit failed: \
> java.util.concurrent.RejectedExecutionException: Task \
> java.util.concurrent.FutureTask@6e894de2 rejected from \
> org.apache.hadoop.util.concurrent.HadoopThreadPoolExecutor@225eed53[Terminated, \
> pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] {code}
> Stack implies thread pool rejected it, but toString says "Terminated". Race \
>                 condition?
> *update 2020-04-22*: it's caused when a task is aborted in the AM -the threadpool \
> is disposed of, and while that is shutting down in one thread, task commit is \
> initiated using the same thread pool. When the task committer's destroy operation \
> times out, it kills all the active uploads.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


[prev in list] [next in list] [prev in thread] [next in thread]