[prev in list] [next in list] [prev in thread] [next in thread] 

List:       flume-dev
Subject:    [jira] [Commented] (FLUME-2866) Add fileTimeMinOffsetSeconds property to Spooling Directory Source
From:       "Jonathan Smith (JIRA)" <jira () apache ! org>
Date:       2016-02-24 17:00:20
Message-ID: JIRA.12933215.1453420634000.135061.1456333220498 () Atlassian ! JIRA
[Download RAW message or body]


    [ https://issues.apache.org/jira/browse/FLUME-2866?page=com.atlassian.jira.plugin. \
system.issuetabpanels:comment-tabpanel&focusedCommentId=15163321#comment-15163321 ] 

Jonathan Smith commented on FLUME-2866:
---------------------------------------

This would also be useful for uploading rolling logs, where we don't want flume to \
process the file until the program generating it stops writing to it.

> Add fileTimeMinOffsetSeconds property to Spooling Directory Source
> ------------------------------------------------------------------
> 
> Key: FLUME-2866
> URL: https://issues.apache.org/jira/browse/FLUME-2866
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Philippe Lantin
> Priority: Minor
> 
> When using a spooling directory source, it would be useful to have the ability to \
> specify that files must have a last modified timestamp that is off by a \
> configurable amount of seconds, either in the future or the past. For example, if I \
> copy a large file to the spooling directory and it takes several minutes to copy, I \
> do not want my file to started being processed before it is completed. A practical \
> way to do this is by looking at the last modified timestamp: files that are being \
> transferred will update this timestamp. In many filesystems, it is possible for \
> clients to set the time in the future, though this is usually done after a file has \
> been completed transferred. For example "cp -p" on linux. I propose a new property \
> for the Spooling Directory Source: fileTimeMinOffsetSeconds. The default would be \
> "0", preserving current behavior. If fileTimeMinOffsetSeconds=60, files will only \
> be picked up if the last modified time is +/- 60 seconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[prev in list] [next in list] [prev in thread] [next in thread]