[prev in list] [next in list] [prev in thread] [next in thread]
List: james-user
Subject: Re: Maildir support for Windows.
From: Eric Charles <eric () apache ! org>
Date: 2013-08-31 3:20:20
Message-ID: 522160F4.6040000 () apache ! org
[Download RAW message or body]
That's really good news.
I am not sure you fail on move. In the stacktrack you gave (yes, java
like hiding lines), the SEEN flag is set but the file renaming fails.
>
'..\var\store\maildir\mydomain.co.uk\wibble\cur\1377463834.cd32e9d983446360.MEBBE,S=2284;2,' \
> after copy to
>
'..\var\store\maildir\mydomain.co.uk\wibble\cur\1377463834.cd32e9d983446360.MEBBE,S=2284;2,S' \
JAVA File.delete behaves differently on Windows and Linux
http://markmail.org/message/5onrj3rahwz3b3sw
So yes, you should find what is keeping the handles/locks and see if you
can close it.
On 30/08/13 18:05, Robin Bankhead wrote:
>
> Hi,
>
> I've made some progress insofar as I managed to complete a build from
> trunk with my changes, and got mail being delivered to the WinMailDir
> (like it ;)) via FetchMail.
>
> I'm running into a few probably Windows-centric issues now though.
> (This is on Win7 Pro incidentally.)
>
> First, I had to change the logon identity of the James service
> definition to that of my Administrator user, as under the default
> (LocalSystem), the maildir folders couldn't be created (I think this was
> a perms issue but I don't know exactly why this worked tbh).
>
> Now, it's impossible to move messages (e.g. from inbox to Trash) using
> an IMAP client (Thunderbird, in this instance) as the message gets
> copied to the destination but not deleted from the source. Here's
> typical output:
>
> <snip>
> INFO 15:24:05,365 | james.imapserver | ID=3298599 Store failed for
> mailbox #private:wibble@mydomain.co.uk:INBOX
> org.apache.james.mailbox.exception.MailboxException: Failure while save
> Message MaildirMessage 1 {S} Sun Aug 25 21:50:34 BST 2013 in Mailbox
> #private:wibble@mydomain.co.uk:INBOX
> at
> org.apache.james.mailbox.maildir.mail.MaildirMessageMapper.updateFlags(MaildirMessageMapper.java:233)
>
> at
> org.apache.james.mailbox.store.StoreMessageManager$2.run(StoreMessageManager.java:536)
>
> at
> org.apache.james.mailbox.store.StoreMessageManager$2.run(StoreMessageManager.java:533)
>
> at
> org.apache.james.mailbox.store.transaction.TransactionalMapper.execute(TransactionalMapper.java:37)
>
> at
> org.apache.james.mailbox.store.StoreMessageManager.setFlags(StoreMessageManager.java:533)
>
> at
> org.apache.james.imap.processor.StoreProcessor.setFlags(StoreProcessor.java:245)
>
> at
> org.apache.james.imap.processor.StoreProcessor.doProcess(StoreProcessor.java:164)
>
> at
> org.apache.james.imap.processor.StoreProcessor.doProcess(StoreProcessor.java:57)
>
> at
> org.apache.james.imap.processor.AbstractMailboxProcessor.doProcess(AbstractMailboxProcessor.java:100)
>
> at
> org.apache.james.imap.processor.AbstractMailboxProcessor.process(AbstractMailboxProcessor.java:89)
>
> at
> org.apache.james.imap.processor.AbstractMailboxProcessor.doProcess(AbstractMailboxProcessor.java:83)
>
> at
> org.apache.james.imap.processor.AbstractMailboxProcessor.doProcess(AbstractMailboxProcessor.java:66)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:52)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imap.processor.base.AbstractChainedProcessor.process(AbstractChainedProcessor.java:54)
>
> at
> org.apache.james.imapserver.netty.ImapChannelUpstreamHandler.messageReceived(ImapChannelUpstreamHandler.java:187)
>
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
>
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
>
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777)
>
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:327)
>
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:305)
>
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:207)
>
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
>
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
>
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777)
>
> at
> org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.run(ChannelUpstreamEventRunnable.java:44)
>
> at
> org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:312)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Failed to delete original file
> '..\var\store\maildir\mydomain.co.uk\wibble\cur\1377463834.cd32e9d983446360.MEBBE,S=2284;2,'
> after copy to
> '..\var\store\maildir\mydomain.co.uk\wibble\cur\1377463834.cd32e9d983446360.MEBBE,S=2284;2,S'
>
> at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2835)
> at
> org.apache.james.mailbox.maildir.mail.MaildirMessageMapper.updateFlags(MaildirMessageMapper.java:219)
>
> ... 48 more
> </snip>
>
> (I'm not sure how I get to see the "48 more", I've already turned up
> everything to DEBUG and then to TRACE in log4j.properties.)
>
> Based on some research, I guess that the problem here is that the
> original file still has a lock on it when deletion is attempted. I've
> had a look at the process handles on the message file using Process
> Explorer (SysInternals tool), and accessing a message from the client
> typically opens 2-5 handles on the given file, which are released over a
> few seconds after accessing it (this is garbage-collection I assume).
> However the pattern is similar when moving/deleting a message (even
> after any previous handles have expired) so the issue isn't escapable as
> easily as waiting 10sec between accessing and moving/deleting a message
> (not that that would be practicable anyway).
>
> So maybe what I need to do there is force GC (or find what is keeping
> the handles/locks alive and make it release them) before doing the move?
>
> Also, I'm curious why that operation is copy-then-delete, when other
> move operations in maildir (e.g. from new -> cur when read, which works
> fine) are presumably "normal" moves.
>
> Another issue is that the server seems to report all messages as Draft,
> but one thing at a time ;)
>
> I should have known it wouldn't be so easy... Any thoughts or advice on
> the above?
>
> Robin Bankhead
>
>
> Quoting Eric Charles <eric@apache.org>:
>
> > On 16/08/13 18:36, Robin Bankhead wrote:
> > > Hi Eric,
> > >
> > > Thanks for your reply.
> > >
> > > Quoting Eric Charles <eric@apache.org>:
> > >
> > > > 1. The support on windows has been asked and the answer has been that
> > > > the maildir de-facto norm does not support this.
> > > >
> > > Fair enough; if this document [1] is THE (de facto) spec, then its scope
> > > is obviously narrow insofar as it's *nix-centric and I'm sure Windows
> > > implementation was not considered by the author. Nevertheless, barring
> > > this one technical incompatibility, I can't see anything else standing
> > > against such implementation, can you?
> > >
> > > If the rigidity of the "standard" is considered of such importance that
> > > a deviant implementation isn't acceptable, then an alternative could be
> > > to clone/extend it, implement the necessary character-switch, and call
> > > the result something else (AwesomeDir, whatever) to avoid confusion ;)
> > >
> >
> > WinMailDir?
> >
> > > > 2. Where is the [1] you are referring to? Btw I think we could make it
> > > > configurable to allow the mailbox-maildir to use a windows-friendly
> > > > character. Maybe that character has already been proposed somewhere
> > > > else? The default would of course be the standard norm.
> > > >
> > > Sorry about that, I cut but forgot to paste... That link was just to the
> > > maildir subfolder in the svn repo; instead of that, here's a patch! Not
> > > sure if it's the preferred format (and is against 0.4 version, not
> > > trunk, as my test target initially will be the server 3.0beta4 release)
> > > but should be apparent what it does at least. (Actually it does exactly
> > > nothing functionally different, but replacing all hard-coded references
> > > to the colon with a constant that's easier to flip.)
> > >
> > > I've seen semicolon and (I think) comma suggested in the pages I came
> > > across, but really it matters little as long as it's not a colon, other
> > > NTFS/FAT reserved character, or character that could appear elsewhere in
> > > the filename - which leaves us plenty of options. James can't be the
> > > first project to have wrestled with this particular conundrum when
> > > porting to Windows, so it might be the case that there already is a
> > > favoured choice among those projects, but I'd need to investigate
> > > further.
> > >
> > > Certainly there would have to be a different default for Windows because
> > > the global default simply doesn't work there, but many users wouldn't
> > > care what character was used instead, so that decision ought not to be
> > > forced on them.
> > >
> > > Next I'll look into implementing an OS-detecting switch for this and an
> > > optional preference to specify the character used. I realise this
> > > remains very hypothetical for actual submission, but does that sound
> > > like the best approach?
> > >
> >
> > Rather than os-detecting system, the failing character could be
> > defined by configuration, default being the standard one.
> >
> > > Thanks,
> > > Robin Bankhead
> > >
> > > [1] http://cr.yp.to/proto/maildir.html
> > >
> > > > On 14/08/13 16:11, Robin Bankhead wrote:
> > > > > Hi,
> > > > >
> > > > > I've been looking at James as a replacement for our legacy Windows-
> > > > > based
> > > > > email setup, at which point my hopes of a maildir-based message store
> > > > > were dashed (because of a filesystem reserved character (colon) in the
> > > > > spec).
> > > > >
> > > > > Moving to maildir appealed to me because the legacy software uses mbox
> > > > > but the local spool is now up to nearly 2GB and the risk of corruption
> > > > > is starting to bother me (well, that and the next Windows update could
> > > > > render that abandonware unusable). I daresay it would improve
> > > > > performance somewhat as well.
> > > > >
> > > > > I've had a look at the source, and from what I can see, the edits
> > > > > required to use an alternative character (eg semicolon) in message
> > > > > file
> > > > > naming only amount to a couple of lines, or a bit more to make it a
> > > > > configurable option.
> > > > >
> > > > > I'm ready to try building and testing a patch myself, but before I
> > > > > fling
> > > > > myself into that I felt it sensible to ask:
> > > > >
> > > > > 1. Has this been proposed before, and if so, are there technical
> > > > > reasons
> > > > > why it was decided against? (No sense in going over old ground)
> > > > >
> > > > > 2. Should I be looking further afield in the source than the specific
> > > > > Maildir code, i.e. the contents of [1], for potential conflicts? (I
> > > > > guess in theory the answer should be "no", but y'all know the codebase
> > > > > better than I!)
> > > > >
> > > > > Any advice would be very welcome.
> > > > >
> > > > > Regards,
> > > > > Robin Bankhead
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> > > > > For additional commands, e-mail: server-user-help@james.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> > > > For additional commands, e-mail: server-user-help@james.apache.org
> > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> > > For additional commands, e-mail: server-user-help@james.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> > For additional commands, e-mail: server-user-help@james.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic