[prev in list] [next in list] [prev in thread] [next in thread] 

List:       linux-rdma
Subject:    Re: ibv_fork_init() question
From:       Roland Dreier <roland () kernel ! org>
Date:       2012-02-27 19:58:45
Message-ID: CAG4TOxMSS_2V8peLn6fcd9q6oWgWOFnCsxazAxKvhdRhnes46w () mail ! gmail ! com
[Download RAW message or body]

On Mon, Feb 27, 2012 at 11:54 AM, Steve Wise
<swise@opengridcomputing.com> wrote:
> I'm experimenting with doing RDMA operations from a child process and I have
> a question.  My application calls ibv_fork_init() at startup (first thing in
> main())  and it succeeds, and I see, for instance, that libibverbs has
> initialized mm_root.  Next my application registers page-aligned memory that
> is 8KB in size.  The memory was allocated with memalign().  Then the app
> sets up a connection to an RDMA peer.  Next the application calls fork().
>  The parent does a wait() and the child does an rdma read operation into
> this registered memory from a valid mr of the peer.  The read succeeds from
> the rdma device perspective.  But when the child process tries to read the
> memory (to verify the rdma operation worked), I get a seg fault.
>
> Q: Am I doing something incorrect here?  I can't find much documentation on
> how this all should work.

Yeah, unfortunately this isn't really intended to work and I don't have a
good suggestion on how to make it work.

Even with the libibverbs fork support, the expectation is that all RDMA
stuff will be done from the master process.  All that the ibv_fork_init()
call does is trigger tracking so that the address space that is used for
RDMA isn't mapped into the child process at all.

So yeah, what you see happening is exactly what I would expect.

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
[prev in list] [next in list] [prev in thread] [next in thread]