'Re: [zeromq-dev] PUSH socket blocks when no client is connected'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       zeromq-dev
Subject:    Re: [zeromq-dev] PUSH socket blocks when no client is connected
From:       Robert Pickering <robertfpickering () fastmail ! com>
Date:       2015-09-17 13:53:33
Message-ID: 1442498013.657835.386312073.40F88984 () webmail ! messagingengine ! com
[Download RAW message or body]

Pieter - thanks for the RFCs reference, I've now read and it helped
clarify the behaviour. Messages are only dropped if the the owner
doesn't realizes they still own the message :)

Doron - I was unaware of Malamute, so thanks for the pointer. I'm now
investigating.

Rob

On Mon, Sep 14, 2015, at 06:24 PM, Pieter Hintjens wrote:
> Robert, if you send with DONTWAIT and get back an error, the message
> is not queued, and it's not dropped, simply still owned by the caller.
> 
> If you connect out, then messages are accepted, as a connect creates a
> pipe. This behavior is explained in the various RFCs
> (http://rfc.zeromq.org/spec:30)
> 
> -Pieter
> 
> On Mon, Sep 14, 2015 at 5:56 PM, Doron Somech <somdoron@gmail.com> wrote:
> > Try Malamute, the pubsub does something similar.
> >
> > Also if you reverse the connection it will work. Make publisher connect to
> > the subscriber.
> >
> > On Sep 14, 2015 9:36 AM, "Robert Pickering" <robertfpickering@fastmail.com>
> > wrote:
> >>
> >> Thanks for all the very quick responses. Setting ZMQ_DONTWAIT does
> >> indeed stop the send method from blocking, but message is dropped rather
> >> than queued, as per Justin's explanation below. I'll look into storing
> >> the messages outside ZMQ, as it looks like ZMQ isn't really the place
> >> store messages in this scenario.
> >>
> >> Thanks again!
> >> Rob
> >>
> >> On Mon, Sep 14, 2015, at 05:27 PM, Justin Karneges wrote:
> >> > It's better to use REQ/REP for recovery if a message is missed. Even if
> >> > you could queue messages in an PUSH socket for a client that has not
> >> > arrived, it wouldn't be a very stable place to store the messages since
> >> > no ZMQ socket actually guarantees delivery. Better to store the messages
> >> > outside of ZMQ.
> >> >
> >> > Btw if you're wondering why your PUSH socket blocks, here's why. For
> >> > each ZMQ socket, there is a queue per "known" peer. If you connect(),
> >> > then the socket immediately gains a queue for that peer (which never
> >> > goes away), and if you call connect() multiple times then the socket
> >> > gets multiple queues (one per peer). If you bind(), then the socket
> >> > gains a queue for every incoming connection, but only for as long as
> >> > each connection exists; if a peer disconnects, then the queue for that
> >> > peer is destroyed. In short, if you have a socket that binds and nobody
> >> > is connected, then you have no queues and so a write will always block
> >> > or drop, depending on the socket type. There is simply nowhere to put
> >> > the message.
> >> >
> >> > On Mon, Sep 14, 2015, at 07:56 AM, Robert Pickering wrote:
> >> > > Hi all,
> >> > >
> >> > > I'm running into a problem that PUSH sockets always block on send when
> >> > > no PULL socket is connected.
> >> > >
> >> > > First a brief note about what I'm trying to achieve: I'm trying to
> >> > > distribute notifications from one server to a number of clients. The
> >> > > scenario looks a lot like sub/pub, but there's an important difference
> >> > > that means sub/pub is not suitable. The difference is I'd like all
> >> > > clients to receive all notifications and tolerate a short
> >> > > disconnection
> >> > > period. When a client crashes the server should queue messages for the
> >> > > client until the client restarts. If the client is disconnected for
> >> > > too
> >> > > long then messages can be thrown away.
> >> > >
> >> > > With with SUB/PUB sockets, if a crash in the client occurs the
> >> > > notification received are received by all remaining active clients but
> >> > > are lost to the client that has crashed. I'm trying to implement this
> >> > > scenario using PUSH/PULL sockets but running into the problem that
> >> > > send
> >> > > blocks when no PULL client is connected.
> >> > >
> >> > > I know the number of clients in advance, so each client has it's own
> >> > > PUSH socket on the server. To send a notification I loop over all the
> >> > > sockets on the server and send the notification. This works well until
> >> > > one client is disconnected then it's socket starts to block, so no
> >> > > further notifications can be send.
> >> > >
> >> > > I've tried many things to get the PUSH to queue the message instead of
> >> > > blocking, including setting the socket options: ZMQ_SNDBUF to 1000000,
> >> > > ZMQ_SNDHWM to 10000 and ZMQ_IMMEDIATE to 1, although from my reading
> >> > > of
> >> > > the docs none of these should be necessary.
> >> > >
> >> > > Is there anyway to get the PUSH socket to queue the messages rather
> >> > > than
> >> > > block when no client is connected? Or, if what I'm trying is not the
> >> > > best pattern to implement the scenario described I'd be very
> >> > > interested
> >> > > to hear ideas on other ways to implement it.
> >> > >
> >> > > Many thanks,
> >> > > Robert
> >> > > _______________________________________________
> >> > > zeromq-dev mailing list
> >> > > zeromq-dev@lists.zeromq.org
> >> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> > _______________________________________________
> >> > zeromq-dev mailing list
> >> > zeromq-dev@lists.zeromq.org
> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev@lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev@lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[prev in list] [next in list] [prev in thread] [next in thread] 