[prev in list] [next in list] [prev in thread] [next in thread]
List: owfs-developers
Subject: Re: [Owfs-developers] owserver polling so cached values never delay
From: Stefano Miccoli <mocme () icloud ! com>
Date: 2015-09-23 20:36:51
Message-ID: 83893204-D109-44E6-8572-F80E6FF3A362 () icloud ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
I would follow Colin suggestion:
do not tweak owserver, but deploy an in memory cache that acts like a buffer between \
owserver and the clients.
I would suggest redis http://redis.io <http://redis.io/> (or memcached, but I prefer \
redis).
A basic setup would be to have a daemon process that continuously polls owserver and \
feeds data into the redis data store, while your clients fetch data from redis. If \
you use python this approach will require just a few lines of code.
Here is a minimalistic example (not production ready, just to get started.)
The updater code snippet:
from pyownet import protocol
import redis
EXPTIME = 60 # expiry time in seconds, to have stale data evicted from cache
owproxy = protocol.proxy(host="localhost", port=4304, flags=protocol.FLG_UNCACHED)
rdb = redis.StrictRedis(host="localhost", port=6379, db=0)
while True:
for sensor in owproxy.dir("/", slash=False, bus=False):
temperature = owproxy.read(sensor + "/temperature")
rdb.set(sensor, temperature, ex=EXPTIME)
The client code snippet:
import redis
rdb = redis.StrictRedis(host='localhost', port=6379, db=0)
temperature = rdb.get('/28.67C6697351FF')
On my rpi2 the rdb.get call takes less than 1ms, but have a look on \
http://redis.io/topics/benchmarks <http://redis.io/topics/benchmarks> to have an idea \
on how fast redis can be. Installation is painless, python-side you just "pip install \
pyownet redis", while redis server is for sure already packaged in CentOS.
Bye
Stefano
> On 23 Sep 2015, at 14:35, Alex R. Gibbs <agibbs@lpl.arizona.edu \
> <mailto:agibbs@lpl.arizona.edu>> wrote:
> Hello,
>
> We have dozens of DS18B20 temperature sensors at 3 telescopes. I'd like to poll
> them and cache the results so that other programs can retrieve the latest
> temperatures with less than 50ms delay. It doesn't matter if the values are a
> few minutes old; just that clients don't have to wait for a response. I am
> using owserver 3.1p0 on CentOS 6.5 64bit with a LinkHub-E.
>
> Is it possible to configure owserver to continuously poll devices on it's own?
> The owserver cache works but of course once a value in the cache expires the
> next request for it forces a read of the device, making the client wait about
> 800ms. If owserver can be set to poll devices then I can set the expire time
> such that this usually never happens.
>
> I tried having a background program poll the uncached values in owserver more
> often than the volatile timeout so that the cached values are never considered
> expired. However, a client still has to wait for a cached value if it is
> currently being read out due to an uncached request, even if the cached value
> isn't expired. If that was changed I think this would work.
>
> Any suggestions on how to setup polling so that a client can always get a cached
> value immediately? I have a feeling I'll need to either modify owserver or
> write my own server wrapper.
>
> Thanks,
> Alex
>
> --
> Alex R. Gibbs - agibbs@lpl.arizona.edu <mailto:agibbs@lpl.arizona.edu>
> Principal Engineer - Catalina Sky Survey
> Lunar & Planetary Lab - University of Arizona
> --
>
> ------------------------------------------------------------------------------
> Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
> Get real-time metrics from all of your servers, apps and tools
> in one place.
> SourceForge users - Click here to start your Free Trial of Datadog now!
> http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140 \
> <http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140> \
> _______________________________________________ Owfs-developers mailing list
> Owfs-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/owfs-developers
[Attachment #5 (unknown)]
<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"><meta \
http-equiv="Content-Type" content="text/html charset=utf-8"></head><body \
style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: \
after-white-space;" class="">I would follow Colin suggestion:<div class=""><br \
class=""></div><div class="">do not tweak owserver, but deploy an in memory cache \
that acts like a buffer between owserver and the clients.<br class=""><div \
class=""><br class=""></div><div class="">I would suggest redis <a \
href="http://redis.io" class="">http://redis.io</a> (or memcached, but I prefer \
redis).</div><div class=""><br class=""></div><div class="">A basic setup would be to \
have a daemon process that continuously polls owserver and feeds data into the redis \
data store, while your clients fetch data from redis. If you use python this approach \
will require just a few lines of code.</div><div class=""><br class=""></div><div \
class="">Here is a minimalistic example (not production ready, just to get \
started.)</div><div class=""><br class=""></div><div class="">The updater code \
snippet:</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; \
font-size: 14px; font-family: Menlo;" class="">from pyownet import \
protocol</div></div><div style="margin: 0px; font-size: 14px; font-family: Menlo;" \
class="">import redis</div><div style="margin: 0px; font-size: 14px; font-family: \
Menlo;" class=""><br class=""></div><div style="margin: 0px; font-size: 14px; \
font-family: Menlo;" class="">EXPTIME = 60 # expiry time in seconds, to have stale \
data evicted from cache</div><div style="margin: 0px; font-size: 14px; font-family: \
Menlo;" class=""><br class=""></div><div style="margin: 0px;" class=""><span \
style="font-family: Menlo; font-size: 14px;" class="">owproxy = \
protocol.proxy(host="localhost", port=4304, flags=</span><font face="Menlo" size="4" \
class="">protocol.FLG_UNCACHED)</font></div><div style="margin: 0px;" class=""><font \
face="Menlo" size="4" \
class="">rdb = redis.StrictRedis(host="localhost", port=6379, db=0)</font></div><div \
style="margin: 0px;" class=""><font face="Menlo" size="4" class=""><br \
class=""></font></div><div style="margin: 0px;" class=""><font face="Menlo" size="4" \
class="">while True:</font></div><div style="margin: 0px;" class=""><font \
face="Menlo" size="4" class=""> for sensor in owproxy.dir("/", \
slash=False, bus=False):</font></div><div style="margin: 0px;" class=""><span \
style="font-family: Menlo; font-size: 14px;" class=""> \
temperature = ow</span><font face="Menlo" size="4" class="">proxy.read(sensor + \
"/temperature")</font></div><div style="margin: 0px;" class=""><font face="Menlo" \
size="4" class=""> rdb.set(sensor, temperature, \
ex=EXPTIME)</font></div><div style="margin: 0px;" class=""><br class=""></div><div \
class=""><br class=""></div><div class="">The client code snippet:</div><div \
class=""><br class=""></div><div class=""><span style="font-family: Menlo; font-size: \
14px;" class="">import redis</span></div><div class=""><span style="font-family: \
Menlo; font-size: large;" class="">rdb </span><span style="font-family: Menlo; \
font-size: large;" class="">=</span><span style="font-family: Menlo; font-size: \
large;" class=""> </span><span style="font-family: Menlo; font-size: large;" \
class="">redis</span><span style="font-family: Menlo; font-size: large;" \
class="">.</span><span style="font-family: Menlo; font-size: large;" \
class="">StrictRedis</span><span style="font-family: Menlo; font-size: large;" \
class="">(</span><span style="font-family: Menlo; font-size: large;" \
class="">host</span><span style="font-family: Menlo; font-size: large;" \
class="">=</span><span style="font-family: Menlo; font-size: large;" \
class="">'localhost'</span><span style="font-family: Menlo; font-size: large;" \
class="">,</span><span style="font-family: Menlo; font-size: large;" \
class=""> </span><span style="font-family: Menlo; font-size: large;" \
class="">port</span><span style="font-family: Menlo; font-size: large;" \
class="">=</span><span style="font-family: Menlo; font-size: large;" \
class="">6379</span><span style="font-family: Menlo; font-size: large;" \
class="">,</span><span style="font-family: Menlo; font-size: large;" \
class=""> </span><span style="font-family: Menlo; font-size: large;" \
class="">db</span><span style="font-family: Menlo; font-size: large;" \
class="">=</span><span style="font-family: Menlo; font-size: large;" \
class="">0)</span></div><div class=""><br class=""></div><div class=""><span \
style="font-family: Menlo; font-size: large;" class="">temperature \
= </span><span style="font-family: Menlo; font-size: 14px;" \
class="">rdb.get('/28.67C6697351FF')</span></div><div class=""><br \
class=""></div><div class=""><br class=""></div><div class="">On my rpi2 the rdb.get \
call takes less than 1ms, but have a look on <a \
href="http://redis.io/topics/benchmarks" \
class="">http://redis.io/topics/benchmarks</a> to have an idea on how fast redis \
can be. Installation is painless, python-side you just "pip install pyownet redis", \
while redis server is for sure already packaged in CentOS.</div><div class=""><br \
class=""></div><div class="">Bye</div><div class=""><br class=""></div><div \
class="">Stefano</div><div class=""><br class=""><div class=""><blockquote \
type="cite" class=""><div class="">On 23 Sep 2015, at 14:35, Alex R. Gibbs <<a \
href="mailto:agibbs@lpl.arizona.edu" class="">agibbs@lpl.arizona.edu</a>> \
wrote:</div><br class="Apple-interchange-newline"><div class="">Hello,<br \
class=""><br class="">We have dozens of DS18B20 temperature sensors at 3 telescopes. \
I'd like to poll <br class="">them and cache the results so that other programs \
can retrieve the latest <br class="">temperatures with less than 50ms delay. It \
doesn't matter if the values are a <br class="">few minutes old; just that clients \
don't have to wait for a response. I am <br class="">using owserver 3.1p0 on \
CentOS 6.5 64bit with a LinkHub-E.<br class=""><br class="">Is it possible to \
configure owserver to continuously poll devices on it's own? <br class="">The \
owserver cache works but of course once a value in the cache expires the <br \
class="">next request for it forces a read of the device, making the client wait \
about <br class="">800ms. If owserver can be set to poll devices then I can set \
the expire time <br class="">such that this usually never happens.<br class=""><br \
class="">I tried having a background program poll the uncached values in owserver \
more <br class="">often than the volatile timeout so that the cached values are never \
considered <br class="">expired. However, a client still has to wait for a \
cached value if it is <br class="">currently being read out due to an uncached \
request, even if the cached value <br class="">isn't expired. If that was \
changed I think this would work.<br class=""><br class="">Any suggestions on how to \
setup polling so that a client can always get a cached <br class="">value \
immediately? I have a feeling I'll need to either modify owserver or <br \
class="">write my own server wrapper.<br class=""><br class="">Thanks,<br \
class="">Alex<br class=""><br class="">-- <br class="">Alex R. Gibbs - <a \
href="mailto:agibbs@lpl.arizona.edu" class="">agibbs@lpl.arizona.edu</a><br \
class="">Principal Engineer - Catalina Sky Survey<br class="">Lunar & Planetary \
Lab - University of Arizona<br class="">--<br class=""><br \
class="">------------------------------------------------------------------------------<br \
class="">Monitor Your Dynamic Infrastructure at Any Scale With Datadog!<br \
class="">Get real-time metrics from all of your servers, apps and tools<br \
class="">in one place.<br class="">SourceForge users - Click here to start your Free \
Trial of Datadog now!<br class=""><a \
href="http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140" \
class="">http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140</a><br \
class="">_______________________________________________<br class="">Owfs-developers \
mailing list<br class=""><a href="mailto:Owfs-developers@lists.sourceforge.net" \
class="">Owfs-developers@lists.sourceforge.net</a><br \
class="">https://lists.sourceforge.net/lists/listinfo/owfs-developers<br \
class=""></div></blockquote></div><br class=""></div></div></body></html>
------------------------------------------------------------------------------
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140
_______________________________________________
Owfs-developers mailing list
Owfs-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/owfs-developers
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic