[prev in list] [next in list] [prev in thread] [next in thread] 

List:       freebsd-stable
Subject:    Re: zfs listing and CPU
From:       Tenzin Lhakhang <tenzin.lhakhang () gmail ! com>
Date:       2017-08-14 11:15:07
Message-ID: CALcn87z8=FVey=KNARtG5PcbE6dz-Z+Pj+tHuDoVx7MsztKYEA () mail ! gmail ! com
[Download RAW message or body]

Interesting, even the illumos man page doesn't have the -c defined.  I
remember reading about it at one point.

I can only get `zfs get -Hrpc all` working, even though -c is not
documented.  So, I retract my statement on the zfs list with cached.

# zfs get -*Hpc* -o property,value name,used,avail zones
name    zones
used    1219313598464
available       761203195904

# I use the above way to create easily parseable zfs_dataset objects.

As for needing immediate/latest version of zfs list on entity creation --
you ideally want a way to do cache invalidation or use some sort of
message/event driven system between the client that creates the entity and
the server.  You can perform the cache invalidation by monitoring the
'creation' events that come to your API, or by tailing zpool history [pool]
for changes.

- I recently read this article and they show a nice way to decouple the
storage events and the api, architecture is not entirely new but can be
adapted to your case.
- https://medium.freecodecamp.org/million-websockets-and-go-cc58418460bb

Thanks,
Tenzin

On Sun, Aug 13, 2017 at 2:25 PM, Eugene M. Zheganin <emz@norma.perm.ru>
wrote:

> On 13.08.2017 16:13, Tenzin Lhakhang wrote:
>
>> You may want to have an async zfs-get program/script that regularly does
>> a zfs get -Ho and stores then in a local cache (redis or your own program)
>> at a set interval and then the api can hit the cache instead of directly
>> running get or list.
>>
> I cannot because the cache will become stale on first new entity creation,
> which happens all the time.
>
> - Some silly person will try to benchmark your zfs web-API and overload
>> your server with zfs processes.
>> - Example: let me run [ ab -c 10 -n 10000 http://yourserver/zfs-api/list
>> ]
>> -- Let me run 10 concurrent connection with a total of 10k requests to
>> your api (it's a simple one liner -- people will be tempted to benchmark
>> like this).
>>
>> Example:
>> https://github.com/tlhakhan/ideal-potato/blob/master/zdux/ro
>> uters/zfs/service.js#L9
>> - This is a JS example, but you can easily script it or another language
>> (golang) for cache separation and another program for the API.
>>
>> Also, zfs does have a -c property to get cached values -- these values
>> are stored in an internal zfs process cache. The -c doesn't help if you
>> have 1000(0)s of filesystems, a single list can still take minutes.
>> Sending the list is also several megabytes.
>>
> Doesn't have on FreeBSD.
>
>
>
> Thanks.
> Eugene.
> _______________________________________________
> freebsd-fs@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
>
_______________________________________________
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
[prev in list] [next in list] [prev in thread] [next in thread]