[prev in list] [next in list] [prev in thread] [next in thread] 

List:       dns-operations
Subject:    [dns-operations] 'dnstap' (Re: Prevalence of query/response logging?)
From:       rdobbins () arbor ! net (Roland Dobbins)
Date:       2014-07-08 3:15:07
Message-ID: 952BF723-FC94-4A15-989C-EE88F018BABD () arbor ! net
[Download RAW message or body]


On Jul 8, 2014, at 9:33 AM, Robert Edmonds <edmonds at mycre.ws> wrote:

> It looks like maintaining the #2/#3 "transport/encoding" split with IPFIX is \
> impossible

It's an abstract goal which isn't an operational consideration, IMHO.

> If IPFIX is well-suited for applications other than representing IP flows, it is \
> awfully hard to tell from the outside without plowing through a ton of \
> specifications.  This is itself a downside; we have to convince not just ourselves, \
> but DNS software vendors to import this code and DNS software users that they might \
> want to use this code.

It's actually pretty well-known in the operational community that IPFIX is extensive \
and well-suited to this sort of task.

In terms of collection/analysis tools, putting this sort of data into a standardized \
format would go a long ways towards ensuring that tools are available which can use \
it, and offer easy combinatorial analysis with data such as more classical flow \
records from routers/layer-3 switches, policy and even data from things like \
firewalls and load-balancers, etc.  Adoption by DNS vendors/coders is only half the \
battle, and since this is Something New to DNS vendors/coders, but not to \
vendors/coders of telemetry analysis systems, it's a consideration.

> to represent dnstap payloads with maximally sized DNS messages in a single frame, \
> which I wanted to make a hard requirement for dnstap.

Is this intended to help with performance, or . . . ?

> (Possibly IPFIX can split payloads across multiple messages,

It can.

> What if you want to send payloads over an AF_UNIX socket, or via an HTTP(S) \
> GET/POST, WebSockets connection, some new technology that hasn't been invented yet, \
> etc.?

These are corner-cases, IMHO.

> I say "appears" above because my next complaint is that there are too many \
> specifications documents for IPFIX.

One can say the same of DNS.

;>

All it would've taken to get the relevant information was asking folks plugged into \
the flow telemetry community about these various issues, rather than wading through \
lots of docs.

> Also, I found the following blog post rather interesting:

Without wasting a lot of space here dissecting it, there's a great deal he says which \
I don't agree with, FWIW.v

> It just doesn't look like a good fit for what we want to make possible, and there \
> are a lot of general purpose technologies out there that I would consider first \
> before considering IPFIX for a particular application.


The barrier to adoption, as I see it, is that this is now a one-off telemetry format, \
which generally (not always) tends to lead to low-to-no adoption.  I fear that's the \
case here (hopefully, I'm wrong!).

I personally think going the one-off route (pardon the pun, heh), no matter the \
perceived benefits, is self-defeating; I'll look into what's necessary to somehow \
transcode this into IPFIX, but it would've been much simpler (and much more likely to \
be widely adopted, IMHO) if IPFIX had simply been adopted in the first place.

I'm really grateful for the detailed explanation and insight into your thinking; even \
though I don't agree with the goal prioritization (a primary goal of any form of \
telemetry export should be relatively easy compatibility with existing \
collection/analysis systems and the use of formats with which there's likely going to \
be some degree of familiarity and experience with same, in order to maximize the \
chances of broad adoption) - it's educational and much appreciated!

----------------------------------------------------------------------
Roland Dobbins <rdobbins at arbor.net> // <http://www.arbornetworks.com>

                   Equo ne credite, Teucri.

    		   	  -- Laoco?n


[prev in list] [next in list] [prev in thread] [next in thread]