[prev in list] [next in list] [prev in thread] [next in thread]
List: cassandra-user
Subject: Re: Logging
From: oleg yusim <olegyusim () gmail ! com>
Date: 2016-01-27 16:12:49
Message-ID: CAKd4e_HB_praHjzuhbRpDjK=zBO7xgTmxtTGrE8qc46-Ni_58w () mail ! gmail ! com
[Download RAW message or body]
Sam, Paulo,
One more question on logging. Can I add IP and hostname to the log message?
If it is possible, can you give me example of how I would need to
change <pattern>%-5level %date{HH:mm:ss,SSS} %msg%n</pattern> to add this
information?
Thanks,
Oleg
On Tue, Jan 26, 2016 at 4:42 PM, oleg yusim <olegyusim@gmail.com> wrote:
> Sam, Paulo,
>
> Thank you very much for explanations and references.
>
> Oleg
>
> On Mon, Jan 25, 2016 at 10:08 AM, Sam Tunnicliffe <sam@beobal.com> wrote:
>
>> Paulo is correct in saying that C* doesn't have a direct equivalent of
>> SecurityContextHolder. Authenticated principal info is retrievable from the
>> QueryState during query execution but a) this isn't available to every
>> method in the call chain and b) its scope is limited to the coordinator for
>> the request. That is, it isn't serialized and included in the read/mutation
>> messages which the coordinator distributes to the replicas. So you could
>> produce a level of audit trail by providing a custom QueryHandler (See
>> CASSANDRA-6659) that logs each statement along with the principal. But if
>> the goal is indeed that "every log message in file should start with
>> username of the user, who initiated this action", it's isn't really
>> feasible right now
>>
>> On Mon, Jan 25, 2016 at 3:52 PM, Paulo Motta <pauloricardomg@gmail.com>
>> wrote:
>>
>>> That would work, but afaik Cassandra doesn't have an equivalent of
>>> RequestContextHolder/SecurityContextHolder that is able to retrieve the
>>> user/session of a given thread/request (maybe I'm wrong as I'm no auth
>>> expert), so if these don't exist we'd need to add equivalent to those or do
>>> it via MDC (set the context when request arrives, propagate to down stream
>>> threads, cleanup), which can become quite messy as shown in CASSANDRA-7276.
>>>
>>> For CQL statements perhaps the query tracing infrastructure could be
>>> reused to provide that info, but that would require further investigation.
>>> See CASSANDRA-1123 for more details on that.
>>>
>>> 2016-01-25 12:30 GMT-03:00 oleg yusim <olegyusim@gmail.com>:
>>>
>>>> Paulo,
>>>>
>>>> Ideally - all the actions (security purposes, preserving completness of
>>>> the audit trail). How about this approach:
>>>> http://www.codelord.net/2010/08/27/logging-with-a-context-users-in-logback-and-spring-security/ ?
>>>> Would that work? Or you would rather suggest to go MDC way?
>>>>
>>>> Thanks,
>>>>
>>>> Oleg
>>>>
>>>> On Mon, Jan 25, 2016 at 9:23 AM, Paulo Motta <pauloricardomg@gmail.com>
>>>> wrote:
>>>>
>>>>> What kind of actions? nodetool/system actions or cql statements?
>>>>>
>>>>> You could probably achieve identity-based logging with logback Mapped
>>>>> Diagnostic Context (MDC - logback.qos.ch/manual/mdc.html), but you'd
>>>>> need to patch your own Cassandra jars in many locations to provide that
>>>>> information to the logging context, so not exactly a trivial thing to do.
>>>>> We tried using that to print ks/cf names on log messages but it became a
>>>>> bit messy due to the SEDA architecture as you need to patch executors to
>>>>> inherit identifiers from parent threads and cleanup afterwards. See
>>>>> CASSANDRA-7276 for more background.
>>>>>
>>>>> 2016-01-25 12:09 GMT-03:00 oleg yusim <olegyusim@gmail.com>:
>>>>>
>>>>>> I want to try to re-phrase my question here... what I'm trying to
>>>>>> achieve is identity-based logging. I.e. every log message in file should
>>>>>> start with username of the user, who initiated this action. Would that be
>>>>>> possible to achieve? If so, can you give me a brief example?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Oleg
>>>>>>
>>>>>> On Thu, Jan 21, 2016 at 2:57 PM, oleg yusim <olegyusim@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Joel,
>>>>>>>
>>>>>>> Thanks for reference. What I'm trying to achieve, is to add the name
>>>>>>> of the user, who initiated logged action. I tried c{5}, but what I see is
>>>>>>> that;
>>>>>>>
>>>>>>> TRACE [GossipTasks:1] c{5} 2016-01-21 20:51:17,619 Gossiper.java:700
>>>>>>> - Performing status check ...
>>>>>>>
>>>>>>> I think, I'm missing something here. Any suggestions?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Oleg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 21, 2016 at 1:30 PM, Joel Knighton <
>>>>>>> joel.knighton@datastax.com> wrote:
>>>>>>>
>>>>>>>> Cassandra uses logback as its backend for logging.
>>>>>>>>
>>>>>>>> You can find information about configuring logging in Cassandra by
>>>>>>>> searching for "Configuring logging" on docs.datastax.com and
>>>>>>>> selecting the documentation for your version.
>>>>>>>>
>>>>>>>> The documentation for PatternLayouts (the pattern string about
>>>>>>>> which you're asking) in logback is available in the logback manual under
>>>>>>>> the section for Conversion Words
>>>>>>>> http://logback.qos.ch/manual/layouts.html#conversionWord
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 21, 2016 at 1:21 PM, oleg yusim <olegyusim@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Greetings,
>>>>>>>>>
>>>>>>>>> Guys, can you, please, point me to documentation on how to
>>>>>>>>> configure format of logs? I want make it clear, I'm talking about
>>>>>>>>> formatting i.e. this:
>>>>>>>>>
>>>>>>>>> <pattern>%-5level %date{HH:mm:ss,SSS} %msg%n</pattern>
>>>>>>>>>
>>>>>>>>> What if I want to add another parameters into this string? Is
>>>>>>>>> there a list of available parameters here and syntax?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Oleg
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> <http://www.datastax.com/>
>>>>>>>>
>>>>>>>> Joel Knighton
>>>>>>>> Cassandra Developer | joel.knighton@datastax.com
>>>>>>>>
>>>>>>>> <https://www.linkedin.com/company/datastax>
>>>>>>>> <https://www.facebook.com/datastax> <https://twitter.com/datastax>
>>>>>>>> <https://plus.google.com/+Datastax/about>
>>>>>>>> <http://feeds.feedburner.com/datastax>
>>>>>>>> <https://github.com/datastax/>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
[Attachment #3 (text/html)]
<div dir="ltr">Sam, Paulo,<div><br></div><div>One more question on logging. Can I add \
IP and hostname to the log message? If it is possible, can you give me example of how \
I would need to change <pattern>%-5level %date{HH:mm:ss,SSS} \
%msg%n</pattern> to add this \
information?</div><div><br></div><div>Thanks,</div><div><br></div><div>Oleg</div></div><div \
class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 26, 2016 at 4:42 PM, \
oleg yusim <span dir="ltr"><<a href="mailto:olegyusim@gmail.com" \
target="_blank">olegyusim@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Sam, Paulo,<div><br></div><div>Thank you very \
much for explanations and references.</div><span class="HOEnZb"><font \
color="#888888"><div><br></div><div>Oleg</div></font></span></div><div \
class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div \
class="gmail_quote">On Mon, Jan 25, 2016 at 10:08 AM, Sam Tunnicliffe <span \
dir="ltr"><<a href="mailto:sam@beobal.com" \
target="_blank">sam@beobal.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Paulo is correct in saying that C* doesn't \
have a direct equivalent of SecurityContextHolder. Authenticated principal info is \
retrievable from the QueryState during query execution but a) this isn't \
available to every method in the call chain and b) its scope is limited to the \
coordinator for the request. That is, it isn't serialized and included in the \
read/mutation messages which the coordinator distributes to the replicas. So you \
could produce a level of audit trail by providing a custom QueryHandler (See \
CASSANDRA-6659) that logs each statement along with the principal. But if the goal is \
indeed that "every log message in file should start with username of the user, \
who initiated this action", it's isn't really feasible right \
now</div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan \
25, 2016 at 3:52 PM, Paulo Motta <span dir="ltr"><<a \
href="mailto:pauloricardomg@gmail.com" \
target="_blank">pauloricardomg@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>That would work, but afaik Cassandra \
doesn't have an equivalent of RequestContextHolder/SecurityContextHolder that is \
able to retrieve the user/session of a given thread/request (maybe I'm wrong as \
I'm no auth expert), so if these don't exist we'd need to add equivalent \
to those or do it via MDC (set the context when request arrives, propagate to down \
stream threads, cleanup), which can become quite messy as shown in \
CASSANDRA-7276.<br><br></div>For CQL statements perhaps the query tracing \
infrastructure could be reused to provide that info, but that would require further \
investigation. See CASSANDRA-1123 for more details on that.<br></div><div><div><div \
class="gmail_extra"><br><div class="gmail_quote">2016-01-25 12:30 GMT-03:00 oleg \
yusim <span dir="ltr"><<a href="mailto:olegyusim@gmail.com" \
target="_blank">olegyusim@gmail.com</a>></span>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Paulo,<div><br></div><div>Ideally - all the \
actions (security purposes, preserving completness of the audit trail). How about \
this approach: <a href="http://www.codelord.net/2010/08/27/logging-with-a-context-users-in-logback-and-spring-security/" \
style="font-size:12.8px" \
target="_blank">http://www.codelord.net/2010/08/27/logging-with-a-context-users-in-logback-and-spring-security/</a> \
? Would that work? Or you would rather suggest to go MDC \
way?</div><div><br></div><div>Thanks,</div><div><br></div><div>Oleg</div></div><div><div><div \
class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 25, 2016 at 9:23 AM, \
Paulo Motta <span dir="ltr"><<a href="mailto:pauloricardomg@gmail.com" \
target="_blank">pauloricardomg@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">What kind of actions? nodetool/system actions \
or cql statements?<br><div><br>You could probably achieve identity-based logging with \
logback Mapped Diagnostic Context (MDC - <a \
href="http://logback.qos.ch/manual/mdc.html" \
target="_blank">logback.qos.ch/manual/mdc.html</a>), but you'd need to patch your \
own Cassandra jars in many locations to provide that information to the logging \
context, so not exactly a trivial thing to do. We tried using that to print ks/cf \
names on log messages but it became a bit messy due to the SEDA architecture as you \
need to patch executors to inherit identifiers from parent threads and cleanup \
afterwards. See CASSANDRA-7276 for more background. </div></div><div><div><div \
class="gmail_extra"><br><div class="gmail_quote">2016-01-25 12:09 GMT-03:00 oleg \
yusim <span dir="ltr"><<a href="mailto:olegyusim@gmail.com" \
target="_blank">olegyusim@gmail.com</a>></span>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">I want to try to re-phrase my question here... \
what I'm trying to achieve is identity-based logging. I.e. every log message in \
file should start with username of the user, who initiated this action. Would that be \
possible to achieve? If so, can you give me a brief \
example?<div><br></div><div>Thanks,</div><div><br></div><div>Oleg</div></div><div><div><div \
class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 21, 2016 at 2:57 PM, \
oleg yusim <span dir="ltr"><<a href="mailto:olegyusim@gmail.com" \
target="_blank">olegyusim@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Joel,<div><br></div><div>Thanks for reference. \
What I'm trying to achieve, is to add the name of the user, who initiated logged \
action. I tried c{5}, but what I see is that;</div><div><br></div><div><div>TRACE \
[GossipTasks:1] c{5} 2016-01-21 20:51:17,619 Gossiper.java:700 - Performing status \
check ...</div></div><div><br></div><div>I think, I'm missing something here. Any \
suggestions?</div><div><br></div><div>Thanks,</div><div><br></div><div>Oleg</div><div><br></div><div><br></div></div><div><div><div \
class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 21, 2016 at 1:30 PM, \
Joel Knighton <span dir="ltr"><<a href="mailto:joel.knighton@datastax.com" \
target="_blank">joel.knighton@datastax.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Cassandra uses logback as its backend for \
logging.<div><br></div><div>You can find information about configuring logging in \
Cassandra by searching for "Configuring logging" on <a \
href="http://docs.datastax.com" target="_blank">docs.datastax.com</a> and selecting \
the documentation for your version.</div><div><br></div><div>The documentation for \
PatternLayouts (the pattern string about which you're asking) in logback is \
available in the logback manual under the section for Conversion Words <a \
href="http://logback.qos.ch/manual/layouts.html#conversionWord" \
target="_blank">http://logback.qos.ch/manual/layouts.html#conversionWord</a></div><div><br></div></div><div \
class="gmail_extra"><div><div><br><div class="gmail_quote">On Thu, Jan 21, 2016 at \
1:21 PM, oleg yusim <span dir="ltr"><<a href="mailto:olegyusim@gmail.com" \
target="_blank">olegyusim@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Greetings,<div><br></div><div>Guys, can you, \
please, point me to documentation on how to configure format of logs? I want make it \
clear, I'm talking about formatting i.e. \
this:</div><div><br></div><div><div><pattern>%-5level %date{HH:mm:ss,SSS} \
%msg%n</pattern></div></div><div><br></div><div>What if I want to add another \
parameters into this string? Is there a list of available parameters here and \
syntax?</div><div><br></div><div>Thanks,</div><div><br></div><div>Oleg</div><div><br></div></div>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span><font \
color="#888888">-- <br><div><div dir="ltr"><div><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><span><p \
dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.datastax.com/" style="text-decoration:none" target="_blank"><span \
style="font-size:15px;font-family:Calibri;color:rgb(17,85,204);text-decoration:underli \
ne;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><img \
src="https://lh6.googleusercontent.com/zujWCxiKGPoNTpU9rYwVo1GIvn0MXxFMiHJMgB_UZdESbDD \
0cn0gKnUw4UcYkyjI3uEbnxwVcjrGuvnXaTR4FoTt3F6u2whn_5qLRoExWTqSMYpv-9OtXIysw9rN5mXz564v5oY" \
style="border:none" height="31px;" width="150px;"></span></a></p><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:16px;font-family:Calibri;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Joel \
Knighton</span><span \
style="font-size:16px;font-family:Calibri;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><br></span><span \
style="font-size:16px;font-family:Calibri;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Cassandra \
Developer </span><span \
style="font-size:16px;font-family:Calibri;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">| \
<a href="mailto:joel.knighton@datastax.com" \
target="_blank">joel.knighton@datastax.com</a></span></p><br><a \
href="https://www.linkedin.com/company/datastax" style="text-decoration:none" \
target="_blank"><span \
style="font-size:12px;font-family:Arial;color:rgb(17,85,204);text-decoration:underline;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/yF5_lV4h7D2UxEplm05DKDejwQH2rN54kQRIAMwYl3cOFAW \
VwodcGTBiCsOkOVBSKD644UsZpdwaHZTpN4TH56tTGAjI6X0QAPdSy3BHUHKM_L6-38Z4iZnaju6iTXyRNFFJYYs" \
style="border:none" height="25px;" width="25px;"></span></a><span \
style="font-size:15px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> \
</span><a href="https://www.facebook.com/datastax" style="text-decoration:none" \
target="_blank"><span \
style="font-size:12px;font-family:Arial;color:rgb(17,85,204);text-decoration:underline;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/rfmg_Y2DLoU0blnPfsgAXGhlvS5qFs_Env_XvgglzoC8oYj \
pMrXkeeGtCOs1n4O-c4hWt7sB6JP_bVoshrvSDRs9d6t2h-94rPgih78BO7eizEHIkojHoFjFlbp9ev6VXowy9Uc" \
style="border:none" height="25px;" width="25px;"></span></a><span \
style="font-size:15px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> \
</span><a href="https://twitter.com/datastax" style="text-decoration:none" \
target="_blank"><span \
style="font-size:12px;font-family:Arial;color:rgb(17,85,204);text-decoration:underline;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh5.googleusercontent.com/f51dqYsxdPnH8vFbRcv01-CfYgwLWMRy6h0duHVx20vZdVG \
ofchf9EwXO-QbK2iYu4B_XK39s-CUkTALWWRKAT5h5muJlJDE1G9aP0AS6_CHEehFXHal9QhmRqxEy0APsne2jRY" \
style="border:none" height="25px;" width="25px;"></span></a><span \
style="font-size:15px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> \
</span><a href="https://plus.google.com/+Datastax/about" style="text-decoration:none" \
target="_blank"><span \
style="font-size:12px;font-family:Arial;color:rgb(17,85,204);text-decoration:underline;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh4.googleusercontent.com/1nF3jCTQgssSgkR8t8YGl29Xh0m4j6cjnwK-f8MYw3hs0nt \
HRPekqX7nOgsUTC8pe1skAHyqQJk58mYl1O02CYcT9Dm_QF_bITwZrperb5ufpSCNLVAHRnzWldryaRDe5Q3AsBQ" \
style="border:none" height="25px;" width="25px;"></span></a><span \
style="font-size:15px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> \
</span><a href="http://feeds.feedburner.com/datastax" style="text-decoration:none" \
target="_blank"><span \
style="font-size:12px;font-family:Arial;color:rgb(17,85,204);text-decoration:underline;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh5.googleusercontent.com/vkQxsPuzc0ZSSFU5mxnSHN7WaQN2GKK3hhORrINgHpxl6QB \
0eReJ3RjSNvKgFTjB85qTrvxHm125957h_vWszsE5GF2sjBJa8_kbEdN8tRZfLbCSZ2JbnrIpNH1r-PHafmFhPEE" \
style="border:none" height="25px;" width="25px;"></span></a><span \
style="font-size:15px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> \
</span><span style="text-decoration:underline;font-size:12px;font-family:Arial;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap"><a \
href="https://github.com/datastax/" style="text-decoration:none" target="_blank"><img \
src="https://lh4.googleusercontent.com/70iAX8N2qH7_bxPX42x9HswoVfGFFSl4ueEXoYyp4APr3S1 \
O1chNJSVTCVbY0Ta1-IjnqtirbcoFI9SGxkFBncliESx4uz8XHsQ5SsG82WOFDnMMkSKcpe10rr3Lbv9jkIYgQsU" \
style="border:none" height="25px;" \
width="25px;"></a></span></span></div></div></div></div></div></div></div></div></div></div></div></div>
</font></span></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic