[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ceph-users
Subject:    [ceph-users] get error when use prometheus plugin of ceph-mgr
From:       jspray () redhat ! com (John Spray)
Date:       2017-08-30 16:55:38
Message-ID: CALe9h7frDpRWZTXFZRO6ox4Q9SLCTVVrn4RGw=2NRL8TmdgcNw () mail ! gmail ! com
[Download RAW message or body]

On Wed, Aug 30, 2017 at 5:54 PM, shawn tim <tontinme at gmail.com> wrote:
> Hi, John,
> Do you mean this error
> text format parsing error in line 8: invalid metric name in comment

Yes, iirc that's the one you get from the plus/minus signs in MDS metric names.

> I install ceph from rpm, but I will try to patch it.
> many thanks
>
>
> when using prometheus 1.7.1 instead of 2.0.0-beta, the "no token found
> error" disappeared.
> Maybe prometheus 2.0.0-beta still has bugs about it.

Huh, weird, I think I was only ever using 2.0.  No idea then!

John

>
> John Spray <jspray at redhat.com>?2017?8?30??? ??7:34???
>>
>> On Wed, Aug 30, 2017 at 3:47 AM, shawn tim <tontinme at gmail.com> wrote:
>> >
>> >
>> > Hello,
>> > I just want to try prometheus plugin of ceph-mgr.
>> > Following this doc(http://docs.ceph.com/docs/master/mgr/prometheus/). I
>> > get
>> > output like
>> >
>> > [root at ceph01 ~]# curl localhost:9283/metrics/ | head
>> >   % Total    % Received % Xferd  Average Speed   Time    Time     Time
>> > Current
>> >                                  Dload  Upload   Total   Spent    Left
>> > Speed
>> >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--
>> > 0
>> > # HELP mds_mem_dir_ Directories closed
>> > # TYPE mds_mem_dir_ counter
>> > mds_mem_dir_{daemon="mds.ceph01"} 0.0
>> > # HELP throttle_msgr_dispatch_throttler_radosclient_max Max value for
>> > throttle
>> > # TYPE throttle_msgr_dispatch_throttler_radosclient_max gauge
>> > throttle_msgr_dispatch_throttler_radosclient_max{daemon="rgw.ceph01"}
>> > 104857600.0
>> > # HELP mds_mem_dir+ Directories opened
>> > # TYPE mds_mem_dir+ counter
>> > mds_mem_dir+{daemon="mds.ceph01"} 12.0
>> > .
>>
>> I notice you have some MDS metrics in there, which cause some invalid
>> output -- can you try this fix?
>> https://github.com/ceph/ceph/pull/17318
>>
>> The "no token found stuff" is not familiar to me: when testing the
>> prometheus plugin I've always just been using it without any
>> authentication.
>>
>> John
>>
>> > It seems everythins was  OK. But when prometheus connect it, I got
>> > errors:
>> >
>> > WARN[0002] append failed                                 err="no token
>> > found" source="scrape.go:648" target="{__address__="10.10.0.125:9283",
>> > __metrics_path__="/metric
>> > s", __scheme__="http", group="production", instance="10.10.0.125:9283",
>> > job="ceph"}"
>> > WARN[0007] append failed                                 err="no token
>> > found" source="scrape.go:648" target="{__address__="10.10.0.125:9283",
>> > __metrics_path__="/metric
>> > s", __scheme__="http", group="production", instance="10.10.0.125:9283",
>> > job="ceph"}"
>> > WARN[0012] append failed                                 err="no token
>> > found" source="scrape.go:648" target="{__address__="10.10.0.125:9283",
>> > __metrics_path__="/metric
>> > s", __scheme__="http", group="production", instance="10.10.0.125:9283",
>> > job="ceph"}"
>> > ^CWARN[0014] Received SIGTERM, exiting gracefully...
>> > source="main.go:340"
>> >
>> > My prometheus config yml is
>> >
>> >   - job_name: 'ceph mgr'
>> >     scrape_interval: 5s
>> >     # metrics_path: /metrics/
>> >     # uncomment this still get error (Get
>> > http://10.10.0.125:9283/metrics:
>> > net/http: invalid header field value "Bearer ..." for key Authorization
>> > )
>> >     # bearer_token_file: /etc/ceph/ceph.mgr.ceph01.keyring
>> >     # bearer_token: AQD6359ZCMK3DxAAyJYErZayMX/CRZyiyk/UGg==
>> >
>> >     static_configs:
>> >      - targets: ['10.10.0.125:9283']
>> >        labels:
>> >          group: 'production'
>> >
>> >
>> > My ceph version is
>> >
>> > [root at ceph01 ~]# ceph --version
>> > ceph version 12.1.4 (a5f84b37668fc8e03165aaf5cbb380c78e4deba4) luminous
>> > (rc)
>> >
>> > My prometheus version is
>> >
>> > [root at prom01 prometheus-2.0.0-beta.2.linux-amd64]# ./prometheus
>> > --version
>> > prometheus, version 2.0.0-beta.2 (branch: HEAD, revision:
>> > a52f082939a566d5269671e98be06fc6bdf61d09)
>> >   build user:       root at 41a0740ea598
>> >   build date:       20170818-08:16:50
>> >   go version:       go1.8.3
>> >
>> > Is there any steps I missed
>> >
>> > Thanks for help !
>> >
>> >
>> >
>> > _______________________________________________
>> > ceph-users mailing list
>> > ceph-users at lists.ceph.com
>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> >

[prev in list] [next in list] [prev in thread] [next in thread]