[prev in list] [next in list] [prev in thread] [next in thread]
List: avro-user
Subject: Re: C impl, getting a schema back as json C string
From: Matt Massie <matt () cloudera ! com>
Date: 2010-03-16 22:50:55
Message-ID: 35538fbe1003161550r30edb0c3vb1f0ab815cb1faab () mail ! gmail ! com
[Download RAW message or body]
That's true that you'll have to hold on to the buffer yourself. Currently
the memory writer wraps a fixed buffer. Alternatively, we could have a
second type of memory writer that dynamically grows in size as needed (to
say some maximum size).
Implementation details aside, what would be the ideal API/process to do
this?
-Matt
On Tue, Mar 16, 2010 at 3:02 PM, Jeff Hodges <jhodges@twitter.com> wrote:
> Yeah, I got that part. The problem I'm having is getting the data back
> out of the writer. _tell just gives you the size of buffer sure, and I
> would still have to hold on to buffer myself.
> --
> Jeff
>
> On Tue, Mar 16, 2010 at 2:35 PM, Matt Massie <matt@cloudera.com> wrote:
> > Jeff-
> >
> > If you look in avro.h, you'll see the following function...
> >
> > int avro_schema_to_json(avro_schema_t schema, avro_writer_t out);
> >
> > .... which write a schema to an avro_writer_t. This allows you to write
> the
> > schema to memory or a file... create the writer using...
> >
> > avro_writer_t avro_writer_file(FILE * fp);
> > avro_writer_t avro_writer_memory(const char *buf, int64_t len);
> >
> > I just realized that
> >
> > int64_t avro_writer_tell(avro_writer_t writer)
> >
> > is not in avro.h and it should be. It will tell you the size of the data
> > written to the writer.
> >
> > Hope this help. Keep the questions coming.
> >
> > -Matt
> >
> >
> >
> >
> >
> >
> >
> > On Sun, Mar 14, 2010 at 10:14 PM, Jeff Hodges <jhodges@twitter.com>
> wrote:
> >> (Note, that I've passed this in memory avro_writer_t to
> >> avro_schema_to_json and I'm just trying to get the string inside of it
> >> out.)
> >> --
> >> Jeff
> >>
> >> On Sun, Mar 14, 2010 at 12:39 AM, Jeff Hodges <jhodges@twitter.com>
> wrote:
> >>> I can't figure out the API for turning an avro_schema_t back into an
> >>> in-memory string of json data.
> >>>
> >>> It seems I'm supposed to get it out of an avro_writer_t that I've
> >>> created with avro_writer_memory() but I'm a little lost once we start
> >>> having to use avro_writer_to_memory() to get back the original
> >>> avro_memory_writer_t.
> >>>
> >>> Sample code?
> >>> --
> >>> Jeff
> >>>
> >>
>
[Attachment #3 (text/html)]
That's true that you'll have to hold on to the buffer yourself. Currently \
the memory writer wraps a fixed buffer. Alternatively, we could have a second type \
of memory writer that dynamically grows in size as needed (to say some maximum \
size).<div>
<br></div><div>Implementation details aside, what would be the ideal API/process to \
do this?</div><div><br></div><div>-Matt</div><div><br><br><div class="gmail_quote">On \
Tue, Mar 16, 2010 at 3:02 PM, Jeff Hodges <span dir="ltr"><<a \
href="mailto:jhodges@twitter.com">jhodges@twitter.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex;">Yeah, I got that part. The problem I'm having is getting \
the data back<br> out of the writer. _tell just gives you the size of buffer sure, \
and I<br> would still have to hold on to buffer myself.<br>
--<br>
<font color="#888888">Jeff<br>
</font><div><div></div><div class="h5"><br>
On Tue, Mar 16, 2010 at 2:35 PM, Matt Massie <<a \
href="mailto:matt@cloudera.com">matt@cloudera.com</a>> wrote:<br> > Jeff-<br>
><br>
> If you look in avro.h, you'll see the following function...<br>
><br>
> int avro_schema_to_json(avro_schema_t schema, avro_writer_t out);<br>
><br>
> .... which write a schema to an avro_writer_t. This allows you to write the<br>
> schema to memory or a file... create the writer using...<br>
><br>
> avro_writer_t avro_writer_file(FILE * fp);<br>
> avro_writer_t avro_writer_memory(const char *buf, int64_t len);<br>
><br>
> I just realized that<br>
><br>
> int64_t avro_writer_tell(avro_writer_t writer)<br>
><br>
> is not in avro.h and it should be. It will tell you the size of the data<br>
> written to the writer.<br>
><br>
> Hope this help. Keep the questions coming.<br>
><br>
> -Matt<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> On Sun, Mar 14, 2010 at 10:14 PM, Jeff Hodges <<a \
href="mailto:jhodges@twitter.com">jhodges@twitter.com</a>> wrote:<br> >> \
(Note, that I've passed this in memory avro_writer_t to<br> >> \
avro_schema_to_json and I'm just trying to get the string inside of it<br> \
>> out.)<br> >> --<br>
>> Jeff<br>
>><br>
>> On Sun, Mar 14, 2010 at 12:39 AM, Jeff Hodges <<a \
href="mailto:jhodges@twitter.com">jhodges@twitter.com</a>> wrote:<br> >>> \
I can't figure out the API for turning an avro_schema_t back into an<br> \
>>> in-memory string of json data.<br> >>><br>
>>> It seems I'm supposed to get it out of an avro_writer_t that \
I've<br> >>> created with avro_writer_memory() but I'm a little lost \
once we start<br> >>> having to use avro_writer_to_memory() to get back the \
original<br> >>> avro_memory_writer_t.<br>
>>><br>
>>> Sample code?<br>
>>> --<br>
>>> Jeff<br>
>>><br>
>><br>
</div></div></blockquote></div><br></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic