[prev in list] [next in list] [prev in thread] [next in thread]
List: haskell-cafe
Subject: Re: [Haskell-cafe] Streaming to JuicyPixels
From: Vincent Berthoux <vincent.berthoux () gmail ! com>
Date: 2012-02-26 18:55:32
Message-ID: CALCmskU+w5CdrMG24wCJVphGHspe7g3qgZM2mx_aoQ4UP_yFaQ () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi,
I didn't thought it would be a full rewrite, when I finish Jpeg writing
functionality, I will look at it in depth, which will not be immediately.
Feel free to use any code snippet from JuicyPixels if it help your for your
own library.
Regards
Vincent Berthoux
Le 24 f=E9vrier 2012 06:42, Myles C. Maxfield <myles.maxfield@gmail.com> a
=E9crit :
So I started working on moving JuicyPixels to a streaming interface, and
> have some observations. This is going to be a pretty major change, touchi=
ng
> pretty much every function, and the end result will end up looking very
> close to the code that I have already written. I'm not nearly as close to
> the code as the author is, and I've already made some mistakes due to not
> understanding how the original code is structured and layed out.
>
> Because this is essentially turning out to be a rewrite, I think it makes
> more sense for me to just work on my own library, and release it as a
> streaming alternative to JuicyPixels. How do you feel about this, Vincent=
?
>
> Thanks,
> Myles
>
> On Wed, Feb 22, 2012 at 12:30 PM, Vincent Berthoux <
> vincent.berthoux@gmail.com> wrote:
>
>> Hi,
>> Please go ahead, and github is the perfect medium for code sharing :)
>>
>> Regards
>>
>> Vincent Berthoux
>>
>> Le 22 f=E9vrier 2012 20:20, Myles C. Maxfield <myles.maxfield@gmail.com>=
a
>> =E9crit :
>>
>> Let's put aside the issue of getting access to the pixels before the
>>> stream is complete.
>>>
>>> How would you feel if I implemented use of the STT monad transformer on
>>> top of Get in JuicyPixels, in order to get rid of the (remaining >>
>>> getBytes) call, and then expose the underlying Get interface to callers=
?
>>> This would allow use for streaming.
>>>
>>> Is this something that you feel that I should pursue? I can send you a
>>> GitHub Pull Request when I'm done.
>>>
>>> Thanks,
>>> Myles
>>>
>>> On Wed, Feb 22, 2012 at 5:01 AM, Vincent Berthoux <
>>> vincent.berthoux@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I can understand your performance problems, I bumped into them befor=
e
>>>> the first release of Juicy Pixels and took a long time to get 'correct=
'
>>>> performance out of the box, and the IDCT is not the only 'hot point', =
I got
>>>> problems with intermediate data structure as well. Any list has proven=
a
>>>> bit problematic performance wise, so I rewrote some things which would=
have
>>>> been easily implemented with forM_ or mapM_ with manual recursion.
>>>>
>>>> I didn't knew STT existed, so it open a new area of reflection for the
>>>> streaming Jpeg decoder, instead of using the (remaining >> getBytes) c=
ombo,
>>>> staying in the Get monad might help. The reason to expose the ST monad=
is
>>>> that I use it internally to mutate the final image directly, and I'd p=
refer
>>>> to avoid freezing/unfreezing the underlying array. So in order to give
>>>> access to the intermediate representation, a type could be (STT s (Sta=
teT
>>>> JpegDecodingState Get) (MutableImage s PixelYCbCr8)) (the Jpeg decoder=
only
>>>> produce (Image PixelYCbCr8 internally)). This should allow a freeze th=
en a
>>>> color conversion.
>>>>
>>>> As it would induce performance loss, this version should exist
>>>> alongside current implementation. This is not trivial, but it's far fr=
om
>>>> impossible. For the IDCT implementation, I don't think a package make =
much
>>>> sense, if you want to use it, just grab it and customize the interface=
to
>>>> your needs :).
>>>>
>>>> Regards
>>>>
>>>> Vincent Berthoux
>>>>
>>>> Le 21 f=E9vrier 2012 06:16, Myles C. Maxfield <myles.maxfield@gmail.co=
m>a =E9crit :
>>>>
>>>> Hello, and thanks for the quick reply.
>>>>>
>>>>> You're right that using (remaining >> getBytes) won't work for
>>>>> streaming, as it would pull the rest of the stream into a buffer, the=
reby
>>>>> delaying further processing until the image is completely downloaded.=
:-(
>>>>>
>>>>> I'm a little unclear about what you mean about the use of the ST
>>>>> monad. There is an STT<http://hackage.haskell.org/packages/archive/ST=
MonadTrans/0.2/doc/html/Control-Monad-ST-Trans.html> monad
>>>>> transformer, so you could wrap that around Get. Is that what you're
>>>>> referring to?
>>>>>
>>>>> As an aside: I didn't realize that JuicyPixels existed, so I wrote a
>>>>> JPEG decoder specifically designed for streaming - it doesn't request=
a
>>>>> byte of input unless it has to, uses a StateT (wrapped around Attopar=
sec)
>>>>> to keep track of which bit in the current byte is "next", and does th=
e
>>>>> Huffman decoding in-line. However, I didn't use ST for the IDCT, so m=
y own
>>>>> decoder has serious performance problems. This prompted me to start
>>>>> searching around for a solution, and I came across JuicyPixels, which
>>>>> already exists and is much faster than my own implementation. I'm hop=
ing to
>>>>> get rid of my own decoder and just use JuicyPixels. If you're curious=
, my
>>>>> own code is here: https://github.com/litherum/jpeg.
>>>>>
>>>>> Is it reasonable to extend JuicyPixels to fit my use case? It sounds
>>>>> like JuicyPixels wouldn't work so well as it stands. I'd be happy to =
do
>>>>> whatever work is necessary to help out and get JuicyPixels usable for=
me.
>>>>> However, if that would require a full (or near-full) rewrite, it migh=
t make
>>>>> more sense for me to use my own implementation with your IDCT. Is the=
re a
>>>>> way we can share just the IDCT between our two repositories? Perhaps =
making
>>>>> a new IDCT8 library that we can both depend on?
>>>>>
>>>>> As for what API I'd like to be able to use, just a "Get DynamicImage"
>>>>> should suffice (assuming it has streaming semantics as described abov=
e). It
>>>>> would be really nice if it was possible to get at the incomplete imag=
e
>>>>> before the stream is completed (so the image could slowly update as m=
ore
>>>>> data arrives from the network), but I'm not quite sure how to elegant=
ly
>>>>> express that. Do you have any ideas?
>>>>>
>>>>> I think that having 2 native jpeg decoders (Actually 3, because of th=
is
>>>>> package <http://hackage.haskell.org/package/jpeg>) is detrimental to
>>>>> the Haskell community, and I would really like to use JuicyPixels :D
>>>>>
>>>>> Thanks,
>>>>> Myles C. Maxfield
>>>>>
>>>>>
>>>>> On Mon, Feb 20, 2012 at 3:01 PM, Vincent Berthoux <
>>>>> vincent.berthoux@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I can expose the low level parsing, but you would only get the
>>>>>> chunks/frames/sections of the image, Cereal is mainly used to parse =
the
>>>>>> structure of the image, not to do the raw processing. For the raw
>>>>>> processing, I rely on `remaining >> getBytes` to be able to manipula=
te data
>>>>>> at bit level or to feed it to zlib, and the documentation clearly st=
ate
>>>>>> that remaining doesn't work well with runGetPartial, so no read ahea=
d, but
>>>>>> even worse for streaming :).
>>>>>>
>>>>>> To be fair, I never thought of this use case, and exposing a
>>>>>> partially decoded image would impose the use of the ST Monad somehow=
, and
>>>>>> Serialize is not a monad transformer, making it a bit hard to implem=
ent.
>>>>>>
>>>>>> By curiosity what kind of API would you hope for this kind of
>>>>>> functionality?
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Vincent Berthoux
>>>>>>
>>>>>> Le 20 f=E9vrier 2012 22:08, Myles C. Maxfield <myles.maxfield@gmail.=
com
>>>>>> > a =E9crit :
>>>>>>
>>>>>> Hello,
>>>>>>> I am interested in the possibility of using JuicyPixels for
>>>>>>> streaming images from the web. It doesn't appear to expose any of i=
ts
>>>>>>> internally-used Serialize.Get functionality, which is problematic f=
or
>>>>>>> streaming - I would not like to have to stream the whole image into=
a
>>>>>>> buffer before the decoder can start decoding. Are there any plans o=
n
>>>>>>> exposing this API, so I can use the runGetPartial function to facil=
itate
>>>>>>> streaming?
>>>>>>>
>>>>>>> In addition, does the library do much readahead? There's no point i=
n
>>>>>>> exposing a Get interface if it's just going to wait until the strea=
m is
>>>>>>> done to start decoding anyway.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Myles C. Maxfield
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Haskell-Cafe mailing list
>>>> Haskell-Cafe@haskell.org
>>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe@haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
>>
>
[Attachment #5 (text/html)]
<div class="gmail_quote">Hi,<br><br> I didn't thought it would be a full \
rewrite, when I finish Jpeg writing functionality, I will look at it in depth, which \
will not be immediately. Feel free to use any code snippet from JuicyPixels if it \
help your for your own library.<br>
<br> Regards<br><br> Vincent Berthoux<br><br><div class="gmail_quote">Le 24 \
février 2012 06:42, Myles C. Maxfield <span dir="ltr"><<a \
href="mailto:myles.maxfield@gmail.com" \
target="_blank">myles.maxfield@gmail.com</a>></span> a écrit :<div> <div \
class="h5"><br> <blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex">So I started working on moving \
JuicyPixels to a streaming interface, and have some observations. This is going to be \
a pretty major change, touching pretty much every function, and the end result will \
end up looking very close to the code that I have already written. I'm not nearly \
as close to the code as the author is, and I've already made some mistakes due to \
not understanding how the original code is structured and layed out.<div>
<br></div><div>Because this is essentially turning out to be a rewrite, I think it \
makes more sense for me to just work on my own library, and release it as a streaming \
alternative to JuicyPixels. How do you feel about this, Vincent?</div>
<div><br></div><div>Thanks,</div><div>Myles</div><div><div><div><br><div \
class="gmail_quote">On Wed, Feb 22, 2012 at 12:30 PM, Vincent Berthoux <span \
dir="ltr"><<a href="mailto:vincent.berthoux@gmail.com" \
target="_blank">vincent.berthoux@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div><div>Hi,<br><div class="gmail_quote"> Please go ahead, \
and github is the perfect medium for code sharing :)<br>
<br> Regards<br><br> Vincent Berthoux<br><br><div class="gmail_quote">Le 22 \
février 2012 20:20, Myles C. Maxfield <span dir="ltr"><<a \
href="mailto:myles.maxfield@gmail.com" \
target="_blank">myles.maxfield@gmail.com</a>></span> a écrit :<div>
<div><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">Let's put aside the issue of getting access to the pixels \
before the stream is complete.<div><br></div><div>How would you feel if I implemented \
use of the STT monad transformer on top of Get in JuicyPixels, in order to get rid of \
the (remaining >> getBytes) call, and then expose the underlying Get interface \
to callers? This would allow use for streaming.</div>
<div><br></div><div>Is this something that you feel that I should pursue? I can send \
you a GitHub Pull Request when I'm \
done.</div><div><br></div><div>Thanks,</div><div>Myles<br><br><div \
class="gmail_quote"><div><div>
On Wed, Feb 22, 2012 at 5:01 AM, Vincent Berthoux <span dir="ltr"><<a \
href="mailto:vincent.berthoux@gmail.com" \
target="_blank">vincent.berthoux@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex"><div><div><div><div><div class="gmail_quote">Hi,<br><br> \
I can understand your performance problems, I bumped into them before the first \
release of Juicy Pixels and took a long time to get 'correct' performance out \
of the box, and the IDCT is not the only 'hot point', I got problems with \
intermediate data structure as well. Any list has proven a bit problematic \
performance wise, so I rewrote some things which would have been easily implemented \
with forM_ or mapM_ with manual recursion.<br>
<br>I didn't knew STT existed, so it open a new area of reflection for the \
streaming Jpeg decoder, instead of using the (remaining >> getBytes) combo, \
staying in the Get monad might help. The reason to expose the ST monad is that I use \
it internally to mutate the final image directly, and I'd prefer to avoid \
freezing/unfreezing the underlying array. So in order to give access to the \
intermediate representation, a type could be (STT s (StateT JpegDecodingState Get) \
(MutableImage s PixelYCbCr8)) (the Jpeg decoder only produce (Image PixelYCbCr8 \
internally)). This should allow a freeze then a color conversion.<br>
<br> As it would induce performance loss, this version should exist alongside \
current implementation. This is not trivial, but it's far from impossible. For \
the IDCT implementation, I don't think a package make much sense, if you want to \
use it, just grab it and customize the interface to your needs :).<br>
<br> Regards<br><br> Vincent Berthoux<br><br><div class="gmail_quote">Le 21 \
février 2012 06:16, Myles C. Maxfield <span dir="ltr"><<a \
href="mailto:myles.maxfield@gmail.com" \
target="_blank">myles.maxfield@gmail.com</a>></span> a écrit :<div>
<div><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">Hello, and thanks for the quick \
reply.<div><br></div><div>You're right that using (remaining >> getBytes) \
won't work for streaming, as it would pull the rest of the stream into a buffer, \
thereby delaying further processing until the image is completely downloaded. \
:-(</div>
<div><br></div><div>I'm a little unclear about what you mean about the use of the \
ST monad. There is an <a \
href="http://hackage.haskell.org/packages/archive/STMonadTrans/0.2/doc/html/Control-Monad-ST-Trans.html" \
target="_blank">STT</a> monad transformer, so you could wrap that around Get. Is that \
what you're referring to?</div>
<div><br></div><div>As an aside: I didn't realize that JuicyPixels existed, so I \
wrote a JPEG decoder specifically designed for streaming - it doesn't request a \
byte of input unless it has to, uses a StateT (wrapped around Attoparsec) to keep \
track of which bit in the current byte is "next", and does the Huffman \
decoding in-line. However, I didn't use ST for the IDCT, so my own decoder has \
serious performance problems. This prompted me to start searching around for a \
solution, and I came across JuicyPixels, which already exists and is much faster than \
my own implementation. I'm hoping to get rid of my own decoder and just use \
JuicyPixels. If you're curious, my own code is here: <a \
href="https://github.com/litherum/jpeg" \
target="_blank">https://github.com/litherum/jpeg</a>.</div>
<div><br></div><div>Is it reasonable to extend JuicyPixels to fit my use case? It \
sounds like JuicyPixels wouldn't work so well as it stands. I'd be happy to \
do whatever work is necessary to help out and get JuicyPixels usable for me. However, \
if that would require a full (or near-full) rewrite, it might make more sense for me \
to use my own implementation with your IDCT. Is there a way we can share just the \
IDCT between our two repositories? Perhaps making a new IDCT8 library that we can \
both depend on?</div>
<div><br></div><div>As for what API I'd like to be able to use, just a "Get \
DynamicImage" should suffice (assuming it has streaming semantics as described \
above). It would be really nice if it was possible to get at the incomplete image \
before the stream is completed (so the image could slowly update as more data arrives \
from the network), but I'm not quite sure how to elegantly express that. Do you \
have any ideas?</div>
<div><br></div><div>I think that having 2 native jpeg decoders (Actually 3, because \
of <a href="http://hackage.haskell.org/package/jpeg" target="_blank">this \
package</a>) is detrimental to the Haskell community, and I would really like to use \
JuicyPixels :D</div>
<div><br></div><div>Thanks,</div><div>Myles C. Maxfield<div><div><br><br><div \
class="gmail_quote">On Mon, Feb 20, 2012 at 3:01 PM, Vincent Berthoux <span \
dir="ltr"><<a href="mailto:vincent.berthoux@gmail.com" \
target="_blank">vincent.berthoux@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">Hi,<br><br> I can expose the low level parsing, but you would \
only get the chunks/frames/sections of the image, Cereal is mainly used to parse the \
structure of the image, not to do the raw processing. For the raw processing, I rely \
on `remaining >> getBytes` to be able to manipulate data at bit level or to \
feed it to zlib, and the documentation clearly state that remaining doesn't work \
well with runGetPartial, so no read ahead, but even worse for streaming :).<br>
<br>To be fair, I never thought of this use case, and exposing a partially decoded \
image would impose the use of the ST Monad somehow, and Serialize is not a monad \
transformer, making it a bit hard to implement.<br><br>By curiosity what kind of API \
would you hope for this kind of functionality?<br>
<br>Regards<br> <br>Vincent Berthoux<br><br><div class="gmail_quote">Le 20 février \
2012 22:08, Myles C. Maxfield <span dir="ltr"><<a \
href="mailto:myles.maxfield@gmail.com" \
target="_blank">myles.maxfield@gmail.com</a>></span> a écrit :<div>
<div><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">Hello,<div>I am interested in the possibility of using \
JuicyPixels for streaming images from the web. It doesn't appear to expose any of \
its internally-used Serialize.Get functionality, which is problematic for streaming - \
I would not like to have to stream the whole image into a buffer before the decoder \
can start decoding. Are there any plans on exposing this API, so I can use the \
runGetPartial function to facilitate streaming?</div>
<div><br></div><div>In addition, does the library do much readahead? There's no \
point in exposing a Get interface if it's just going to wait until the stream is \
done to start decoding anyway.</div><div><br></div>
<div>
Thanks,</div><div>Myles C. Maxfield</div>
</blockquote></div></div></div><br>
</blockquote></div><br></div></div></div>
</blockquote></div></div></div><br>
</div><br>
</div></div><br></div></div>_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org" \
target="_blank">Haskell-Cafe@haskell.org</a><br> <a \
href="http://www.haskell.org/mailman/listinfo/haskell-cafe" \
target="_blank">http://www.haskell.org/mailman/listinfo/haskell-cafe</a><br> \
<br></blockquote></div><br></div> </blockquote></div></div></div><br>
</div><br>
</div></div><br>_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org" \
target="_blank">Haskell-Cafe@haskell.org</a><br> <a \
href="http://www.haskell.org/mailman/listinfo/haskell-cafe" \
target="_blank">http://www.haskell.org/mailman/listinfo/haskell-cafe</a><br> \
<br></blockquote></div><br></div> </div></div></blockquote></div></div></div><br>
</div><br>
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic