'[cuckoo] Lacking unicode support in Mako html and json template'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       cuckoo
Subject:    [cuckoo] Lacking unicode support in Mako html and json template
From:       Joakim.Berge () evry ! com (Berge Joakim)
Date:       2012-04-13 8:03:50
Message-ID: F0C5212ED1CF144E8F7606ACA5344CF12A3DEEC8 () EDBCL17EX001 ! edb ! local
[Download RAW message or body]

I just downloaded the latest version from the development branch, and still get a \
                unicode error.
                UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position \
1576: ordinal not in range(128)

Have the fix been added yet?

This gives me the error:
python submit.py --url \
"http://hmortensen.tns-cs.net/j0='''r=http://forum.kvinneguiden.no/index.php?showtopic \
=522233;+'s=hmortensen+cp=klikk/kvinneguiden/forum+url=http://www.klikk.no/kvinneguide \
n/index.jsp?service=klikk_kolofon&tmsecVertical=kvinneguiden&tmsecSection=forum&inc" \
--package firefox

BR
Joakim Berge

From: cuckoo-bounces@public.honeynet.org [mailto:cuckoo-bounces at \
                public.honeynet.org] On Behalf Of Holst, Edvard
Sent: 22. februar 2012 09:12
To: mailing list for Cuckoo users and developers
Subject: Re: [cuckoo] Lacking unicode support in Mako html and json template

While I still wish for a better system for code contribution, I will go with a \
pastebin option then.

In an attempt to add at least some sort of structure to this, I will describe this \
specific change as follows:

FILE: cuckoo/cuckoo/reporting/tasks/reporthtml.py
MODIFICATIONS: http://paste2.org/p/1912404
DESCRIPTION OF CHANGES:
Included the template rendering in the try/catch block. Otherwise cuckoo will crash \
when hitting a problem with the rendering (unicode issues for example) which means \
that no further reports are generated. Template rendering is now done with \
render_unicode() instead of render() as this will return the template output as a \
unicode object. Added input_encoding = 'utf-8' to specify the template encoding.
Fixed the syntax error in the except statement. It previously referred to a variable \
"e" which was not defined, resulting in a crash when hitting the statement. Changed \
                from "e" to "why" and it now works.
NOTE: This does not fix the unicode related issues, but its a start.

Ill send in my additional code later. ;)

On Tue, Feb 21, 2012 at 11:56 PM, Alessandro Tanasi <alessandro at \
lonerunners.net<mailto:alessandro at lonerunners.net>> wrote: You can put your code \
where you want :) Some guys send us patch file via email, others github pull \
requests, other a simple pastebin.

Contributing to cuckoo is simple: put your code somewhere, and give a us a pointer.

Anyway on github code contributions tend to be git pull requests.

We are happy to see community patches for cuckoo, but I think there aren't a more \
easy way than "put your patch where you want" because not all like gerrit and not all \
are able to use it or learn it.

Regards,
Alessandro `jekil` Tanasi
alessandro at tanasi.it<mailto:alessandro at tanasi.it>

On Feb 21, 2012, at 11:15 PM, Holst, Edvard wrote:

Ok. I guess that would be doable, I just prefer open source projects to be open for \
everyone to contribute code to. When I found a significant bug in Android's mms \
handling I didn't email Google a patch, I just committed code to Gerrit, which after \
review was merged into source. ;) I think Cuckoo can develop into something awesome, \
but I think the easier it is for people to contribute, the better.

Anyways, I'll email the code changes and you can do with it as you see fit.

I haven't been able to look into the Unicode issues much yet, but my theory is still \
that the problem must be in the template files. Python 2.7 will handle anything as \
ASCII by default unless specifically stated otherwise. Python 3.x solves this by \
defaulting to Unicode which makes a lot more sense, but getting people to actually \
learn python 3 is quite the challenge it seems, I still havent touched it much myself \
either.

Good luck and keep up the good work!

On Tuesday, February 21, 2012, Alessandro Tanasi wrote:
A patch attached by mail can fit all your needs.

Regards,
Alessandro `jekil` Tanasi
alessandro at tanasi.it<mailto:alessandro at tanasi.it>

On Feb 21, 2012, at 12:00 PM, Holst, Edvard wrote:

> Unfortunately, I am not able to share the analysis either. Sorry for
> the inconvenience. However, I would be more than happy to share the code
> changes Ive made. It would be really easy if I could just get access to
> git, or gerrit if you use that as well, but I understand if you are not
> looking to expand the development team.
> 
> The unicode issue is a bit of a show stopper though..
> 
> Kind regards
> 
> On Sat, Feb 18, 2012 at 2:29 PM, Claudio <claudio at \
> shadowserver.org<mailto:claudio at shadowserver.org>> wrote: 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > He doesn't need the sample, just the analysis.
> > 
> > On 2/18/12 7:04 AM, Holst, Edvard wrote:
> > > Hi there!
> > > 
> > > Glad to hear its being worked on. Whats the best way for me to
> > > contribute code? Ive made quite a few improvements, but Im not
> > > allowed to commit to the git repository.
> > > 
> > > Unfortunately, I am not able to send you the malware sample that
> > > triggers the unicode bug, but hopefully the traceback will be of
> > > assistance.
> > > 
> > > Kind regards
> > > 
> > > On Sat, Feb 18, 2012 at 12:59 PM, Alessandro Tanasi
> > > <alessandro at lonerunners.net <mailto:alessandro at \
> > > lonerunners.net<mailto:alessandro at lonerunners.net%20%3cmailto:alessandro at \
> > > lonerunners.net>>> wrote:
> > > 
> > > Hi, Yes, it's true, exception handling must be improved. We are
> > > already working on that. And not only on that, we want to
> > > significant improve the cuckoo codebase quality. Anyway if you want
> > > to suggest something any patch is welcome.
> > > 
> > > Can you (privately) send me the zipped analysis which triggers
> > > this bug? I opened a ticket about that and this will investigate
> > > and fix if needed.
> > > 
> > > Thanks, Alessandro `jekil` Tanasi alessandro at tanasi.it<mailto:alessandro at \
> > > tanasi.it> <mailto:alessandro at tanasi.it>
> > > 
> > > On Feb 17, 2012, at 2:42 PM, Holst, Edvard wrote:
> > > 
> > > > Been studying the Cuckoobox source code now for about two days.
> > > > Firstly, exception handling is lacking, but that is easy to fix
> > > > and Ive made quite a few changes I can commit if its okay with
> > > > the dev team.
> > > > 
> > > > Secondly though, unicode support in the Mako templates is not
> > > > working. Python 2.7 uses ASCII by default, so when Cuckoo sends
> > > > results containing non-ascii characters to reporthtml.py, you
> > > > get an exception when trying to render, as shown here:
> > > > http://paste.pound-python.org/show/16853/
> > > > 
> > > > Ive been trying to fix that issue, but so far I havent been
> > > > successful. I havent used Mako before, but Ive read through the
> > > > entire documentation, especially the parts about unicode. The
> > > > problem seems to be that there is something wrong with the
> > > > templates and that somewhere, there are strings that are not
> > > > treated as unicode. Im hoping the dev team knows more about this
> > > > than I do, but I will be happy to help in any way I can.
> > > > 
> > > > A fix would be awesome!
> > > > _______________________________________________ Cuckoo mailing
> > > > list Cuckoo at public.honeynet.org<mailto:Cuckoo at public.honeynet.org>
> > > > <mailto:Cuckoo at public.honeynet.org>
> > > > https://public.honeynet.org/mailman/listinfo/cuckoo
> > > 
> > > 
> > > _______________________________________________ Cuckoo mailing
> > > list
_______________________________________________
Cuckoo mailing list
Cuckoo at public.honeynet.org<mailto:Cuckoo at public.honeynet.org>
https://public.honeynet.org/mailman/listinfo/cuckoo

_______________________________________________
Cuckoo mailing list
Cuckoo at public.honeynet.org<mailto:Cuckoo at public.honeynet.org>
https://public.honeynet.org/mailman/listinfo/cuckoo

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://public.honeynet.org/pipermail/cuckoo/attachments/20120413/5b41277d/attachment.html \

[prev in list] [next in list] [prev in thread] [next in thread] 