[prev in list] [next in list] [prev in thread] [next in thread]
List: pypy-dev
Subject: Re: [pypy-dev] Great experience with PyPy
From: Phyo Arkar <phyo.arkarlwin () gmail ! com>
Date: 2013-02-07 14:23:16
Message-ID: CA+HjJrizQ1JL92bD8ux0bBj122Djv10GzvTGL0b6S9OnNc_Q-A () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Pypy should have a page for "Success Stories!"
Now with this and Quora proving Power of PyPy , i am beginning to start
converting my projects into PyPy soon!
I am only withholding right now because my projects uses a lot of C
Libraries and Numpy/Matplotlib/scilit-learn.
Thanks
Phyo.
On Thursday, February 7, 2013, Maciej Fijalkowski wrote:
> On Thu, Feb 7, 2013 at 1:55 PM, Marko Tasic <mtasic85@gmail.com<javascript:;>>
> wrote:
> > Hi,
> >
> > I would like to share short story with you and share what we have
> > accomplished with PyPy and its friends so far.
> >
> > Company that I have worked for last 7 months (intentionally unnamed)
> > gave me absolute permission to pick up technologies on which we based
> > our solution. What we do is: crawl for PDFs and newspapers articles,
> > download, translate them if needed, OCR if needed, do extensive
> > analysis of downloaded PDFs and articles, store them in more organized
> > structures for faster querying, search for them and generate bunch of
> > complex reports.
> >
> > From very beginning I decided to go with PyPy no matter what. What we
> > picked is following:
> > * Flask for web framework, and few of its extensions such as
> > Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.
> > * Cassandra as database because of its features and great experience
> > with it. PyCassa is used as client to talk to Cassandra server.
> > * ElasticSearch as distributed search engine, and its client library
> pyes.
> > * Whoosh as search engine, but with some modifications to support
> > Cassandra as storage and distributed locking.
> > * Redis, and its client library redis-py, for caching and to speed up
> > common auto-completion patterns.
> > * ZooKeeper, and its client library Kazoo, for distributed locking
> > which plays essential role in system for transaction-like behavior
> > over many services at once.
> > * Celery in conjunction with RabbitMQ for task distribution.
> > * Sentry for error logging.
> >
> > What we have developed on our own are wrappers and clients for:
> > * Moses which is language translator
> > * Tesseract which is OCR engine
> > * Cassandra store for Whoosh
> > * wkhtmltopdf and wkhtmltoimage which are used for conversion of HTML
> > to PDF/Image
> > * etc
> >
> > Now when product is finished and in final testing phase, I can say
> > that we did not regret because we used PyPy and stack around it.
> > Typical speed improvement is 2x-3x over CPython in our case, but
> > anyway we are mostly IO and memory bound, expect for Celery workers
> > where we do analysis which are again many small CPU intensive tasks
> > that are exchanged via RabbitMQ. Another reason why we don't see
> > speedup us is that we are dependent on external software (servers)
> > written in Erlang and Java.
> >
> > I'm already planing to do Cassandra (distributed key/value only
> > database without index features), ZooKeeper, Redis and ElasticSearch
> > ports in Python for next projects, and hopefully opensource them.
> >
> > Regards,
> > Marko Tasic
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev@python.org <javascript:;>
> > http://mail.python.org/mailman/listinfo/pypy-dev
>
> Awesome!
>
> I'm glad people can make pypy work for non-trivial tasks which require
> a lot of dependencies. We're trying to lower the bar, however it takes
> time.
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev mailing list
> pypy-dev@python.org <javascript:;>
> http://mail.python.org/mailman/listinfo/pypy-dev
>
[Attachment #5 (text/html)]
Pypy should have a page for "Success Stories!"<div><br></div><div>Now with \
this and Quora proving Power of PyPy , i am beginning to start converting my projects \
into PyPy soon!</div><div>I am only withholding right now because my projects uses a \
lot of C Libraries and Numpy/Matplotlib/scilit-learn.</div> \
<div><br></div><div>Thanks</div><div><br></div><div>Phyo.</div><div><br>On Thursday, \
February 7, 2013, Maciej Fijalkowski wrote:<br><blockquote class="gmail_quote" \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On Thu, Feb 7, \
2013 at 1:55 PM, Marko Tasic <<a href="javascript:;" onclick="_e(event, \
'cvml', 'mtasic85@gmail.com')">mtasic85@gmail.com</a>> wrote:<br> \
> Hi,<br> ><br>
> I would like to share short story with you and share what we have<br>
> accomplished with PyPy and its friends so far.<br>
><br>
> Company that I have worked for last 7 months (intentionally unnamed)<br>
> gave me absolute permission to pick up technologies on which we based<br>
> our solution. What we do is: crawl for PDFs and newspapers articles,<br>
> download, translate them if needed, OCR if needed, do extensive<br>
> analysis of downloaded PDFs and articles, store them in more organized<br>
> structures for faster querying, search for them and generate bunch of<br>
> complex reports.<br>
><br>
> From very beginning I decided to go with PyPy no matter what. What we<br>
> picked is following:<br>
> * Flask for web framework, and few of its extensions such as<br>
> Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.<br>
> * Cassandra as database because of its features and great experience<br>
> with it. PyCassa is used as client to talk to Cassandra server.<br>
> * ElasticSearch as distributed search engine, and its client library pyes.<br>
> * Whoosh as search engine, but with some modifications to support<br>
> Cassandra as storage and distributed locking.<br>
> * Redis, and its client library redis-py, for caching and to speed up<br>
> common auto-completion patterns.<br>
> * ZooKeeper, and its client library Kazoo, for distributed locking<br>
> which plays essential role in system for transaction-like behavior<br>
> over many services at once.<br>
> * Celery in conjunction with RabbitMQ for task distribution.<br>
> * Sentry for error logging.<br>
><br>
> What we have developed on our own are wrappers and clients for:<br>
> * Moses which is language translator<br>
> * Tesseract which is OCR engine<br>
> * Cassandra store for Whoosh<br>
> * wkhtmltopdf and wkhtmltoimage which are used for conversion of HTML<br>
> to PDF/Image<br>
> * etc<br>
><br>
> Now when product is finished and in final testing phase, I can say<br>
> that we did not regret because we used PyPy and stack around it.<br>
> Typical speed improvement is 2x-3x over CPython in our case, but<br>
> anyway we are mostly IO and memory bound, expect for Celery workers<br>
> where we do analysis which are again many small CPU intensive tasks<br>
> that are exchanged via RabbitMQ. Another reason why we don't see<br>
> speedup us is that we are dependent on external software (servers)<br>
> written in Erlang and Java.<br>
><br>
> I'm already planing to do Cassandra (distributed key/value only<br>
> database without index features), ZooKeeper, Redis and ElasticSearch<br>
> ports in Python for next projects, and hopefully opensource them.<br>
><br>
> Regards,<br>
> Marko Tasic<br>
> _______________________________________________<br>
> pypy-dev mailing list<br>
> <a href="javascript:;" onclick="_e(event, 'cvml', \
'pypy-dev@python.org')">pypy-dev@python.org</a><br> > <a \
href="http://mail.python.org/mailman/listinfo/pypy-dev" \
target="_blank">http://mail.python.org/mailman/listinfo/pypy-dev</a><br> <br>
Awesome!<br>
<br>
I'm glad people can make pypy work for non-trivial tasks which require<br>
a lot of dependencies. We're trying to lower the bar, however it takes<br>
time.<br>
<br>
Cheers,<br>
fijal<br>
_______________________________________________<br>
pypy-dev mailing list<br>
<a href="javascript:;" onclick="_e(event, 'cvml', \
'pypy-dev@python.org')">pypy-dev@python.org</a><br> <a \
href="http://mail.python.org/mailman/listinfo/pypy-dev" \
target="_blank">http://mail.python.org/mailman/listinfo/pypy-dev</a><br> \
</blockquote></div>
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic