[prev in list] [next in list] [prev in thread] [next in thread]
List: pypy-dev
Subject: Re: [pypy-dev] Great experience with PyPy
From: Gelin Yan <dynamicgl () gmail ! com>
Date: 2013-02-07 15:12:53
Message-ID: CABkOF6THh6K-+6jAzuPzS0SY1K7g2XKmwRPoXaUQnRx_2cRzDQ () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
On Thu, Feb 7, 2013 at 10:11 PM, Phyo Arkar <phyo.arkarlwin@gmail.com>wrote:
> Pypy should have a page for "Success Stories!"
>
> Now with this and Quora proving Power of PyPy , i am beginning to start
> converting my projects into PyPy soon!
> I am only withholding right now because my projects uses a lot of C
> Libraries and Numpy/Matplotlib/scilit-learn.
>
> Thanks
>
> Phyo.
>
> On Thursday, February 7, 2013, Maciej Fijalkowski wrote:
>
>> On Thu, Feb 7, 2013 at 1:55 PM, Marko Tasic <mtasic85@gmail.com> wrote:
>> > Hi,
>> >
>> > I would like to share short story with you and share what we have
>> > accomplished with PyPy and its friends so far.
>> >
>> > Company that I have worked for last 7 months (intentionally unnamed)
>> > gave me absolute permission to pick up technologies on which we based
>> > our solution. What we do is: crawl for PDFs and newspapers articles,
>> > download, translate them if needed, OCR if needed, do extensive
>> > analysis of downloaded PDFs and articles, store them in more organized
>> > structures for faster querying, search for them and generate bunch of
>> > complex reports.
>> >
>> > From very beginning I decided to go with PyPy no matter what. What we
>> > picked is following:
>> > * Flask for web framework, and few of its extensions such as
>> > Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.
>> > * Cassandra as database because of its features and great experience
>> > with it. PyCassa is used as client to talk to Cassandra server.
>> > * ElasticSearch as distributed search engine, and its client library
>> pyes.
>> > * Whoosh as search engine, but with some modifications to support
>> > Cassandra as storage and distributed locking.
>> > * Redis, and its client library redis-py, for caching and to speed up
>> > common auto-completion patterns.
>> > * ZooKeeper, and its client library Kazoo, for distributed locking
>> > which plays essential role in system for transaction-like behavior
>> > over many services at once.
>> > * Celery in conjunction with RabbitMQ for task distribution.
>> > * Sentry for error logging.
>> >
>> > What we have developed on our own are wrappers and clients for:
>> > * Moses which is language translator
>> > * Tesseract which is OCR engine
>> > * Cassandra store for Whoosh
>> > * wkhtmltopdf and wkhtmltoimage which are used for conversion of HTML
>> > to PDF/Image
>> > * etc
>> >
>> > Now when product is finished and in final testing phase, I can say
>> > that we did not regret because we used PyPy and stack around it.
>> > Typical speed improvement is 2x-3x over CPython in our case, but
>> > anyway we are mostly IO and memory bound, expect for Celery workers
>> > where we do analysis which are again many small CPU intensive tasks
>> > that are exchanged via RabbitMQ. Another reason why we don't see
>> > speedup us is that we are dependent on external software (servers)
>> > written in Erlang and Java.
>> >
>> > I'm already planing to do Cassandra (distributed key/value only
>> > database without index features), ZooKeeper, Redis and ElasticSearch
>> > ports in Python for next projects, and hopefully opensource them.
>> >
>> > Regards,
>> > Marko Tasic
>> > _______________________________________________
>> > pypy-dev mailing list
>> > pypy-dev@python.org
>> > http://mail.python.org/mailman/listinfo/pypy-dev
>>
>> Awesome!
>>
>> I'm glad people can make pypy work for non-trivial tasks which require
>> a lot of dependencies. We're trying to lower the bar, however it takes
>> time.
>>
>> Cheers,
>> fijal
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev@python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
Hi, It might be off topic. I want to know whether pypy support postgres.
The last time I noticed ctypes based psycopg2 was still beta. I mainly use
twisted & postgres. pypy supports twisted well but not good for psycopg2.
Regards
gelin yan
[Attachment #5 (text/html)]
<br><br><div class="gmail_quote">On Thu, Feb 7, 2013 at 10:11 PM, Phyo Arkar <span \
dir="ltr"><<a href="mailto:phyo.arkarlwin@gmail.com" \
target="_blank">phyo.arkarlwin@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> Pypy should have a page for "Success \
Stories!"<div><br></div><div>Now with this and Quora proving Power of PyPy , i \
am beginning to start converting my projects into PyPy soon!</div><div>I am only \
withholding right now because my projects uses a lot of C Libraries and \
Numpy/Matplotlib/scilit-learn.</div>
<div><br></div><div>Thanks</div><span class="HOEnZb"><font \
color="#888888"><div><br></div><div>Phyo.</div></font></span><div class="HOEnZb"><div \
class="h5"><div><br>On Thursday, February 7, 2013, Maciej Fijalkowski wrote:<br> \
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> On Thu, Feb 7, 2013 at 1:55 PM, Marko Tasic \
<<a>mtasic85@gmail.com</a>> wrote:<br> > Hi,<br>
><br>
> I would like to share short story with you and share what we have<br>
> accomplished with PyPy and its friends so far.<br>
><br>
> Company that I have worked for last 7 months (intentionally unnamed)<br>
> gave me absolute permission to pick up technologies on which we based<br>
> our solution. What we do is: crawl for PDFs and newspapers articles,<br>
> download, translate them if needed, OCR if needed, do extensive<br>
> analysis of downloaded PDFs and articles, store them in more organized<br>
> structures for faster querying, search for them and generate bunch of<br>
> complex reports.<br>
><br>
> From very beginning I decided to go with PyPy no matter what. What we<br>
> picked is following:<br>
> * Flask for web framework, and few of its extensions such as<br>
> Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.<br>
> * Cassandra as database because of its features and great experience<br>
> with it. PyCassa is used as client to talk to Cassandra server.<br>
> * ElasticSearch as distributed search engine, and its client library pyes.<br>
> * Whoosh as search engine, but with some modifications to support<br>
> Cassandra as storage and distributed locking.<br>
> * Redis, and its client library redis-py, for caching and to speed up<br>
> common auto-completion patterns.<br>
> * ZooKeeper, and its client library Kazoo, for distributed locking<br>
> which plays essential role in system for transaction-like behavior<br>
> over many services at once.<br>
> * Celery in conjunction with RabbitMQ for task distribution.<br>
> * Sentry for error logging.<br>
><br>
> What we have developed on our own are wrappers and clients for:<br>
> * Moses which is language translator<br>
> * Tesseract which is OCR engine<br>
> * Cassandra store for Whoosh<br>
> * wkhtmltopdf and wkhtmltoimage which are used for conversion of HTML<br>
> to PDF/Image<br>
> * etc<br>
><br>
> Now when product is finished and in final testing phase, I can say<br>
> that we did not regret because we used PyPy and stack around it.<br>
> Typical speed improvement is 2x-3x over CPython in our case, but<br>
> anyway we are mostly IO and memory bound, expect for Celery workers<br>
> where we do analysis which are again many small CPU intensive tasks<br>
> that are exchanged via RabbitMQ. Another reason why we don't see<br>
> speedup us is that we are dependent on external software (servers)<br>
> written in Erlang and Java.<br>
><br>
> I'm already planing to do Cassandra (distributed key/value only<br>
> database without index features), ZooKeeper, Redis and ElasticSearch<br>
> ports in Python for next projects, and hopefully opensource them.<br>
><br>
> Regards,<br>
> Marko Tasic<br>
> _______________________________________________<br>
> pypy-dev mailing list<br>
> <a>pypy-dev@python.org</a><br>
> <a href="http://mail.python.org/mailman/listinfo/pypy-dev" \
target="_blank">http://mail.python.org/mailman/listinfo/pypy-dev</a><br> <br>
Awesome!<br>
<br>
I'm glad people can make pypy work for non-trivial tasks which require<br>
a lot of dependencies. We're trying to lower the bar, however it takes<br>
time.<br>
<br>
Cheers,<br>
fijal<br>
_______________________________________________<br>
pypy-dev mailing list<br>
<a>pypy-dev@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/pypy-dev" \
target="_blank">http://mail.python.org/mailman/listinfo/pypy-dev</a><br> \
</blockquote></div> </div></div><br>_______________________________________________<br>
pypy-dev mailing list<br>
<a href="mailto:pypy-dev@python.org">pypy-dev@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/pypy-dev" \
target="_blank">http://mail.python.org/mailman/listinfo/pypy-dev</a><br> \
<br></blockquote></div><br><div><br></div><div>Hi, It might be off topic. I want to \
know whether pypy support postgres. The last time I noticed ctypes based psycopg2 was \
still beta. I mainly use twisted & postgres. pypy supports twisted well but not \
good for psycopg2.</div> <div><br></div><div>Regards</div><div><br></div><div>gelin \
yan</div>
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic