[prev in list] [next in list] [prev in thread] [next in thread] 

List:       pypy-dev
Subject:    Re: [pypy-dev] Great experience with PyPy
From:       <konstantin.lopuhin () chtd ! ru>
Date:       2013-02-07 15:28:38
Message-ID: CAAvCB=RGFsturqNwGu5iz9AXmhSWMYBGGitbKKDVVuFVu8OW0g () mail ! gmail ! com
[Download RAW message or body]

PyPy supports postgres with either psycopg2cffi or psycopg2-ctypes
bindings. We use psycopg2cffi in production (and maintain them), and
here http://chtd.ru/blog/bystraya-rabota-s-postgres-pod-pypy/?lang=en
are some benchmarks.
And yes, PyPy is cool :) Typically giving 3x speedups, and some memory
savings sometimes.

2013/2/7 Gelin Yan <dynamicgl@gmail.com>:
>
>
> On Thu, Feb 7, 2013 at 10:11 PM, Phyo Arkar <phyo.arkarlwin@gmail.com>
> wrote:
>>
>> Pypy should have a page for "Success Stories!"
>>
>> Now with this and Quora proving Power of PyPy , i am beginning to start
>> converting my projects into PyPy soon!
>> I am only withholding right now because my projects uses a lot of C
>> Libraries and Numpy/Matplotlib/scilit-learn.
>>
>> Thanks
>>
>> Phyo.
>>
>> On Thursday, February 7, 2013, Maciej Fijalkowski wrote:
>>>
>>> On Thu, Feb 7, 2013 at 1:55 PM, Marko Tasic <mtasic85@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I would like to share short story with you and share what we have
>>> > accomplished with PyPy and its friends so far.
>>> >
>>> > Company that I have worked for last 7 months (intentionally unnamed)
>>> > gave me absolute permission to pick up technologies on which we based
>>> > our solution. What we do is: crawl for PDFs and newspapers articles,
>>> > download, translate them if needed, OCR if needed, do extensive
>>> > analysis of downloaded PDFs and articles, store them in more organized
>>> > structures for faster querying, search for them and generate bunch of
>>> > complex reports.
>>> >
>>> > From very beginning I decided to go with PyPy no matter what. What we
>>> > picked is following:
>>> > * Flask for web framework, and few of its extensions such as
>>> > Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.
>>> > * Cassandra as database because of its features and great experience
>>> > with it. PyCassa is used as client to talk to Cassandra server.
>>> > * ElasticSearch as distributed search engine, and its client library
>>> > pyes.
>>> > * Whoosh as search engine, but with some modifications to support
>>> > Cassandra as storage and distributed locking.
>>> > * Redis, and its client library redis-py, for caching and to speed up
>>> > common auto-completion patterns.
>>> > * ZooKeeper, and its client library Kazoo, for distributed locking
>>> > which plays essential role in system for transaction-like behavior
>>> > over many services at once.
>>> > * Celery in conjunction with RabbitMQ for task distribution.
>>> > * Sentry for error logging.
>>> >
>>> > What we have developed on our own are wrappers and clients for:
>>> > * Moses which is language translator
>>> > * Tesseract which is OCR engine
>>> > * Cassandra store for Whoosh
>>> > * wkhtmltopdf and wkhtmltoimage which are used for conversion of HTML
>>> > to PDF/Image
>>> > * etc
>>> >
>>> > Now when product is finished and in final testing phase, I can say
>>> > that we did not regret because we used PyPy and stack around it.
>>> > Typical speed improvement is 2x-3x over CPython in our case, but
>>> > anyway we are mostly IO and memory bound, expect for Celery workers
>>> > where we do analysis which are again many small CPU intensive tasks
>>> > that are exchanged via RabbitMQ. Another reason why we don't see
>>> > speedup us is that we are dependent on external software (servers)
>>> > written in Erlang and Java.
>>> >
>>> > I'm already planing to do Cassandra (distributed key/value only
>>> > database without index features), ZooKeeper, Redis and ElasticSearch
>>> > ports in Python for next projects, and hopefully opensource them.
>>> >
>>> > Regards,
>>> > Marko Tasic
>>> > _______________________________________________
>>> > pypy-dev mailing list
>>> > pypy-dev@python.org
>>> > http://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>> Awesome!
>>>
>>> I'm glad people can make pypy work for non-trivial tasks which require
>>> a lot of dependencies. We're trying to lower the bar, however it takes
>>> time.
>>>
>>> Cheers,
>>> fijal
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev@python.org
>>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev@python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>
>
> Hi, It might be off topic. I want to know whether pypy support postgres. The
> last time I noticed ctypes based psycopg2 was still beta. I mainly use
> twisted & postgres. pypy supports twisted well but not good for psycopg2.
>
> Regards
>
> gelin yan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>



--
Константин Лопухин, разра отчик
Компания ЧТД -- http://chtd.ru
+7 (495) 646-87-45, до авочный 333
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

[prev in list] [next in list] [prev in thread] [next in thread]