[prev in list] [next in list] [prev in thread] [next in thread]
List: pypy-dev
Subject: Re: [pypy-dev] Great experience with PyPy
From: <kostia.lopuhin () gmail ! com>
Date: 2013-02-07 16:08:51
Message-ID: CAAvCB=RNGsXGa81nh1CRpC-UMNfRuQuwBopQjo1UNh5aH7P8yg () mail ! gmail ! com
[Download RAW message or body]
Hi! I did not test it on Windows, there may be problems with
installation (searching for postgres header files, the config is not
very smart - https://github.com/chtd/psycopg2cffi/blob/master/psycopg2cffi/_impl/libpq.py#L209),
but they should be solvable I hope - submit a bug if you have
problems.
2013/2/7 Gelin Yan <dynamicgl@gmail.com>:
>
>
> On Thu, Feb 7, 2013 at 11:28 PM, Костя Лопухин <konstantin.lopuhin@chtd.ru>
> wrote:
>>
>> PyPy supports postgres with either psycopg2cffi or psycopg2-ctypes
>> bindings. We use psycopg2cffi in production (and maintain them), and
>> here http://chtd.ru/blog/bystraya-rabota-s-postgres-pod-pypy/?lang=en
>> are some benchmarks.
>> And yes, PyPy is cool :) Typically giving 3x speedups, and some memory
>> savings sometimes.
>>
>> 2013/2/7 Gelin Yan <dynamicgl@gmail.com>:
>> >
>> >
>> > On Thu, Feb 7, 2013 at 10:11 PM, Phyo Arkar <phyo.arkarlwin@gmail.com>
>> > wrote:
>> >>
>> >> Pypy should have a page for "Success Stories!"
>> >>
>> >> Now with this and Quora proving Power of PyPy , i am beginning to start
>> >> converting my projects into PyPy soon!
>> >> I am only withholding right now because my projects uses a lot of C
>> >> Libraries and Numpy/Matplotlib/scilit-learn.
>> >>
>> >> Thanks
>> >>
>> >> Phyo.
>> >>
>> >> On Thursday, February 7, 2013, Maciej Fijalkowski wrote:
>> >>>
>> >>> On Thu, Feb 7, 2013 at 1:55 PM, Marko Tasic <mtasic85@gmail.com>
>> >>> wrote:
>> >>> > Hi,
>> >>> >
>> >>> > I would like to share short story with you and share what we have
>> >>> > accomplished with PyPy and its friends so far.
>> >>> >
>> >>> > Company that I have worked for last 7 months (intentionally unnamed)
>> >>> > gave me absolute permission to pick up technologies on which we
>> >>> > based
>> >>> > our solution. What we do is: crawl for PDFs and newspapers articles,
>> >>> > download, translate them if needed, OCR if needed, do extensive
>> >>> > analysis of downloaded PDFs and articles, store them in more
>> >>> > organized
>> >>> > structures for faster querying, search for them and generate bunch
>> >>> > of
>> >>> > complex reports.
>> >>> >
>> >>> > From very beginning I decided to go with PyPy no matter what. What
>> >>> > we
>> >>> > picked is following:
>> >>> > * Flask for web framework, and few of its extensions such as
>> >>> > Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.
>> >>> > * Cassandra as database because of its features and great experience
>> >>> > with it. PyCassa is used as client to talk to Cassandra server.
>> >>> > * ElasticSearch as distributed search engine, and its client library
>> >>> > pyes.
>> >>> > * Whoosh as search engine, but with some modifications to support
>> >>> > Cassandra as storage and distributed locking.
>> >>> > * Redis, and its client library redis-py, for caching and to speed
>> >>> > up
>> >>> > common auto-completion patterns.
>> >>> > * ZooKeeper, and its client library Kazoo, for distributed locking
>> >>> > which plays essential role in system for transaction-like behavior
>> >>> > over many services at once.
>> >>> > * Celery in conjunction with RabbitMQ for task distribution.
>> >>> > * Sentry for error logging.
>> >>> >
>> >>> > What we have developed on our own are wrappers and clients for:
>> >>> > * Moses which is language translator
>> >>> > * Tesseract which is OCR engine
>> >>> > * Cassandra store for Whoosh
>> >>> > * wkhtmltopdf and wkhtmltoimage which are used for conversion of
>> >>> > HTML
>> >>> > to PDF/Image
>> >>> > * etc
>> >>> >
>> >>> > Now when product is finished and in final testing phase, I can say
>> >>> > that we did not regret because we used PyPy and stack around it.
>> >>> > Typical speed improvement is 2x-3x over CPython in our case, but
>> >>> > anyway we are mostly IO and memory bound, expect for Celery workers
>> >>> > where we do analysis which are again many small CPU intensive tasks
>> >>> > that are exchanged via RabbitMQ. Another reason why we don't see
>> >>> > speedup us is that we are dependent on external software (servers)
>> >>> > written in Erlang and Java.
>> >>> >
>> >>> > I'm already planing to do Cassandra (distributed key/value only
>> >>> > database without index features), ZooKeeper, Redis and ElasticSearch
>> >>> > ports in Python for next projects, and hopefully opensource them.
>> >>> >
>> >>> > Regards,
>> >>> > Marko Tasic
>> >>> > _______________________________________________
>> >>> > pypy-dev mailing list
>> >>> > pypy-dev@python.org
>> >>> > http://mail.python.org/mailman/listinfo/pypy-dev
>> >>>
>> >>> Awesome!
>> >>>
>> >>> I'm glad people can make pypy work for non-trivial tasks which require
>> >>> a lot of dependencies. We're trying to lower the bar, however it takes
>> >>> time.
>> >>>
>> >>> Cheers,
>> >>> fijal
>> >>> _______________________________________________
>> >>> pypy-dev mailing list
>> >>> pypy-dev@python.org
>> >>> http://mail.python.org/mailman/listinfo/pypy-dev
>> >>
>> >>
>> >> _______________________________________________
>> >> pypy-dev mailing list
>> >> pypy-dev@python.org
>> >> http://mail.python.org/mailman/listinfo/pypy-dev
>> >>
>> >
>> >
>> > Hi, It might be off topic. I want to know whether pypy support postgres.
>> > The
>> > last time I noticed ctypes based psycopg2 was still beta. I mainly use
>> > twisted & postgres. pypy supports twisted well but not good for
>> > psycopg2.
>> >
>> > Regards
>> >
>> > gelin yan
>> >
>> > _______________________________________________
>> > pypy-dev mailing list
>> > pypy-dev@python.org
>> > http://mail.python.org/mailman/listinfo/pypy-dev
>> >
>>
>>
>>
>> --
>> Константин Лопухин, разра отчик
>> Компания ЧТД -- http://chtd.ru
>> +7 (495) 646-87-45, до авочный 333
>
>
> Hi
>
> Glad to hear that. I will give it a try. By the way, Can i use it on
> windows? It looks like cffi support windows.
>
> Regards
>
> gelin yan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic