'Re: [pypy-dev] Great experience with PyPy'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       pypy-dev
Subject:    Re: [pypy-dev] Great experience with PyPy
From:       David Fraser <davidf () sjsoft ! com>
Date:       2013-02-13 7:53:10
Message-ID: 24890242.381.1360741988037.JavaMail.davidf () jackdaw ! local
[Download RAW message or body]

You may also want to try pg8000; this is a pure-Python driver that works on Windows

On Thursday, February 7, 2013 at 6:08:51 PM, "Костя Лопухин" \
<kostia.lopuhin@gmail.com> wrote:
> Hi! I did not test it on Windows, there may be problems with
> installation (searching for postgres header files, the config is not
> very smart -
> https://github.com/chtd/psycopg2cffi/blob/master/psycopg2cffi/_impl/libpq.py#L209),
> but they should be solvable I hope - submit a bug if you have
> problems.
> 
> 2013/2/7 Gelin Yan <dynamicgl@gmail.com>:
> > 
> > 
> > On Thu, Feb 7, 2013 at 11:28 PM, Костя Лопухин
> > <konstantin.lopuhin@chtd.ru>
> > wrote:
> > > 
> > > PyPy supports postgres with either psycopg2cffi or psycopg2-ctypes
> > > bindings. We use psycopg2cffi in production (and maintain them),
> > > and
> > > here
> > > http://chtd.ru/blog/bystraya-rabota-s-postgres-pod-pypy/?lang=en
> > > are some benchmarks.
> > > And yes, PyPy is cool :) Typically giving 3x speedups, and some
> > > memory
> > > savings sometimes.
> > > 
> > > 2013/2/7 Gelin Yan <dynamicgl@gmail.com>:
> > > > 
> > > > 
> > > > On Thu, Feb 7, 2013 at 10:11 PM, Phyo Arkar
> > > > <phyo.arkarlwin@gmail.com>
> > > > wrote:
> > > > > 
> > > > > Pypy should have a page for "Success Stories!"
> > > > > 
> > > > > Now with this and Quora proving Power of PyPy , i am beginning
> > > > > to start
> > > > > converting my projects into PyPy soon!
> > > > > I am only withholding right now because my projects uses a lot
> > > > > of C
> > > > > Libraries and Numpy/Matplotlib/scilit-learn.
> > > > > 
> > > > > Thanks
> > > > > 
> > > > > Phyo.
> > > > > 
> > > > > On Thursday, February 7, 2013, Maciej Fijalkowski wrote:
> > > > > > 
> > > > > > On Thu, Feb 7, 2013 at 1:55 PM, Marko Tasic
> > > > > > <mtasic85@gmail.com>
> > > > > > wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > I would like to share short story with you and share what we
> > > > > > > have
> > > > > > > accomplished with PyPy and its friends so far.
> > > > > > > 
> > > > > > > Company that I have worked for last 7 months (intentionally
> > > > > > > unnamed)
> > > > > > > gave me absolute permission to pick up technologies on which
> > > > > > > we
> > > > > > > based
> > > > > > > our solution. What we do is: crawl for PDFs and newspapers
> > > > > > > articles,
> > > > > > > download, translate them if needed, OCR if needed, do
> > > > > > > extensive
> > > > > > > analysis of downloaded PDFs and articles, store them in more
> > > > > > > organized
> > > > > > > structures for faster querying, search for them and generate
> > > > > > > bunch
> > > > > > > of
> > > > > > > complex reports.
> > > > > > > 
> > > > > > > From very beginning I decided to go with PyPy no matter
> > > > > > > what. What
> > > > > > > we
> > > > > > > picked is following:
> > > > > > > * Flask for web framework, and few of its extensions such as
> > > > > > > Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.
> > > > > > > * Cassandra as database because of its features and great
> > > > > > > experience
> > > > > > > with it. PyCassa is used as client to talk to Cassandra
> > > > > > > server.
> > > > > > > * ElasticSearch as distributed search engine, and its client
> > > > > > > library
> > > > > > > pyes.
> > > > > > > * Whoosh as search engine, but with some modifications to
> > > > > > > support
> > > > > > > Cassandra as storage and distributed locking.
> > > > > > > * Redis, and its client library redis-py, for caching and to
> > > > > > > speed
> > > > > > > up
> > > > > > > common auto-completion patterns.
> > > > > > > * ZooKeeper, and its client library Kazoo, for distributed
> > > > > > > locking
> > > > > > > which plays essential role in system for transaction-like
> > > > > > > behavior
> > > > > > > over many services at once.
> > > > > > > * Celery in conjunction with RabbitMQ for task distribution.
> > > > > > > * Sentry for error logging.
> > > > > > > 
> > > > > > > What we have developed on our own are wrappers and clients
> > > > > > > for:
> > > > > > > * Moses which is language translator
> > > > > > > * Tesseract which is OCR engine
> > > > > > > * Cassandra store for Whoosh
> > > > > > > * wkhtmltopdf and wkhtmltoimage which are used for
> > > > > > > conversion of
> > > > > > > HTML
> > > > > > > to PDF/Image
> > > > > > > * etc
> > > > > > > 
> > > > > > > Now when product is finished and in final testing phase, I
> > > > > > > can say
> > > > > > > that we did not regret because we used PyPy and stack around
> > > > > > > it.
> > > > > > > Typical speed improvement is 2x-3x over CPython in our case,
> > > > > > > but
> > > > > > > anyway we are mostly IO and memory bound, expect for Celery
> > > > > > > workers
> > > > > > > where we do analysis which are again many small CPU
> > > > > > > intensive tasks
> > > > > > > that are exchanged via RabbitMQ. Another reason why we don't
> > > > > > > see
> > > > > > > speedup us is that we are dependent on external software
> > > > > > > (servers)
> > > > > > > written in Erlang and Java.
> > > > > > > 
> > > > > > > I'm already planing to do Cassandra (distributed key/value
> > > > > > > only
> > > > > > > database without index features), ZooKeeper, Redis and
> > > > > > > ElasticSearch
> > > > > > > ports in Python for next projects, and hopefully opensource
> > > > > > > them.
> > > > > > > 
> > > > > > > Regards,
> > > > > > > Marko Tasic
> > > > > > > _______________________________________________
> > > > > > > pypy-dev mailing list
> > > > > > > pypy-dev@python.org
> > > > > > > http://mail.python.org/mailman/listinfo/pypy-dev
> > > > > > 
> > > > > > Awesome!
> > > > > > 
> > > > > > I'm glad people can make pypy work for non-trivial tasks which
> > > > > > require
> > > > > > a lot of dependencies. We're trying to lower the bar, however
> > > > > > it takes
> > > > > > time.
> > > > > > 
> > > > > > Cheers,
> > > > > > fijal
> > > > > > _______________________________________________
> > > > > > pypy-dev mailing list
> > > > > > pypy-dev@python.org
> > > > > > http://mail.python.org/mailman/listinfo/pypy-dev
> > > > > 
> > > > > 
> > > > > _______________________________________________
> > > > > pypy-dev mailing list
> > > > > pypy-dev@python.org
> > > > > http://mail.python.org/mailman/listinfo/pypy-dev
> > > > > 
> > > > 
> > > > 
> > > > Hi, It might be off topic. I want to know whether pypy support
> > > > postgres.
> > > > The
> > > > last time I noticed ctypes based psycopg2 was still beta. I
> > > > mainly use
> > > > twisted & postgres. pypy supports twisted well but not good for
> > > > psycopg2.
> > > > 
> > > > Regards
> > > > 
> > > > gelin yan
> > > > 
> > > > _______________________________________________
> > > > pypy-dev mailing list
> > > > pypy-dev@python.org
> > > > http://mail.python.org/mailman/listinfo/pypy-dev
> > > > 
> > > 
> > > 
> > > 
> > > --
> > > Константин Лопухин, разработчик
> > > Компания ЧТД -- http://chtd.ru
> > > +7 (495) 646-87-45, добавочный 333
> > 
> > 
> > Hi
> > 
> > Glad to hear that. I will give it a try. By the way, Can i use
> > it on
> > windows? It looks like cffi support windows.
> > 
> > Regards
> > 
> > gelin yan
> > 
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev@python.org
> > http://mail.python.org/mailman/listinfo/pypy-dev
> > 
> _______________________________________________
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
> 
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

[prev in list] [next in list] [prev in thread] [next in thread] 