'Re: [pypy-dev] Great experience with PyPy'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       pypy-dev
Subject:    Re: [pypy-dev] Great experience with PyPy
From:       David Fraser <davidf () sjsoft ! com>
Date:       2013-02-13 7:53:10
Message-ID: 24890242.381.1360741988037.JavaMail.davidf () jackdaw ! local
[Download RAW message or body]

You may also want to try pg8000; this is a pure-Python driver that works on Windows

On Thursday, February 7, 2013 at 6:08:51 PM, "Костя Лопухин" <kostia.lopuhin@gmail.com> wrote:
> Hi! I did not test it on Windows, there may be problems with
> installation (searching for postgres header files, the config is not
> very smart -
> https://github.com/chtd/psycopg2cffi/blob/master/psycopg2cffi/_impl/libpq.py#L209),
> but they should be solvable I hope - submit a bug if you have
> problems.
> 
> 2013/2/7 Gelin Yan <dynamicgl@gmail.com>:
> >
> >
> > On Thu, Feb 7, 2013 at 11:28 PM, Костя Лопухин
> > <konstantin.lopuhin@chtd.ru>
> > wrote:
> >>
> >> PyPy supports postgres with either psycopg2cffi or psycopg2-ctypes
> >> bindings. We use psycopg2cffi in production (and maintain them),
> >> and
> >> here
> >> http://chtd.ru/blog/bystraya-rabota-s-postgres-pod-pypy/?lang=en
> >> are some benchmarks.
> >> And yes, PyPy is cool :) Typically giving 3x speedups, and some
> >> memory
> >> savings sometimes.
> >>
> >> 2013/2/7 Gelin Yan <dynamicgl@gmail.com>:
> >> >
> >> >
> >> > On Thu, Feb 7, 2013 at 10:11 PM, Phyo Arkar
> >> > <phyo.arkarlwin@gmail.com>
> >> > wrote:
> >> >>
> >> >> Pypy should have a page for "Success Stories!"
> >> >>
> >> >> Now with this and Quora proving Power of PyPy , i am beginning
> >> >> to start
> >> >> converting my projects into PyPy soon!
> >> >> I am only withholding right now because my projects uses a lot
> >> >> of C
> >> >> Libraries and Numpy/Matplotlib/scilit-learn.
> >> >>
> >> >> Thanks
> >> >>
> >> >> Phyo.
> >> >>
> >> >> On Thursday, February 7, 2013, Maciej Fijalkowski wrote:
> >> >>>
> >> >>> On Thu, Feb 7, 2013 at 1:55 PM, Marko Tasic
> >> >>> <mtasic85@gmail.com>
> >> >>> wrote:
> >> >>> > Hi,
> >> >>> >
> >> >>> > I would like to share short story with you and share what we
> >> >>> > have
> >> >>> > accomplished with PyPy and its friends so far.
> >> >>> >
> >> >>> > Company that I have worked for last 7 months (intentionally
> >> >>> > unnamed)
> >> >>> > gave me absolute permission to pick up technologies on which
> >> >>> > we
> >> >>> > based
> >> >>> > our solution. What we do is: crawl for PDFs and newspapers
> >> >>> > articles,
> >> >>> > download, translate them if needed, OCR if needed, do
> >> >>> > extensive
> >> >>> > analysis of downloaded PDFs and articles, store them in more
> >> >>> > organized
> >> >>> > structures for faster querying, search for them and generate
> >> >>> > bunch
> >> >>> > of
> >> >>> > complex reports.
> >> >>> >
> >> >>> > From very beginning I decided to go with PyPy no matter
> >> >>> > what. What
> >> >>> > we
> >> >>> > picked is following:
> >> >>> > * Flask for web framework, and few of its extensions such as
> >> >>> > Flask-Login, Flask-Principal, Flask-WTF, Flask-Mail, etc.
> >> >>> > * Cassandra as database because of its features and great
> >> >>> > experience
> >> >>> > with it. PyCassa is used as client to talk to Cassandra
> >> >>> > server.
> >> >>> > * ElasticSearch as distributed search engine, and its client
> >> >>> > library
> >> >>> > pyes.
> >> >>> > * Whoosh as search engine, but with some modifications to
> >> >>> > support
> >> >>> > Cassandra as storage and distributed locking.
> >> >>> > * Redis, and its client library redis-py, for caching and to
> >> >>> > speed
> >> >>> > up
> >> >>> > common auto-completion patterns.
> >> >>> > * ZooKeeper, and its client library Kazoo, for distributed
> >> >>> > locking
> >> >>> > which plays essential role in system for transaction-like
> >> >>> > behavior
> >> >>> > over many services at once.
> >> >>> > * Celery in conjunction with RabbitMQ for task distribution.
> >> >>> > * Sentry for error logging.
> >> >>> >
> >> >>> > What we have developed on our own are wrappers and clients
> >> >>> > for:
> >> >>> > * Moses which is language translator
> >> >>> > * Tesseract which is OCR engine
> >> >>> > * Cassandra store for Whoosh
> >> >>> > * wkhtmltopdf and wkhtmltoimage which are used for
> >> >>> > conversion of
> >> >>> > HTML
> >> >>> > to PDF/Image
> >> >>> > * etc
> >> >>> >
> >> >>> > Now when product is finished and in final testing phase, I
> >> >>> > can say
> >> >>> > that we did not regret because we used PyPy and stack around
> >> >>> > it.
> >> >>> > Typical speed improvement is 2x-3x over CPython in our case,
> >> >>> > but
> >> >>> > anyway we are mostly IO and memory bound, expect for Celery
> >> >>> > workers
> >> >>> > where we do analysis which are again many small CPU
> >> >>> > intensive tasks
> >> >>> > that are exchanged via RabbitMQ. Another reason why we don't
> >> >>> > see
> >> >>> > speedup us is that we are dependent on external software
> >> >>> > (servers)
> >> >>> > written in Erlang and Java.
> >> >>> >
> >> >>> > I'm already planing to do Cassandra (distributed key/value
> >> >>> > only
> >> >>> > database without index features), ZooKeeper, Redis and
> >> >>> > ElasticSearch
> >> >>> > ports in Python for next projects, and hopefully opensource
> >> >>> > them.
> >> >>> >
> >> >>> > Regards,
> >> >>> > Marko Tasic
> >> >>> > _______________________________________________
> >> >>> > pypy-dev mailing list
> >> >>> > pypy-dev@python.org
> >> >>> > http://mail.python.org/mailman/listinfo/pypy-dev
> >> >>>
> >> >>> Awesome!
> >> >>>
> >> >>> I'm glad people can make pypy work for non-trivial tasks which
> >> >>> require
> >> >>> a lot of dependencies. We're trying to lower the bar, however
> >> >>> it takes
> >> >>> time.
> >> >>>
> >> >>> Cheers,
> >> >>> fijal
> >> >>> _______________________________________________
> >> >>> pypy-dev mailing list
> >> >>> pypy-dev@python.org
> >> >>> http://mail.python.org/mailman/listinfo/pypy-dev
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> pypy-dev mailing list
> >> >> pypy-dev@python.org
> >> >> http://mail.python.org/mailman/listinfo/pypy-dev
> >> >>
> >> >
> >> >
> >> > Hi, It might be off topic. I want to know whether pypy support
> >> > postgres.
> >> > The
> >> > last time I noticed ctypes based psycopg2 was still beta. I
> >> > mainly use
> >> > twisted & postgres. pypy supports twisted well but not good for
> >> > psycopg2.
> >> >
> >> > Regards
> >> >
> >> > gelin yan
> >> >
> >> > _______________________________________________
> >> > pypy-dev mailing list
> >> > pypy-dev@python.org
> >> > http://mail.python.org/mailman/listinfo/pypy-dev
> >> >
> >>
> >>
> >>
> >> --
> >> Константин Лопухин, разработчик
> >> Компания ЧТД -- http://chtd.ru
> >> +7 (495) 646-87-45, добавочный 333
> >
> >
> > Hi
> >
> >    Glad to hear that. I will give it a try. By the way, Can i use
> >    it on
> > windows? It looks like cffi support windows.
> >
> > Regards
> >
> > gelin yan
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev@python.org
> > http://mail.python.org/mailman/listinfo/pypy-dev
> >
> _______________________________________________
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

[prev in list] [next in list] [prev in thread] [next in thread] 