[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wwwsearch-general
Subject:    Re: [wwwsearch-general] using mechanize in google app engine
From:       John J Lee <jjl () pobox ! com>
Date:       2009-06-07 16:07:08
Message-ID: alpine.DEB.2.00.0906071637060.14080 () alice
[Download RAW message or body]

In that case, I imagine you want to replace HTTP{S,}Handler with something 
that uses GAE's urllib2.HTTP{S,}Handler.  You might have significant pain 
with response objects, I'm afraid -- note the last few lines of 
mechanize._http.AbstractHTTPHandler.do_open() .  One would hope that you 
just need something like this (https probably requires a bit more 
complication).  Of course, this is all a bit flaky -- use of private 
interface, subclassing.


# untested

import copy

import mechanize
import mechanize._response


class GAEMechanizeHTTPHandler(mechanize.BaseHandler):

     def __init__(self, gae_http_handler, debuglevel=0):
         self._debuglevel = debuglevel
         self._gae_http_handler = gae_http_handler

     def http_open(self, req):
         return mechanize._response.closeable_response(
             self._gae_http_handler.do_open(req))


def make_gae_handler():
     # XXX return an  instance of GAE's urllib2.HTTPHandler


def make_http_handler():
     gae_handler = make_gae_handler()
     return GAEMechanizeHTTPHandler(gae_handler)


class GAEBrowser(mechanize.Browser):

     handler_classes = copy.copy(mechanize.Browser.handler_classes)
     handler_classes["http"] = make_http_handler


browser = GAEBrowser()
print browser.open("http://python.org").read()


John

On Fri, 5 Jun 2009, Mark Stoddard wrote:

> google app engine supports use of urllib2, so I think that what I must use.
> I'll look into your suggestion
>
> On Fri, Jun 5, 2009 at 1:11 PM, John J Lee <jjl@pobox.com> wrote:
>
>> On Thu, 4 Jun 2009, Mark Stoddard wrote:
>>
>>> I'm looking to use mechanize in a cron job in google app engine.  The
>>> problem is that mechanize uses sockets, and google doesn't allow that.
>>  I'm
>>> looking for some high(er) level insight into what would need to be done
>> to
>>> support that.
>> [...]
>>
>> What do you want to use instead of sockets?
>>
>> You probably want to replace HTTPHandler (and HTTPSHandler).  Currently
>> that would require subclassing mechanize.Browser, which is fragile.
>> Alternatively, you can monkey-patch mechanize.HTTP{S,}Handler.
>>
>>
>> John
>>
>>
>>
>> ------------------------------------------------------------------------------
>> OpenSolaris 2009.06 is a cutting edge operating system for enterprises
>> looking to deploy the next generation of Solaris that includes the latest
>> innovations from Sun and the OpenSource community. Download a copy and
>> enjoy capabilities such as Networking, Storage and Virtualization.
>> Go to: http://p.sf.net/sfu/opensolaris-get
>> _______________________________________________
>> wwwsearch-general mailing list
>> wwwsearch-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/wwwsearch-general
>>
>


------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises 
looking to deploy the next generation of Solaris that includes the latest 
innovations from Sun and the OpenSource community. Download a copy and 
enjoy capabilities such as Networking, Storage and Virtualization. 
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
wwwsearch-general mailing list
wwwsearch-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wwwsearch-general
[prev in list] [next in list] [prev in thread] [next in thread]