[prev in list] [next in list] [prev in thread] [next in thread] 

List:       owasp-webscarab
Subject:    Re: [OWASP-WEBSCARAB] Using WebScarab as a Web Capture program...
From:       Rogan Dawes <rogan () dawes ! za ! net>
Date:       2006-01-05 5:47:35
Message-ID: 43BCB2F7.4030008 () dawes ! za ! net
[Download RAW message or body]

David Stidolph wrote:
> Hello, my name is David Stidolph and I was tasked with building a Proxy 
> Server that can capture Web Pages and play them back in the future when 
> the web server is no longer available.  I wrote a Tomcat Servlet that 
> does this for HTTP sites, but it does NOT work for HTTPS sites.

Ok, before we go off the beaten track, lets take a look at what you 
would need to do to get your Tomcat servlet to work.

The basic problem is that when the browser tries to connect to an HTTPS 
server via your tomcat servlet/proxy (I assume that you configure your 
browser to use your servlet as a proxy in some way?), it issues a 
request that looks like:

CONNECT targethost:port HTTP/1.0\r\n
[optional header lines\r\n]
\r\n

It then expects the proxy to give a "200 Ok" response, and connect it 
directly to the target port, where it can negotiate an SSL connection 
with the real web server. If this happens, your servlet will not be able 
to intercept to modify or respond to the requests.

What WebScarab (and any other intercepting proxy that supports HTTPS) 
does is issue a 200 Ok response, but NOT connect the browser to the 
server. It then tries to negotiate an SSL connection with the browser 
directly, so that the browser is talking to WebScarab, rather than the 
actual web server. WebScarab can then do whatever it wants to with the 
request.

I don't know any servlet programming, so I don't know how hard it would 
be for you to implement this. You might want to take a look at the code 
in ConnectionHandler to see how the SSL connection is negotiated.

> 
> WebScarab looks like a way to get this done for me.
> 
> What I need is for the Proxy Server to work in three modes:  
> Passthrough, Record and Playback.
> 
> Normally the mode would be Passthrough so files would pass through 
> without changes or even monitoring.  Recording would be turned on and 
> headers/files would be cached until recording is ended and the files are 
> written into a zip file.  Playback would load the zip file and as each 
> file request comes in it would respond with the file/headers.
> 
> What I have now is a set of classes.  One class wraps up a file (all 
> files - HTML, GIF, CSS, etc) contents, headers and URL.  Another class 
> is a Table Of Contents (TOC) that tracks individual files (everything 
> from HTML to JS to CSS to GIF, etc).  This code is working as a servlet, 
> but I would have to change it to use the WebScarab framework.
> 
> Rogan Dawes and I have talked and he suggested this might be possible as 
> a Bean Shell script.
> 
> I can certainly see how I can record files, but I will also have to 
> block going to the internet for files and provide them myself (for 
> playback and responding to change of state requests like recording to a 
> zip file).
> 
> Even though we will NOT be shipping this code outside the company I work 
> for, I have gotten permission to share it and contribute it back to the 
> project.
> 
> Ok, given this, what are my questions???
> 
> #1 I see the sample Bean Shell script.  This is real java?  I can add a 
> jar file I build outside of it and call methods in it?

Yes, you can. Take a look at www.beanshell.org. The only possibly tricky 
part would be making sure that your classpath is properly set up, so 
that the classes can be found.
> 
> #2 If I do NOT call nextPlugin.fetchResponse can I construct my own 
> Response object to go back to the browser (for playback and responding 
> to commands)?

Exactly.
> 
> #3 How about debugging?  Will System.out.println commands go to the 
> command window or can I use the Java logging commands?

System.out.println will go to the console (command window from which you 
launched WebScarab), or you can use the java.util.logging.Logger 
classes. WebScarab copies any messages in the org.owasp.webscarab 
hierarchy to the Message Log pane in WebScarab as well.

> 
> Thanks for any answers and feedback.
> 
> David Stidolph
> 
> P.S. Please don't assume I know much - this is my second Java program.  
> Mostly I know C++ Windows programming...be gentle.

Regards,

Rogan


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Owasp-webscarab mailing list
Owasp-webscarab@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/owasp-webscarab
[prev in list] [next in list] [prev in thread] [next in thread]