[prev in list] [next in list] [prev in thread] [next in thread] 

List:       mysql
Subject:    Re: identifying rows that have changed
From:       "Perrin Harkins" <perrin () elem ! com>
Date:       2007-05-30 16:28:02
Message-ID: 66887a3d0705300928t53d8575dw8ab5b09d5bfe995a () mail ! gmail ! com
[Download RAW message or body]

On 5/30/07, Dan Buettner <drbuettner@gmail.com> wrote:
> #1 - it's not a good approach to "hope" your database keeps up.  There are
> fairly common situations that can come up where you never know how long
> something will take - unusually high traffic, table check and repair, a bulk
> load into the same or another database on that db host, etc.

Yes, I expect it would "drift" some over time.  I might need to do a
full reload weekly.  It isn't really a problem for me to process the
same rows twice, but if I make the time window really large it defeats
the purpose of the incremental processing, i.e. keeping the working
data set small.

> One interesting gotcha would be trying to ensure that whatever updates your
> batch process does - do not cause additional entries in the "needs to be
> post processed" table, causing an endless loop...

In my case, this is easy, since I don't write anything back to the
source tables.  The results go into a separate database.

Thanks for your input.

- Perrin

-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:    http://lists.mysql.com/mysql?unsub=mysql-marcsub@progressive-comp.com

[prev in list] [next in list] [prev in thread] [next in thread]