[prev in list] [next in list] [prev in thread] [next in thread] 

List:       loganalysis
Subject:    Re: [logs] syslog/mysql/webGUI.
From:       Chris Adams <cadams () salk ! edu>
Date:       2002-01-30 19:37:34
[Download RAW message or body]

On Wed, Jan 30, 2002 at 09:22:26AM -0500, Marcus J. Ranum wrote:
> Mike Lee wrote:
> >I'm looking for ways to send multiple devices' syslog messages to a mysql
> >database and be able to analyze logs via some sort of web gui.
> 
> 
> Which nicely raises the next thing I wanted to ask:
> Does anyone have any statistics for how many records/second you can
> insert into a MySql database for a reasonable set of assumptions? I.e.:
> assuming one primary index and a reasonably fast machine (say 1Ghz
> with 1GB RAM)*

On decent hardware with a sane schema and tuned mysql, on the order of
thousands.

This includes things like: 
	tuning your mysqld server (esp. throwing some memory at it)

	persistent connections

	using MySQL's optimized insert syntax to batch all available items in a
		single statement (instead of looping 1 INSERT per item) and
		using insert delayed 

	using MyISAM (which is now default) because it allows simultaneous
		selects and inserts but not deletes, which is ideal for
		logging. 

	using the software RAID support to stripe data across multiple devices

	using the MERGE table support so that you insert to, say, a smaller
		daily table and run your aggregate queries across the merge
		table, which avoids touching tables which aren't supposed to
		change.	

---------------------------------------------------------------------
To unsubscribe, e-mail: loganalysis-unsubscribe@securityfocus.com
For additional commands, e-mail: loganalysis-help@securityfocus.com

[prev in list] [next in list] [prev in thread] [next in thread]