[prev in list] [next in list] [prev in thread] [next in thread] 

List:       osgeo-discuss
Subject:    Re: [OSGeo-Discuss] Raster catalog ideas [SEC=UNCLASSIFIED]
From:       "Lucena, Ivan" <ivan.lucena () pmldnet ! com>
Date:       2010-05-04 21:50:58
Message-ID: 4BE096C2.4010001 () pmldnet ! com
[Download RAW message or body]

Hi Mike, Bruce,

We went to that discussion a couple of year ago, and by that time there was some \
strong opinion  against storing raster data on databases, but it look like time has \
changed things a little bit. Am  I right to say that? I can see basically three \
reasons for that.

1. Data volume.

Raster images are getting more and more accessible and whoever deals with big volume \
of those are  starting to feel the pain of managing it on files. I bet that people \
have started to think about why  spending so much money buying and maintaining huge \
Network Attached Storage (NAS) and end up making  a person do the job of a database \
server. They must realize that shared folders accessible by end  users is a problem \
not a solution. Yes, the systematical and disciplinary use of file based storage  is \
possible too, adding to that some sort of raster catalog, but adding to that also the \
trouble of  managing permission access on the NAS, backup procedures, temporary files \
management, and the result  is a full time position or two.

2. Performance.

The myth says: "the is always a overhead to access your raster data from a database \
in comparison to  accessing it directly from the file system". If you are talking \
about a single computer that might  be the case since fopen(), read(), write() \
doesn't need to make connections, check permission,  parse SQL statement to finally \
figure out where to read BLOBs with the same C functions. But as the  complexity of \
your storage grows the file based options will start to experience overhead too and  \
there will be not RDBMS server to do connection pooling or caching for you. Again, \
you can go to the  whole trouble of managing overviews and tile cache yourself. That \
means overhead for a file based  raster data server and more cost of maintaining it. \
Are we tied?

3. Examples

Satellite companies, data provider, government agency and consulting firms are giving \
good testimony  of their use of raster on database, so it might be true for the FOSS \
options too. The two examples  provided by Bruce are pretty good. Rasdaman only works \
with PostgreSQL while Terralib works with  directly with PostgreSQL, MySQL and \
commercial RDBMS systems. Both have some big projects to  showcase their muscles.

Please keep us posted.

Good luck.

Ivan



Bruce Bannerman wrote:
> Hi Mike,
> 
> 
> A product for managing (muti-dimensional) raster data within a database that I've \
> been monitoring since around 2002 has just been released under GPL.  
> They have also applied for OSGeo Incubation.
> 
> This product, Rasdaman, can be used with Postgres as its data store.
> 
> Rasdaman has its lineage going back to around 1995 (I think). They are claiming the \
> French government as a client using the product to manage TB image mosaics. 
> I understand that there is a sister product that will soon be merged in with \
> Rasdaman that offers good OGC support (WCS 1.1 and 2.0, WMS and WCPS). 
> We intend investigating this product as a potential tool for managing grid, model \
> and multi/hyperspectral remotely sensed data relating to the Climate domain.  
> If you look at the product, let me know. I'll be happy to share experiences.
> 
> 
> 
> See [1] for Rasdaman product information and [2] for example implementations.
> 
> 
> ======
> 
> Another option is the Brazillian Terralib Project [3].
> 
> They have developed an integrated suite of server and framework tools for \
> developing integrated applications based around the use of imagery and the \
> management of imagery within a database (Postgres from memory).  
> I'm not doing them justice with this brief description. They have done some very \
> impressive work. I'd recommend Gilberto Camara's intro paper at [4] for an overview \
> of Terralib. 
> 
> =======
> 
> 
> [1] http://www.rasdaman.org/
> 
> [2] www.earthlook.org
> 
> [3] http://www.terralib.org/index.php
> 
> [4] http://www.terralib.org/docs/papers/TerraLib-OSBook-versionJanuary2008.pdf
> 
> 
> 
> 
> Bruce Bannerman
> 
> 
> 
> ________________________________________
> From: discuss-bounces@lists.osgeo.org [discuss-bounces@lists.osgeo.org] On Behalf \
>                 Of Mike Toews [mwtoews@gmail.com]
> Sent: Tuesday, 4 May 2010 6:11 AM
> To: discuss@lists.osgeo.org
> Subject: [OSGeo-Discuss] Raster catalog ideas
> 
> Hi All,
> 
> I'm wondering what existing open source options are available to store rasters with \
> attributes. 
> For example, I have several hundred air photos from northern Canada spanning from \
> the 1940s to now. They have different projections (UTM zones), cell sizes, etc. \
> This data store needs to be accessed from both web and desktop GIS software, but \
> needs to have a definition query for the "year" attribute (so I can take all air \
> photos from 1962, or between 1973 to 1978, or whatever is required). 
> Our (my company) present solution is to use ESRI's raster catalog on a geodatabase. \
> We've had a mixed range of problems on File/Personal/SDE Geodatabases. We've \
> experienced corruption on all levels of storage options, so we keep our file path \
> attributes to the original GeoTIFFs so the raster catalogs can be restored, if \
> required. 
> I'm a bit lost for the available options. The future PostGIS with raster \
> capabilities sounds promising, but I need something that already exists. I don't \
> think a WMS service will work, since it cannot use a query definition (e.g., I \
> don't want to make layers for each year). 
> What are other people doing for large stores of air photos? Thanks in advance.
> 
> -Mike
> _______________________________________________
> Discuss mailing list
> Discuss@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/discuss
> 

_______________________________________________
Discuss mailing list
Discuss@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/discuss


[prev in list] [next in list] [prev in thread] [next in thread]