'Re: [postgis-users] Point cloud questions (PC NEWBIE here)'

[prev in list] [next in list] [prev in thread] [next in thread] 

List: postgis-users
Subject: Re: [postgis-users] Point cloud questions (PC NEWBIE here)
From: "Newcomb, Doug" <doug_newcomb () fws ! gov>
Date: 2013-10-21 12:03:33
Message-ID: CALQGVr2jwWhu1Or9KZyR6dNg8FfZ22pRPGE7dbgdtO1t8Q39fw () mail ! gmail ! com
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]

Bob,
I've had a fair amount of success directly generating rasters from the z
values  from LAS files of 3.3 billion points each using GRASS7 complied
with liblas and the r.in.lidar,
http://grass.osgeo.org/grass70/manuals/r.in.lidar.html command.

As Remi mentioned, it can be quite memory intensive,
http://www.ncgisconference.com/2013/documents/pdfs/Newcomb_Fri_1030.pdf .
Point counts and Range calculations are fairly cheap, but skewness analysis
can take a LOT of memory.  For the 3.3 billion point data sets on partial
coverage of a 753 million cell raster and limiting the amount of map in
memory that was processed at one time to 15%, it still took 20GB of RAM  to
process for a skewness analysis.  The extent of the raster created was much
larger than the area covered by points, so that probably made some
difference.  Only processing an area for which there are points might cut
down on the memory requirements somewhat.

Doug

On Mon, Oct 21, 2013 at 4:52 AM, Rémi Cura <remi.cura@gmail.com> wrote:

> Hey,
> I'm not M.Ramsey,  I'm just a pointcloud user.
>
> 1. : you can "colorize" based on any attributes of points. Traditional
> point cloud softwares expect usually R G B attributes, but for some you can
> choose which attributes to use for coloring (CloudCompare).
>
> 2 : We have thought about visualisation (in this area it's about Level Of
> Details). The best we tried (because of lack of time) is to make another
> PC_Explode so it returns only a certain number of points per patch.
>
> The next step is to writte points in patch following a quad tree structure
> (not deep first but breadth-first) (maybe using
> http://www.postgresql.org/docs/9.3/static/spgist-examples.html , not sure
> about this).
> This way when taking only the first XX points in patches, you ensure these
> points are always well distributed in the patch.
>
> This is the best we thought of without duplicating data (not an option
> with so big data anyway).
>
> 3 : about générating a raster from pointcloud :
> You can do it in QGis , Grass Gis, or using GDAL or LasTool. (also in
> CloudCompare) Usually you have access to sophisticated interpolation
> methods (square inverse distance, etc etc). It is a hard problem in
> practice because of memory limitation and possible multiple holes in data.
>
>
> Please feel free to ask if you want more details.
>
> Cheers,
>
> Rémi-C
>
>
>
>
> 2013/10/18 Basques, Bob (CI-StPaul) <bob.basques@ci.stpaul.mn.us>
>
>>  Hi all,****
>>
>> ** **
>>
>> Juat watched Paul Ramseys video on point cloud storage and it made me
>> think of a few questions to ask.****
>>
>> ** **
>>
>> Quesitions:****
>>
>> **·          **Sort of a DB processing question, probably not thought
>> about yet, but can the pointcloud be colorized based on a raster/vector of
>> view passed in?  As in aerial photos obviously, but less obvious would be
>> for oblique photos.  ****
>>
>> **·          **Generalizing of the pointclouds, from my perspective to
>> be used for zoom in/out processes for visualization.  Could a average
>> height based on a (pyramid) tiling possibly be used to get close to a
>> generalizing scheme, I know this approach might offend purists as far as
>> data integrity . .****
>>
>> **·          **There was a question in the video that I watched related
>> to building a surface grid from the pointcloud, I wonder about a simple
>> tile averaging as a method for generating a grid.  Anyone tried and have
>> available some benchmarks for this idea?****
>>
>> ** **
>>
>> Thanks****
>>
>> ** **
>>
>> Bobb****
>>
>> ** **
>>
>> ** **
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@lists.osgeo.org
>> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
>>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
>

-- 
Doug Newcomb
USFWS
Raleigh, NC
919-856-4520 ext. 14 doug_newcomb@fws.gov
---------------------------------------------------------------------------------------------------------
The opinions I express are my own and are not representative of the
official policy of the U.S.Fish and Wildlife Service or Dept. of the
Interior.   Life is too short for undocumented, proprietary data formats.

[Attachment #5 (text/html)]

<div dir="ltr">Bob,<div>I&#39;ve had a fair amount of success directly generating \
rasters from the z values from LAS files of 3.3 billion points each using GRASS7 \
complied with liblas and the r.in.lidar, <a \
href="http://grass.osgeo.org/grass70/manuals/r.in.lidar.html">http://grass.osgeo.org/grass70/manuals/r.in.lidar.html</a> \
command. </div>

<div> </div><div>As Remi mentioned, it can be quite memory intensive, <a \
href="http://www.ncgisconference.com/2013/documents/pdfs/Newcomb_Fri_1030.pdf">http://www.ncgisconference.com/2013/documents/pdfs/Newcomb_Fri_1030.pdf</a> \
. Point counts and Range calculations are fairly cheap, but skewness analysis can \
take a LOT of memory. For the 3.3 billion point data sets on partial coverage of a \
753 million cell raster and limiting the amount of map in memory that was processed \
at one time to 15%, it still took 20GB of RAM to process for a skewness analysis. \
The extent of the raster created was much larger than the area covered by points, so \
that probably made some difference. Only processing an area for which there are \
points might cut down on the memory requirements somewhat.</div>

<div><br></div><div>Doug</div><div> </div><div><br></div></div><div \
class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Oct 21, 2013 at 4:52 AM, \
Rémi Cura <span dir="ltr">&lt;<a href="mailto:remi.cura@gmail.com" \
target="_blank">remi.cura@gmail.com</a>&gt;</span> wrote:<br>

<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div><div>Hey, </div><div>I&#39;m not \
M.Ramsey, I&#39;m just a pointcloud user.</div> 1. : you can &quot;colorize&quot; \
based on any attributes of points. Traditional point cloud softwares expect usually R \
G B attributes, but for some you can choose which attributes to use for coloring \
(CloudCompare).

</div>2 : We have thought about visualisation (in this area it&#39;s about Level \
Of Details). The best we tried (because of lack of time) is to make another \
PC_Explode so it returns only a certain number of points per patch.

<div><br></div><div>The next step is to writte points in patch following a quad tree \
structure (not deep first but breadth-first) (maybe using <a \
href="http://www.postgresql.org/docs/9.3/static/spgist-examples.html" \
target="_blank">http://www.postgresql.org/docs/9.3/static/spgist-examples.html</a> , \
not sure about this).</div>

<div>This way when taking only the first XX points in patches, you ensure these \
points are always well distributed in the patch.</div><div><br></div><div>This is the \
best we thought of without duplicating data (not an option with so big data \
anyway).</div>

<div> </div><div>3 : about générating a raster from pointcloud :</div><div>You can \
do it in QGis , Grass Gis, or using GDAL or LasTool. (also in CloudCompare) Usually \
you have access to sophisticated interpolation methods (square inverse distance, etc \
etc). It is a hard problem in practice because of memory limitation and possible \
multiple holes in data.

<br><br>Please feel free to ask if you want more \
details.<br><br>Cheers,</div><div><br></div><div>Rémi-C</div><div><br></div><div><br></div></div><div \
class="gmail_extra"><br><br><div class="gmail_quote">2013/10/18 Basques, Bob \
(CI-StPaul) <span dir="ltr">&lt;<a href="mailto:bob.basques@ci.stpaul.mn.us" \
target="_blank">bob.basques@ci.stpaul.mn.us</a>&gt;</span><br>

<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div><div class="h5">

<div lang="EN-US" link="blue" vlink="purple">
<div>
Hi all, \
 Juat watched Paul \
Ramseys video on point cloud storage and it made me think of a few questions to \
ask. \
 Quesitions: · Sort of a DB processing \
question, probably not thought about yet, but can the pointcloud be colorized based \
on a raster/vector of view passed in? As in aerial photos obviously, but less \
obvious would be for oblique photos. · Generalizing of the \
pointclouds, from my perspective to be used for zoom in/out processes for \
visualization. Could a average height based on a (pyramid) tiling possibly be used \
to get close to a generalizing scheme, I know this approach might offend purists as \
far as data integrity . . · There was a question in \
the video that I watched related to building a surface grid from the pointcloud, I \
wonder about a simple tile averaging as a method for generating a grid. Anyone \
tried and have available some benchmarks for this idea? Thanks \
 Bobb \
 \
 </div>
</div>

<br></div></div>_______________________________________________<br>
postgis-users mailing list<br>
<a href="mailto:postgis-users@lists.osgeo.org" \
target="_blank">postgis-users@lists.osgeo.org</a><br> <a \
href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users" \
target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users</a><br></blockquote></div><br></div>
 <br>_______________________________________________<br>
postgis-users mailing list<br>
<a href="mailto:postgis-users@lists.osgeo.org">postgis-users@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users" \
target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users</a><br></blockquote></div><br><br \
clear="all"><div><br></div>-- <br><div>

Doug Newcomb</div><div>USFWS</div><div>Raleigh, NC</div><div>919-856-4520 ext. 14 <a \
href="mailto:doug_newcomb@fws.gov" \
target="_blank">doug_newcomb@fws.gov</a></div><div>---------------------------------------------------------------------------------------------------------</div>

<div>The opinions I express are my own and are not representative of the official \
policy of the U.S.Fish and Wildlife Service or Dept. of the Interior. Life is too \
short for undocumented, proprietary data formats.</div>

</div>

_______________________________________________
postgis-users mailing list
postgis-users@lists.osgeo.org
http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users

[prev in list] [next in list] [prev in thread] [next in thread] 