[prev in list] [next in list] [prev in thread] [next in thread]
List: paraview
Subject: Re: [Paraview] Volume Rendering 17GB 8.5 billion cell volume
From: Aashish Chaudhary <aashish.chaudhary () kitware ! com>
Date: 2015-10-21 14:00:03
Message-ID: CAEw35Z_xcMViNofvzGnrTwdwVx7VyKKhpZVOMBrv=x_5OXWPiA () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Ah, that's worth reporting to Mesa developers like you suggested. Please
let me know (or keep me in the loop) what they have to say on this. I am
surprised that it wasn't reported earlier.
- Aashish
On Wed, Oct 21, 2015 at 9:57 AM, Berk Geveci <berk.geveci@kitware.com>
wrote:
> > No GL error this time I am assuming? Just segfault?
>
> Yes. With the stack totally hosed. I also see Valgrind errors deep in Mesa
> stack.
>
>
> On Wed, Oct 21, 2015 at 9:15 AM, Aashish Chaudhary <
> aashish.chaudhary@kitware.com> wrote:
>
>> On Wed, Oct 21, 2015 at 7:52 AM, Berk Geveci <berk.geveci@kitware.com>
>> wrote:
>>
>>> Yeah, it is not that easy. If you compile Mesa with llvmpipe, the limit
>>> is in something like gallium/drivers/llvmpipe/lp_limits.h. Otherwise it
>>> doesn't work. However, I had crashes when I increased that to something
>>> like 8 GBs and then volume rendered something large.
>>>
>>
>> No GL error this time I am assuming? Just segfault?
>>
>>
>>> It also looks like other drivers have their own limits that are pretty
>>> small. I will talk to some folks doing work with Mesa about this.
>>> Hopefully, we can address it in the upcoming OpenSWR driver. We'll have to
>>> support streaming for other drivers though...
>>>
>>
>> Absolutely.
>>
>>>
>>> -berk
>>>
>>>
>>> On Tue, Oct 20, 2015 at 5:55 PM, Aashish Chaudhary <
>>> aashish.chaudhary@kitware.com> wrote:
>>>
>>>> Here it is. It would be great if someone else try it as well:
>>>>
>>>> ----Steps----
>>>> 1. In src/mesa/main/config.h, there should MAX_TEXTURE_MBYTES defined.
>>>> I believe by default it was 1024 Mbytes. Please change it to 4096 or
>>>> something higher.
>>>>
>>>> 2. Then Compile and Install MESA again (do not forget to set the
>>>> MESA_GL_VERSION_OVERRIDE to 3.2).
>>>>
>>>> 3. Compile paraview again (server)
>>>>
>>>>
>>>> On Tue, Oct 20, 2015 at 5:48 PM, Aashish Chaudhary <
>>>> aashish.chaudhary@kitware.com> wrote:
>>>>
>>>>> Berk,
>>>>>
>>>>> On Tue, Oct 20, 2015 at 4:00 PM, Berk Geveci <berk.geveci@kitware.com>
>>>>> wrote:
>>>>>
>>>>>> Hi folks,
>>>>>>
>>>>>> I wanted to close the loop on this. Here are my findings:
>>>>>>
>>>>>> * ParaView master (4.4 should also do) + OpenGL2 + NVIDIA Tesla w 12
>>>>>> GB memory: I verified that I can volume render data up to the capacity of
>>>>>> the card. I could volume render a 1400x1400x1400 volume of floats.
>>>>>>
>>>>>> * ParaView master (4.4 should also do) + OpenGL2 + Mesa (OSMesa 11,
>>>>>> llvmpipe, swrast): Mesa has some fairly small limits on 3D texture size,
>>>>>> which is what we use for volume rendering. So, ~ 1000x1000x1000 will be the
>>>>>> upper end of what can be done for now. In time, we will implement multiple
>>>>>> textures / streaming to enable rendering of larger volumes.
>>>>>>
>>>>>
>>>>> Did you see my other email? You can change the default for OSMesa. I
>>>>> sent it last week.
>>>>>
>>>>> - Aashish
>>>>>
>>>>>>
>>>>>> Best,
>>>>>> -berk
>>>>>>
>>>>>> On Mon, Sep 28, 2015 at 11:00 AM, David Trudgian <
>>>>>> David.Trudgian@utsouthwestern.edu> wrote:
>>>>>>
>>>>>>> Berk,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks very much for looking into this. Look forward to trying
>>>>>>> things out whenever they're ready.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> DT
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> David Trudgian Ph.D.
>>>>>>> Computational Scientist, BioHPC
>>>>>>> UT Southwestern Medical Center
>>>>>>> Dallas, TX 75390-9039
>>>>>>> Tel: (214) 648-4833
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *From:* Berk Geveci [mailto:berk.geveci@kitware.com]
>>>>>>> *Sent:* Monday, September 28, 2015 9:58 AM
>>>>>>>
>>>>>>> *To:* David Trudgian <David.Trudgian@UTSouthwestern.edu>
>>>>>>> *Cc:* ParaView Mailing List <paraview@paraview.org>
>>>>>>> *Subject:* Re: [Paraview] Volume Rendering 17GB 8.5 billion cell
>>>>>>> volume
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi David,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I have been trying to find some cycles to check this out myself with
>>>>>>> ParaView 4.4. Thanks to hardware issues (i.e. my big workstation's disk
>>>>>>> dying), I haven't been able to. Good news is that I found issues with
>>>>>>> OSMesa + OpenGL2 that we are working through. Give me another 1-1.5 weeks.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Best,
>>>>>>>
>>>>>>> -berk
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Sep 28, 2015 at 10:46 AM, David Trudgian <
>>>>>>> David.Trudgian@utsouthwestern.edu> wrote:
>>>>>>>
>>>>>>> Hi Berk,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Finally managed to grab an allocation of some Tesla K40 nodes on our
>>>>>>> cluster, to check GPU rendering of the full 17GB file with 2 x 12GB GPUs. I
>>>>>>> see the same thing as I did with OSMesa rendering.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> The 9GB downsampled version works great, across 2 nodes both with a
>>>>>>> single K40. Go up to the 17GB original file and nothing is rendered, no
>>>>>>> errors. Same behavior with OPENGL or OPENGL2 backends.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> This is all on paraview 4.3.1 still – I need to find time to build
>>>>>>> OSMesa / MPI versions of 4.4 here. But, does 4.4. have any fixes that would
>>>>>>> be expected to affect this?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> David Trudgian Ph.D.
>>>>>>> Computational Scientist, BioHPC
>>>>>>> UT Southwestern Medical Center
>>>>>>> Dallas, TX 75390-9039
>>>>>>> Tel: (214) 648-4833
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *From:* Berk Geveci [mailto:berk.geveci@kitware.com]
>>>>>>> *Sent:* Tuesday, September 15, 2015 2:43 PM
>>>>>>> *To:* David Trudgian <David.Trudgian@UTSouthwestern.edu>
>>>>>>> *Cc:* ParaView Mailing List <paraview@paraview.org>
>>>>>>> *Subject:* Re: [Paraview] Volume Rendering 17GB 8.5 billion cell
>>>>>>> volume
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hey David,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I am hoping to have some time to play around with volume rendering
>>>>>>> and hopefully tracking this issue, one thing that I wanted to clarify: it
>>>>>>> sounds from you description that you have a short (2 byte) value. Is that
>>>>>>> correct?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -berk
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 9, 2015 at 5:00 PM, David Trudgian <
>>>>>>> david.trudgian@utsouthwestern.edu> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> We have been experimenting with using Paraview to display very
>>>>>>> volumes from very
>>>>>>> large TIFF stacks generated by whole-brain microscopy equipment. The
>>>>>>> test stack
>>>>>>> has dimensions of 5,368x10,695x150. Stack is assembled in ImageJ
>>>>>>> from individual
>>>>>>> TIFFs, exported as a RAW and loaded into paraview. Saved as a .vti
>>>>>>> for
>>>>>>> convenience. Can view slices fine in standalone paraview client on a
>>>>>>> 256GB machine.
>>>>>>>
>>>>>>> When we attempt volume rendering on this data across multiple nodes
>>>>>>> with MPI
>>>>>>> nothing appears in the client. Surface view works as expected. On
>>>>>>> switching to
>>>>>>> volume rendering the client's display will show nothing. There are
>>>>>>> no messages
>>>>>>> from the client or servers - no output.
>>>>>>>
>>>>>>> This is happening when running pvserver across GPU nodes with NVIDIA
>>>>>>> Tesla
>>>>>>> cards, or using CPU only with OSMESA. pvserver memory usage is well
>>>>>>> below what
>>>>>>> we have on the nodes - no memory warnings/errors.
>>>>>>>
>>>>>>> Data is about 17GB, 8 billion cells. If we downsize to ~4GB or ~9GB
>>>>>>> then we can
>>>>>>> get working volume rendering. The 17GB never works regardless of
>>>>>>> scaling
>>>>>>> nodes/mpi processes. The 4/9GB will work on 1 or 2 nodes.
>>>>>>>
>>>>>>> Am confused by the lack of rendering, as we don't have memory
>>>>>>> issues, or an
>>>>>>> other messages at all. Am wondering if there are any inherent
>>>>>>> limitation, or I'm
>>>>>>> missing something stupid.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Dave Trudgian
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Powered by www.kitware.com
>>>>>>>
>>>>>>> Visit other Kitware open-source projects at
>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>
>>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>>>> http://paraview.org/Wiki/ParaView
>>>>>>>
>>>>>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>>>>>
>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> *UT** Southwestern*
>>>>>>>
>>>>>>> Medical Center
>>>>>>>
>>>>>>> The future of medicine, today.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Powered by www.kitware.com
>>>>>>
>>>>>> Visit other Kitware open-source projects at
>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>
>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>>> http://paraview.org/Wiki/ParaView
>>>>>>
>>>>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>>>>
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>>
>>>>>
>>>>> *| Aashish Chaudhary | Technical Leader | Kitware Inc.
>>>>> *
>>>>> *| http://www.kitware.com/company/team/chaudhary.html
>>>>> <http://www.kitware.com/company/team/chaudhary.html>*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>> *| Aashish Chaudhary | Technical Leader | Kitware Inc.
>>>> *
>>>> *| http://www.kitware.com/company/team/chaudhary.html
>>>> <http://www.kitware.com/company/team/chaudhary.html>*
>>>>
>>>
>>>
>>
>>
>> --
>>
>>
>>
>> *| Aashish Chaudhary | Technical Leader | Kitware Inc.
>> *
>> *| http://www.kitware.com/company/team/chaudhary.html
>> <http://www.kitware.com/company/team/chaudhary.html>*
>>
>
>
--
*| Aashish Chaudhary | Technical Leader | Kitware Inc. *
*| http://www.kitware.com/company/team/chaudhary.html
<http://www.kitware.com/company/team/chaudhary.html>*
[Attachment #5 (text/html)]
<div dir="ltr">Ah, that's worth reporting to Mesa developers like you suggested. \
Please let me know (or keep me in the loop) what they have to say on this. I am \
surprised that it wasn't reported earlier. <div><br></div><div>- \
Aashish</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Oct \
21, 2015 at 9:57 AM, Berk Geveci <span dir="ltr"><<a \
href="mailto:berk.geveci@kitware.com" \
target="_blank">berk.geveci@kitware.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><span class="">> No GL error this time I \
am assuming? Just segfault?<div><br></div></span><div>Yes. With the stack totally \
hosed. I also see Valgrind errors deep in Mesa stack.<div><div class="h5"><br><div \
class="gmail_extra"><br><div class="gmail_quote">On Wed, Oct 21, 2015 at 9:15 AM, \
Aashish Chaudhary <span dir="ltr"><<a href="mailto:aashish.chaudhary@kitware.com" \
target="_blank">aashish.chaudhary@kitware.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div \
dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Wed, Oct 21, \
2015 at 7:52 AM, Berk Geveci <span dir="ltr"><<a \
href="mailto:berk.geveci@kitware.com" \
target="_blank">berk.geveci@kitware.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div \
dir="ltr">Yeah, it is not that easy. If you compile Mesa with llvmpipe, the limit is \
in something like gallium/drivers/llvmpipe/lp_limits.h. Otherwise it doesn't \
work. However, I had crashes when I increased that to something like 8 GBs and then \
volume rendered something large. </div></blockquote><div><br></div></span><div>No GL \
error this time I am assuming? Just segfault?</div><span><div> </div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div \
dir="ltr">It also looks like other drivers have their own limits that are pretty \
small. I will talk to some folks doing work with Mesa about this. Hopefully, we can \
address it in the upcoming OpenSWR driver. We'll have to support streaming for \
other drivers though...</div></blockquote><div><br></div></span><div>Absolutely. \
</div><div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div \
dir="ltr"><span><font \
color="#888888"><div><br></div><div>-berk<br><div><br></div></div></font></span></div><div><div><div \
class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 20, 2015 at 5:55 PM, \
Aashish Chaudhary <span dir="ltr"><<a href="mailto:aashish.chaudhary@kitware.com" \
target="_blank">aashish.chaudhary@kitware.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div \
dir="ltr">Here it is. It would be great if someone else try it as well: \
<div><br><div><div style="font-size:16px">----Steps----</div><div \
style="font-size:16px">1. In src/mesa/main/config.h, there should MAX_TEXTURE_MBYTES \
defined. I believe by default it was 1024 Mbytes. Please change it to 4096 or \
something higher. </div><div style="font-size:16px"><br></div><div \
style="font-size:16px">2. Then Compile and Install MESA again (do not forget to set \
the <span style="color:rgb(0,0,0);font-family:'Lucida \
Grande',Geneva,Arial,Verdana,sans-serif;font-size:14px">MESA_GL_VERSION_OVERRIDE</span> \
to 3.2). </div><div style="font-size:16px"><br></div><div style="font-size:16px">3. \
Compile paraview again (server)</div><div \
style="font-size:16px"><br></div></div></div></div><div><div><div \
class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 20, 2015 at 5:48 PM, \
Aashish Chaudhary <span dir="ltr"><<a href="mailto:aashish.chaudhary@kitware.com" \
target="_blank">aashish.chaudhary@kitware.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div \
dir="ltr">Berk, <br><div class="gmail_extra"><br><div class="gmail_quote"><span>On \
Tue, Oct 20, 2015 at 4:00 PM, Berk Geveci <span dir="ltr"><<a \
href="mailto:berk.geveci@kitware.com" \
target="_blank">berk.geveci@kitware.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div \
dir="ltr">Hi folks,<div><br></div><div>I wanted to close the loop on this. Here are \
my findings:</div><div><br></div><div>* ParaView master (4.4 should also do) + \
OpenGL2 + NVIDIA Tesla w 12 GB memory: I verified that I can volume render data up to \
the capacity of the card. I could volume render a 1400x1400x1400 volume of \
floats.</div><div><br></div><div>* ParaView master (4.4 should also do) + OpenGL2 + \
Mesa (OSMesa 11, llvmpipe, swrast): Mesa has some fairly small limits on 3D texture \
size, which is what we use for volume rendering. So, ~ 1000x1000x1000 will be the \
upper end of what can be done for now. In time, we will implement multiple textures / \
streaming to enable rendering of larger \
volumes.</div></div></blockquote><div><br></div></span><div>Did you see my other \
email? You can change the default for OSMesa. I sent it last week. </div><span><font \
color="#888888"><div><br></div><div>- Aashish \
</div></font></span><div><div><blockquote class="gmail_quote" style="margin:0px 0px \
0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div \
dir="ltr"><div><br></div><div>Best,</div><div>-berk</div></div><div><div><div \
class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 28, 2015 at 11:00 AM, \
David Trudgian <span dir="ltr"><<a href="mailto:David.Trudgian@utsouthwestern.edu" \
target="_blank">David.Trudgian@utsouthwestern.edu</a>></span> \
wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Berk,<u></u><u></u></span></p>
<p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Thanks \
very much for looking into this. Look forward to trying things out whenever they're \
ready.<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">DT<u></u><u></u></span></p><span>
<p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:Consolas;color:black">--</span><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><br> David \
Trudgian Ph.D.<br> Computational Scientist, BioHPC<br>
UT Southwestern Medical Center<br>
Dallas, TX 75390-9039<br>
Tel: <a href="tel:%28214%29%20648-4833" value="+12146484833" target="_blank">(214) \
648-4833</a></span><span \
style="font-size:11pt;font-family:Consolas;color:rgb(31,73,125)"><u></u><u></u></span></p>
<p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> \
<u></u></span></p> </span><p class="MsoNormal"><b><span \
style="font-size:11pt;font-family:Calibri,sans-serif">From:</span></b><span \
style="font-size:11pt;font-family:Calibri,sans-serif"> Berk Geveci [mailto:<a \
href="mailto:berk.geveci@kitware.com" target="_blank">berk.geveci@kitware.com</a>] \
<br> <b>Sent:</b> Monday, September 28, 2015 9:58 AM</span></p><div><div><br>
<b>To:</b> David Trudgian <David.Trudgian@UTSouthwestern.edu><br>
<b>Cc:</b> ParaView Mailing List <<a href="mailto:paraview@paraview.org" \
target="_blank">paraview@paraview.org</a>><br> <b>Subject:</b> Re: [Paraview] \
Volume Rendering 17GB 8.5 billion cell \
volume<u></u><u></u></div></div><p></p><div><div> <p class="MsoNormal"><u></u> \
<u></u></p> <div>
<p class="MsoNormal">Hi David,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I have been trying to find some cycles to check this out myself \
with ParaView 4.4. Thanks to hardware issues (i.e. my big workstation's disk \
dying), I haven't been able to. Good news is that I found issues with OSMesa + \
OpenGL2 that we are working through. Give me another 1-1.5 weeks.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Best,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-berk<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Sep 28, 2015 at 10:46 AM, David Trudgian <<a \
href="mailto:David.Trudgian@utsouthwestern.edu" \
target="_blank">David.Trudgian@utsouthwestern.edu</a>> wrote:<u></u><u></u></p> \
<blockquote style="border-style:none none none \
solid;border-left-color:rgb(204,204,204);border-left-width:1pt;padding:0in 0in 0in \
6pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Hi \
Berk,</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Finally \
managed to grab an allocation of some Tesla K40 nodes on our cluster, to check GPU \
rendering of the full 17GB file with 2 x 12GB GPUs. I see the same thing as I did \
with OSMesa rendering.</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">The 9GB \
downsampled version works great, across 2 nodes both with a single K40. Go up to the \
17GB original file and nothing is rendered, no errors. Same behavior with OPENGL or \
OPENGL2 backends.</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">This is \
all on paraview 4.3.1 still – I need to find time to build OSMesa / MPI versions of \
4.4 here. But, does 4.4. have any fixes that would be expected to affect \
this?</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Thanks,</span><u></u><u></u></p>
<p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:Consolas;color:black">--</span><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><br> David \
Trudgian Ph.D.<br> Computational Scientist, BioHPC<br>
UT Southwestern Medical Center<br>
Dallas, TX 75390-9039<br>
Tel: <a href="tel:%28214%29%20648-4833" target="_blank">(214) \
648-4833</a></span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> \
</span><u></u><u></u></p> <p class="MsoNormal"><b><span \
style="font-size:11pt;font-family:Calibri,sans-serif">From:</span></b><span \
style="font-size:11pt;font-family:Calibri,sans-serif"> Berk Geveci [mailto:<a \
href="mailto:berk.geveci@kitware.com" target="_blank">berk.geveci@kitware.com</a>] \
<br> <b>Sent:</b> Tuesday, September 15, 2015 2:43 PM<br>
<b>To:</b> David Trudgian <<a href="mailto:David.Trudgian@UTSouthwestern.edu" \
target="_blank">David.Trudgian@UTSouthwestern.edu</a>><br> <b>Cc:</b> ParaView \
Mailing List <<a href="mailto:paraview@paraview.org" \
target="_blank">paraview@paraview.org</a>><br> <b>Subject:</b> Re: [Paraview] \
Volume Rendering 17GB 8.5 billion cell volume</span><u></u><u></u></p> <p \
class="MsoNormal"> <u></u><u></u></p> <div>
<p class="MsoNormal">Hey David,<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I am hoping to have some time to play around with volume \
rendering and hopefully tracking this issue, one thing that I wanted to clarify: it \
sounds from you description that you have a short (2 byte) value. Is that \
correct?<u></u><u></u></p> </div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-berk<u></u><u></u></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">On Wed, Sep 9, 2015 at 5:00 PM, David Trudgian <<a \
href="mailto:david.trudgian@utsouthwestern.edu" \
target="_blank">david.trudgian@utsouthwestern.edu</a>> wrote:<u></u><u></u></p> \
<blockquote style="border-style:none none none \
solid;border-left-color:rgb(204,204,204);border-left-width:1pt;padding:0in 0in 0in \
6pt;margin:5pt 0in 5pt 4.8pt"> <p class="MsoNormal">Hi,<br>
<br>
We have been experimenting with using Paraview to display very volumes from very<br>
large TIFF stacks generated by whole-brain microscopy equipment. The test stack<br>
has dimensions of 5,368x10,695x150. Stack is assembled in ImageJ from individual<br>
TIFFs, exported as a RAW and loaded into paraview. Saved as a .vti for<br>
convenience. Can view slices fine in standalone paraview client on a 256GB \
machine.<br> <br>
When we attempt volume rendering on this data across multiple nodes with MPI<br>
nothing appears in the client. Surface view works as expected. On switching to<br>
volume rendering the client's display will show nothing. There are no \
messages<br> from the client or servers - no output.<br>
<br>
This is happening when running pvserver across GPU nodes with NVIDIA Tesla<br>
cards, or using CPU only with OSMESA. pvserver memory usage is well below what<br>
we have on the nodes - no memory warnings/errors.<br>
<br>
Data is about 17GB, 8 billion cells. If we downsize to ~4GB or ~9GB then we can<br>
get working volume rendering. The 17GB never works regardless of scaling<br>
nodes/mpi processes. The 4/9GB will work on 1 or 2 nodes.<br>
<br>
Am confused by the lack of rendering, as we don't have memory issues, or an<br>
other messages at all. Am wondering if there are any inherent limitation, or \
I'm<br> missing something stupid.<br>
<br>
Thanks,<br>
<br>
Dave Trudgian<br>
<br>
<br>
_______________________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a \
href="http://www.kitware.com/opensource/opensource.html" target="_blank"> \
http://www.kitware.com/opensource/opensource.html</a><br> <br>
Please keep messages on-topic and check the ParaView Wiki at: <a \
href="http://paraview.org/Wiki/ParaView" target="_blank"> \
http://paraview.org/Wiki/ParaView</a><br> <br>
Search the list archives at: <a href="http://markmail.org/search/?q=ParaView" \
target="_blank"> http://markmail.org/search/?q=ParaView</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://public.kitware.com/mailman/listinfo/paraview" \
target="_blank">http://public.kitware.com/mailman/listinfo/paraview</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<table border="0" cellpadding="0">
<tbody>
<tr>
<td style="padding:0in">
<p align="right" style="text-align:right"><b><span \
style="font-size:17pt;font-family:Helvetica,sans-serif;color:rgb(0,84,160);letter-spacing:-2.25pt">UT</span></b><b><span \
style="font-size:17pt;font-family:Helvetica,sans-serif;color:rgb(0,84,160)"> \
Southwestern</span></b> <u></u><u></u></p>
</td>
</tr>
<tr>
<td style="padding:0in">
<p align="right" style="text-align:right"><span \
style="font-size:13pt;font-family:Helvetica,sans-serif;color:rgb(102,109,112)">Medical \
Center</span><u></u><u></u></p> </td>
</tr>
<tr>
<td style="padding:0in"></td>
</tr>
<tr>
<td style="padding:0in">
<p align="right" style="text-align:right"><span \
style="font-size:10pt;font-family:Helvetica,sans-serif;color:rgb(102,109,112)">The \
future of medicine, today.</span><u></u><u></u></p> </td>
</tr>
</tbody>
</table>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
</blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" \
target="_blank">www.kitware.com</a><br> <br>
Visit other Kitware open-source projects at <a \
href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" \
target="_blank">http://www.kitware.com/opensource/opensource.html</a><br> <br>
Please keep messages on-topic and check the ParaView Wiki at: <a \
href="http://paraview.org/Wiki/ParaView" rel="noreferrer" \
target="_blank">http://paraview.org/Wiki/ParaView</a><br> <br>
Search the list archives at: <a href="http://markmail.org/search/?q=ParaView" \
rel="noreferrer" target="_blank">http://markmail.org/search/?q=ParaView</a><br> <br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://public.kitware.com/mailman/listinfo/paraview" rel="noreferrer" \
target="_blank">http://public.kitware.com/mailman/listinfo/paraview</a><br> \
<br></blockquote></div></div></div><br><br clear="all"><span><div><br></div>-- \
<br><div><div dir="ltr"><font face="trebuchet ms, sans-serif"><i>| Aashish Chaudhary \
<br>| Technical Leader <br>| Kitware Inc. \
<br></i></font><div><i><font face="trebuchet ms, sans-serif">| </font><a \
href="http://www.kitware.com/company/team/chaudhary.html" \
target="_blank">http://www.kitware.com/company/team/chaudhary.html</a></i></div></div></div>
</span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div><div \
dir="ltr"><font face="trebuchet ms, sans-serif"><i>| Aashish Chaudhary <br>| \
Technical Leader <br>| Kitware Inc. \
<br></i></font><div><i><font face="trebuchet ms, sans-serif">| </font><a \
href="http://www.kitware.com/company/team/chaudhary.html" \
target="_blank">http://www.kitware.com/company/team/chaudhary.html</a></i></div></div></div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div></div></div><div><div><br><br \
clear="all"><div><br></div>-- <br><div><div dir="ltr"><font face="trebuchet ms, \
sans-serif"><i>| Aashish Chaudhary <br>| Technical Leader <br>| Kitware \
Inc. <br></i></font><div><i><font face="trebuchet ms, sans-serif">| \
</font><a href="http://www.kitware.com/company/team/chaudhary.html" \
target="_blank">http://www.kitware.com/company/team/chaudhary.html</a></i></div></div></div>
</div></div></div></div>
</blockquote></div><br></div></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div \
class="gmail_signature"><div dir="ltr"><font face="trebuchet ms, sans-serif"><i>| \
Aashish Chaudhary <br>| Technical Leader <br>| Kitware Inc. \
<br></i></font><div><i><font face="trebuchet ms, sans-serif">| </font><a \
href="http://www.kitware.com/company/team/chaudhary.html" \
target="_blank">http://www.kitware.com/company/team/chaudhary.html</a></i></div></div></div>
</div>
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
Search the list archives at: http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic