[prev in list] [next in list] [prev in thread] [next in thread]
List: kstars-devel
Subject: Re: Optimal Exposure and Noise Calculator
From: Wayne Rosing <wrosing () lco ! global>
Date: 2023-03-06 4:05:39
Message-ID: CA+Ec_NhY7iaq=2RawFGbJgaF6FzRSKYS5Ni3A1a=ohT367T+MQ () mail ! gmail ! com
[Download RAW message or body]
My 2 cents on this is that given most mountings are not in fixed setups,
the exposure is limited by your tolerance for image elongation: Hence
register and stack.
If you do that, stack until you drop and you can get good images. Yes, the
price one pays is long computation times. I have a machine with 1 TB of
ram and
64 cores, and it can be slow. But wait -- the result will be good, that is
what math and statistics deliver, just be patient. Most People have NO
IDEA what is behind the
development of the Hubble images. I can guarantee you, it is a substantial
amount of computing. So be patient.
Think of the problem this way. Emission from the sky is incoherent: so
that signal is proportional to the area of the telescope: Photons here-and
-there flying about
all over your imager surface. They build linear with time and the noise is
time^-2. So that noise goes down.
Now a star gets focussed: so it is a small spot. The signal builds up, so
eventually, it dominates. But as the doctor once said: Patients, please
be patient.
These insights are all in Robin Glover's talk. All I am saying is there
are different ways to explain it.
You COULD do a 45000 second properly calculated CCD exposure, except for
this problem of the Sun rising.
You CAN collect 45000 seconds of photons over multiple nights if that is
what you need to get your data for a particular target.
Clear skies,
W
On Mon, Mar 6, 2023 at 1:38 AM Robert Lancaster <rlancaste@gmail.com> wrote:
> My cameras are an SBIG ST8300M, a ZWO 120, a ZWO 224, a Meade DSI, and a
> couple of Canon DSLRs.
>
> I also have access to some other cameras including a couple of other SBIG
> cameras, an Alta Camera, a Fishcamp Starfish, and probably some others if I
> ask nicely.
>
> On Mar 5, 2023, at 5:45 PM, joseph.mcgee@sbcglobal.net wrote:
>
> Thanks Robert,
>
> The process that Dr Glover presented for optimal exposure time considers
> both camera read-noise for a given gain setting, and the sky quality for
> light pollution, (I've also experimented with a simple adjustment for
> filters band-pass but that has a long way to go before it's accurate). So
> if we only consider the read-noise vs light pollution issues, it's sort of
> a balancing act. An exposure needs to be long enough to overwhelm the read
> noise, but not so long that light pollution starts overwhelming the target
> signal. So the optimal sub exposure time is basically an intersection of
> two curves. Maybe that would be a good way to graphically present the
> calculation in KStars.
>
> The points you raise for other factors are very good, interestingly the
> exposures I now tend to use in my fairly light polluted backyard are so
> short that guiding is generally not an issue. But when I'm able get to the
> darker site, I can use longer exposures, and then guiding accuracy, (and
> aircraft traffic), both become a limiting factors for my exposures. My
> guiding accuracy usually varies in a sine wave that spans a few minutes. I
> sometimes manually trigger my exposures to avoid running exposures at the
> extremes of the accuracy issues.
>
> But I think for the short term I should work to revise my current code to
> improve it before I try to integrate it into KStars. I just added data for
> a couple of other cameras. But I also want to work on the process that
> Warren suggested for using bias frames to determine read noise.
>
> Also, it would be helpful for me to have a reality check on this
> calculation with input from you more experienced folks. If anyone wants to
> send me a few details, I can run calculations and send them back for
> review. For this I would need your camera model (preferably a ZWO or QHY
> camera), the focal ratio of the scope, and the SQM of the site.
>
> Thanks
>
>
>
> On 3/5/23 10:18, Robert Lancaster wrote:
>
> Hey, so maybe this is more to the point. The person taking the photos
> may not know how all of this stuff factors into the optimal exposure time.
> There are quite a lot of factors that would affect the best length of an
> exposure and often times a number of us just take a guess.
> For example, I often will try for 5 minute exposures as a default, but
> sometimes I am at a different location or using a different camera and that
> doesn't work.
> Or maybe I have a guiding issue and have to just take a guess at what
> might work.
> I think if this tool only looks at the read noise, then that is not as
> useful, but if you could make it take into account a number of factors and
> then just make a recommendation, that might be extremely valuable.
> KStars knows your geographic coordinates. It knows the camera model that
> is connected. It knows the filter you have selected. It knows the
> brightness of the target you have selected.
> If you are guiding, it might also have a sense for how much guiding error
> there is. It can get the temperature from the sensor and other connected
> devices.
> Since all of these things will affect the optimal exposure time, I think
> it would be really cool if KStars could just make a recommendation for the
> time.
> I don't think we would need a separate module for it, just a box right
> next to the exposure time for the recommendation.
> You could have a button next to it where you could edit parameters that
> affect the calculation of optimal exposure as well, but it could just use
> some defaults to start.
>
> KStars could also use side panel of the FITSViewer to display estimated
> noise levels in addition to the statistics info that is currently displayed
> there. I think this is a separate idea, but the current noise levels could
> be used in the calculation above as well.
>
> Just some thoughts,
>
> Rob
>
> On Mar 5, 2023, at 11:50 AM, Warren <warren.craddock@gmail.com>
> <warren.craddock@gmail.com> wrote:
>
> Hey Joseph, you may inadvertently be getting some light into your bias
> frames. Make sure you cap the camera like you'd do for dark frames.
>
> Here's a more explicit, complete process for measuring read noise from
> bias images. I don't have access to an astro camera at the moment (I'm
> snowed in at Lake Tahoe, boo hoo) to verify this process, but I can try it
> myself in a couple days.
>
>
> http://astro.physics.uiowa.edu/~kaaret/2013f_29c137/Lab03_noise.html#:~:text=The%20read%20noise%20of%20the,removing%20hot%20and%20dead%20pixels
> ).
>
> In regards to subexposure length, I'm not personally against the
> calculator, but maybe it should have a disclaimer. I think it's true that
> almost everyone using recent CMOS cameras should just use, say, two minutes
> by default. This consistency really simplifies workflow.
>
> If you have trouble with tracking, periodic error, tracking, fast high
> clouds, wind gusts, polar alignment, etc. then you can switch to 30- or
> 60-second subs with almost no effect other than using more disk space and
> more CPU time.
>
> There may be people using older CCD cameras with KStars / Ekos though!
>
> - W
>
> On Sat, Mar 4, 2023 at 10:38 PM Wolfgang Reissenberger <
> sterne-jaeger@openfuture.de> wrote:
>
> > Joseph,
> > I'm not sure what type of function we are talking here. Is your intention
> > to calculate the optimal exposure time for a single frame or for the
> > target? If its the first one, I have the same questions as Hy. For the
> > latter, I'm happy to learn more about it.
> >
> > Wolfgang
> > —
> > Wolfgang Reissenberger
> >
> > www.sterne-jaeger.de
> > TSA-120 + FSQ-85 + epsilon-160 | Avalon Linear + M-zero | ASI 1600mm pro
> > + 6200mm pro
> >
> > Am 05.03.2023 um 06:06 schrieb joseph.mcgee@sbcglobal.net:
> >
> > Hi All,
> >
> > Let me explain my reasoning for developing the optimal exposure
> > calculator and noise calculator. I started fairly recently in this A.P.
> > hobby (mid 2019), and had no mentor. Most of the online resources seemed to
> > provide information and suggestions that were geared toward imaging in very
> > dark skies with more advanced equipment than a beginner would own. The
> > typical recommended exposure times I read about were on the order of a many
> > minutes.
> >
> > But when I was experimenting with and learning to use my gear, I
> > typically did so in my own backyard (SQM 19.63). I initially spent quite a
> > few frustrating nights trying to find exposure settings that would produce
> > a decent image. As I acquired filters, I had to repeat the learning
> > process. Then when I had the opportunity to travel to a darker site 90
> > miles from my home, (SQM 21.65), I again had to repeat the learning
> > process. The difference in the exposure times at these two sites was
> > pretty shocking to me.
> >
> > I fully grasp that you all have considerable experience with A.P. but
> > this tool is really not intended to provide benefit to folks that have such
> > experience. The target audience for this tool is the newcomer to this
> > hobby (like me three years ago). I would have been thrilled to have tool
> > that says when I'm in my backyard shooting with gain at 100, and no filter,
> > that my exposure time should only be around 45 seconds.
> >
> > Now, back to the topic...
> >
> > Warren,
> >
> > You raised a suggestion that bias frames could be used to determine
> > sensor read noise. I must be missing some knowledge in this area. I just
> > ran a test with my planetary camera (ASI-178), where I captured a set of
> > bias frames incrementing the gain from 0 to 400 in steps of 50, with an
> > exposure time 32us, (I believe that is the lower limit for the ASI-178). I
> > then used a tool that can assess noise in the image. The noise measured in
> > each image increased as the gain increased; so this did not match the
> > downward trend I expected from the ZWO read-noise graph.
> >
> > Perhaps the tool I used for noise assessment was not the best choice.
> >
> > Can you explain further how I might be able to analyze bias frames to
> > determine read noise?
> >
> > Thanks
> >
> >
> >
> >
> >
>
>
--
Clear skies,
Wayne Rosing
Founder
805 452 3229 cell <<<NEW! Please do not give out widely.
805 880 1603 office
Please note wrosing@lco.global is our preferred address.
[Attachment #3 (text/html)]
<div dir="ltr">My 2 cents on this is that given most mountings are not in fixed \
setups, the exposure is limited by your tolerance for image elongation: Hence \
register and stack.<div><br></div><div>If you do that, stack until you drop and you \
can get good images. Yes, the price one pays is long computation times. I have a \
machine with 1 TB of ram and </div><div>64 cores, and it can be slow. But wait -- \
the result will be good, that is what math and statistics deliver, just be patient. \
Most People have NO IDEA what is behind the </div><div>development of the Hubble \
images. I can guarantee you, it is a substantial amount of computing. So be \
patient.</div><div><br></div><div>Think of the problem this way. Emission from the \
sky is incoherent: so that signal is proportional to the area of the telescope: \
Photons here-and -there flying about</div><div>all over your imager surface. They \
build linear with time and the noise is time^-2. So that noise goes \
down.</div><div><br></div><div>Now a star gets focussed: so it is a small spot. The \
signal builds up, so eventually, it dominates. But as the doctor once said: \
Patients, please be patient.</div><div><br></div><div>These insights are all in Robin \
Glover's talk. All I am saying is there are different ways to explain \
it.</div><div><br></div><div>You COULD do a 45000 second properly calculated CCD \
exposure, except for this problem of the Sun rising.</div><div><br></div><div>You CAN \
collect 45000 seconds of photons over multiple nights if that is what you need to get \
your data for a particular target.</div><div><br></div><div>Clear \
skies,</div><div>W</div><div><br></div></div><br><div class="gmail_quote"><div \
dir="ltr" class="gmail_attr">On Mon, Mar 6, 2023 at 1:38 AM Robert Lancaster <<a \
href="mailto:rlancaste@gmail.com">rlancaste@gmail.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>My cameras are an \
SBIG ST8300M, a ZWO 120, a ZWO 224, a Meade DSI, and a couple of Canon \
DSLRs.<div><br></div><div>I also have access to some other cameras including a couple \
of other SBIG cameras, an Alta Camera, a Fishcamp Starfish, and probably some others \
if I ask nicely.<br><div><br><blockquote type="cite"><div>On Mar 5, 2023, at 5:45 PM, \
<a href="mailto:joseph.mcgee@sbcglobal.net" \
target="_blank">joseph.mcgee@sbcglobal.net</a> wrote:</div><br><div>
<div><p>Thanks Robert,</p><p>The process that Dr Glover presented for optimal \
exposure time considers both camera read-noise for a given gain setting, and the
sky quality for light pollution, (I've also experimented with a
simple adjustment for filters band-pass but that has a long way to
go before it's accurate). So if we only consider the read-noise
vs light pollution issues, it's sort of a balancing act. An
exposure needs to be long enough to overwhelm the read noise, but
not so long that light pollution starts overwhelming the target
signal. So the optimal sub exposure time is basically an
intersection of two curves. Maybe that would be a good way to
graphically present the calculation in KStars. <br>
</p><p>The points you raise for other factors are very good,
interestingly the exposures I now tend to use in my fairly light
polluted backyard are so short that guiding is generally not an
issue. But when I'm able get to the darker site, I can use longer
exposures, and then guiding accuracy, (and aircraft traffic), both
become a limiting factors for my exposures. My guiding accuracy
usually varies in a sine wave that spans a few minutes. I
sometimes manually trigger my exposures to avoid running exposures
at the extremes of the accuracy issues.<br>
</p><p>But I think for the short term I should work to revise my current
code to improve it before I try to integrate it into KStars. I
just added data for a couple of other cameras. But I also want to
work on the process that Warren suggested for using bias frames to
determine read noise.</p><p>Also, it would be helpful for me to have a reality \
check on this calculation with input from you more experienced folks. If anyone
wants to send me a few details, I can run calculations and send
them back for review. For this I would need your camera model
(preferably a ZWO or QHY camera), the focal ratio of the scope,
and the SQM of the site.</p><p>Thanks<br>
</p><p><br>
</p><p><br>
</p>
<div>On 3/5/23 10:18, Robert Lancaster
wrote:<br>
</div>
<blockquote type="cite">
Hey, so maybe this is more to the point. The person taking the
photos may not know how all of this stuff factors into the optimal
exposure time.
<div>There are quite a lot of factors that would affect the best
length of an exposure and often times a number of us just take a
guess.</div>
<div>For example, I often will try for 5 minute exposures as a
default, but sometimes I am at a different location or using a
different camera and that doesn't work.</div>
<div>Or maybe I have a guiding issue and have to just take a guess
at what might work.<br>
<div>I think if this tool only looks at the read noise, then
that is not as useful, but if you could make it take into
account a number of factors and then just make a
recommendation, that might be extremely valuable.</div>
<div>KStars knows your geographic coordinates. It knows the
camera model that is connected. It knows the filter you have
selected. It knows the brightness of the target you have
selected.</div>
<div>If you are guiding, it might also have a sense for how much
guiding error there is. It can get the temperature from the
sensor and other connected devices.</div>
<div>Since all of these things will affect the optimal exposure
time, I think it would be really cool if KStars could just
make a recommendation for the time.</div>
<div>I don't think we would need a separate module for it, just
a box right next to the exposure time for the recommendation.</div>
<div>You could have a button next to it where you could edit
parameters that affect the calculation of optimal exposure as
well, but it could just use some defaults to start.</div>
<div><br>
</div>
<div>KStars could also use side panel of the FITSViewer to
display estimated noise levels in addition to the statistics
info that is currently displayed there. I think this is a
separate idea, but the current noise levels could be used in
the calculation above as well.</div>
<div><br>
</div>
<div>Just some thoughts,</div>
<div><br>
</div>
<div>Rob</div>
<div>
<div><br>
<blockquote type="cite">
<div>On Mar 5, 2023, at 11:50 AM, Warren
<a href="mailto:warren.craddock@gmail.com" \
target="_blank"><warren.craddock@gmail.com></a> wrote:</div> <br>
<div>
<div dir="auto">Hey Joseph, you may inadvertently be
getting some light into your bias frames. Make sure
you cap the camera like you'd do for dark frames.</div>
<div dir="auto"><br>
</div>
<div dir="auto">Here's a more explicit, complete process
for measuring read noise from bias images. I don't
have access to an astro camera at the moment (I'm
snowed in at Lake Tahoe, boo hoo) to verify this
process, but I can try it myself in a couple days.</div>
<div dir="auto"><br>
</div>
<div dir="auto">
<div><a \
href="http://astro.physics.uiowa.edu/~kaaret/2013f_29c137/Lab03_noise.html#:~:text=The%20read%20noise%20of%20the,removing%20hot%20and%20dead%20pixels" \
target="_blank">http://astro.physics.uiowa.edu/~kaaret/2013f_29c137/Lab03_noise.html#: \
~:text=The%20read%20noise%20of%20the,removing%20hot%20and%20dead%20pixels</a>).</div> \
<br> </div>
<div dir="auto">In regards to subexposure length, I'm
not personally against the calculator, but maybe it
should have a disclaimer. I think it's true that
almost everyone using recent CMOS cameras should just
use, say, two minutes by default. This consistency
really simplifies workflow.</div>
<div dir="auto"><br>
</div>
<div dir="auto">If you have trouble with tracking,
periodic error, tracking, fast high clouds, wind
gusts, polar alignment, etc. then you can switch to
30- or 60-second subs with almost no effect other than
using more disk space and more CPU time.</div>
<div dir="auto"><br>
</div>
<div dir="auto">There may be people using older CCD
cameras with KStars / Ekos though!</div>
<div dir="auto"><br>
</div>
<div dir="auto">- W</div>
<div><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sat, Mar 4,
2023 at 10:38 PM Wolfgang Reissenberger <<a \
href="mailto:sterne-jaeger@openfuture.de" \
target="_blank">sterne-jaeger@openfuture.de</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div>Joseph,
<div>I'm not sure what type of function we are
talking here. Is your intention to calculate
the optimal exposure time for a single frame
or for the target? If its the first one, I
have the same questions as Hy. For the latter,
I'm happy to learn more about it.</div>
<div><br>
</div>
<div>Wolfgang<br>
<div>
<div dir="auto" \
style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<div dir="auto" style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<div dir="auto" style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<div dir="auto">
<div \
style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal \
;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">—</div>
<div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps \
:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text \
-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">Wolfgang \
Reissenberger<br> <br>
</div>
<div \
style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal \
;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><a \
href="http://www.sterne-jaeger.de/" style="font-family:Helvetica" \
target="_blank">www.sterne-jaeger.de</a><br> TSA-120 + FSQ-85 + epsilon-160 |
Avalon Linear + M-zero | ASI
1600mm pro + 6200mm pro</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div>
<div>
<div><br>
<blockquote type="cite">
<div>Am 05.03.2023 um 06:06 schrieb <a \
href="mailto:joseph.mcgee@sbcglobal.net" \
target="_blank">joseph.mcgee@sbcglobal.net</a>:</div> <br>
<div>
<div>Hi All,<br>
<br>
Let me explain my reasoning for
developing the optimal exposure
calculator and noise calculator. I
started fairly recently in this A.P.
hobby (mid 2019), and had no mentor.
Most of the online resources seemed to
provide information and suggestions
that were geared toward imaging in
very dark skies with more advanced
equipment than a beginner would own.
The typical recommended exposure times
I read about were on the order of a
many minutes.<br>
<br>
But when I was experimenting with and
learning to use my gear, I typically
did so in my own backyard (SQM
19.63). I initially spent quite a few
frustrating nights trying to find
exposure settings that would produce a
decent image. As I acquired filters,
I had to repeat the learning process.
Then when I had the opportunity to
travel to a darker site 90 miles from
my home, (SQM 21.65), I again had to
repeat the learning process. The
difference in the exposure times at
these two sites was pretty shocking to
me.<br>
<br>
I fully grasp that you all have
considerable experience with A.P. but
this tool is really not intended to
provide benefit to folks that have
such experience. The target audience
for this tool is the newcomer to this
hobby (like me three years ago). I
would have been thrilled to have tool
that says when I'm in my backyard
shooting with gain at 100, and no
filter, that my exposure time should
only be around 45 seconds.<br>
<br>
Now, back to the topic...<br>
<br>
Warren,<br>
<br>
You raised a suggestion that bias
frames could be used to determine
sensor read noise. I must be missing
some knowledge in this area. I just
ran a test with my planetary camera
(ASI-178), where I captured a set of
bias frames incrementing the gain from
0 to 400 in steps of 50, with an
exposure time 32us, (I believe that is
the lower limit for the ASI-178). I
then used a tool that can assess noise
in the image. The noise measured in
each image increased as the gain
increased; so this did not match the
downward trend I expected from the ZWO
read-noise graph.<br>
<br>
Perhaps the tool I used for noise
assessment was not the best choice.<br>
<br>
Can you explain further how I might be
able to analyze bias frames to
determine read noise?<br>
<br>
Thanks<br>
<br>
<br>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
</div></blockquote></div><br></div></div></blockquote></div><br \
clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div \
dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr">Clear skies, \
</div><div dir="ltr"><br></div><div dir="ltr">Wayne Rosing<div>Founder</div><div>805 \
452 3229 cell <<<NEW! Please do not give out widely.</div><div>805 880 \
1603 office</div><div><br></div><div>Please note wrosing@lco.global is our preferred \
address.</div></div></div></div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic