From kstars-devel Sat Oct 31 22:35:26 2020 From: Akarsh Simha Date: Sat, 31 Oct 2020 22:35:26 +0000 To: kstars-devel Subject: Re: Observation workflows in KStars -- looking for feedback and ideas Message-Id: X-MARC-Message: https://marc.info/?l=kstars-devel&m=160418372504198 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--0000000000007fe5f005b2ff2008" --0000000000007fe5f005b2ff2008 Content-Type: text/plain; charset="UTF-8" Am Sa., 31. Okt. 2020 um 15:31 Uhr schrieb Valentin : > Hi, > Just a quick remark without having read the whole thing yet. > > It would be great if the observation log and info wasn't stored in the sky > object itself, because in the new catalog system it isn't guaranteed that > there is a unique instance of that object. > Very valid point. Should be in a DB associated in some way with a UUID. > On October 31, 2020 11:16:51 PM GMT+01:00, Akarsh Simha < > akarshsimha@gmail.com> wrote: >> >> >> >> Am Sa., 31. Okt. 2020 um 05:30 Uhr schrieb Jasem Mutlaq < >> mutlaqja@ikarustech.com>: >> >>> Hello Akarsh, >>> >> >> Hi Jasem >> >> I agree that the observing wizard is due for a revamp! The GUI itself >>> could reuse some design as well. With one object occupying a single row >>> that includes all information, including a view for Alt vs Time for EACh >>> object (so you can compare a few against each other). Furthermore, I think >>> it should also cater now to astrophotography use. For this to work, I have >>> a couple of ideas: >>> >> >> I agree that the ability to compare a few objects against each other is >> cool, but I think having the Alt vs Time in the row will make the rows too >> big, to the point where you will only be able to display 5--6 objects in a >> smaller screen. This is not ideal if you want to search for an object by >> just scanning the list. Instead, I propose that you can select multiple >> objects, and just like in the actual Alt vs Time tool, the Alt vs Time >> widget that's in the observation planner should plot multiple traces, one >> for each object that was selected -- does this serve the purpose instead? >> I'm a bit surprised we don't already do this... >> >> >>> 1. Filter by Band: Broadband vs Narrow band targets. >>> >> >> This would be extremely useful! But this is somewhat challenging because >> we don't have a field in KStars that describes whether something is a >> narrow-band target or not. We could make some guesses by catalog (eg: Sh2, >> Abell PN), but it's hard because I think we only have one designation in >> KStars ("Diffuse Nebula") for both emission and reflection nebulae. Any >> ideas on how to make this happen? >> >> >>> 2. Filter by size: Show objects that fit within say 60% to 120% of the >>> FOV. So they're not too small or too big to image. Of course, this is only >>> applicable to extended objects with known angular resolutions. >>> >> >> This is totally doable. >> >> >>> >>> Also, I think instead of having to use the wizard page-by-page to >>> produce this, these should always be accessible to filter from. So no more >>> wizard, but controls to filter ALL the objects. Since we're limited by >>> memory, we only show the first 100 or so applicable objects by default, and >>> perhaps this can be increased. So we can have these filters: >>> >> >> I agree. I think this is a major itch to scratch! The step-by-step wizard >> is a bit too "elaborate" and yet inflexible. I would add the ability to >> search all objects in KStars (including CatalogComponent), as well as apply >> the filters only on the existing Wish List (while adding to the Session >> Plan). This way, for those of us who have targeted wish lists that are >> getting "too large to manage", we can still filter from the wishlist to >> plan a given night easily. >> >> Also, showing 100 objects "live" might be hard because we'll need to >> filter a very large database repeatedly. So I think we can use a mechanism >> similar to the present "Update Count" button, with an "Update List" button >> that can take 1--2 seconds to process the entire database (since we cannot >> possibly have an index on every field). The other solution is to have an >> QTimer to update it whenever the user has not changed the filters for a >> while. This feature is very easy to bring on top of the wish list, but it's >> very hard to bring for the entire database, especially if we are going to >> lazy-load things like PGC galaxies at a later stage (Valentin's merge >> request). >> >> So for this project, I think we should "leave out" the ability to filter >> the entire database including custom catalogs, but we can readily provide >> the ability to filter the wishlist as well as the NGC/IC catalogs. I think >> we should merge Valentin's work before we can do any filter on the entire >> database. >> >> >>> 1. Type: Stars, Planets, Nebulae..etc >>> 2. Band: RGB, Narrowband..etc >>> 3. Size: arcmins steps? maybe have option for "camera FOV" as well which >>> is auto-calculated from INDI. >>> 4. Region (Constellation or NSWE) >>> 5. Magnitude >>> 6. Altitude >>> 7. ??? >>> >> >> On the FOV subject, we could also allow calculation from FOV symbols (and >> for the FOV symbols, we have the ability to use various calculations while >> defining them). >> >> If I can add a few more here: >> 1. Regex on the name (so someone can search a specific catalog from the >> long name etc.) >> 2. Does the object transit meridian during the dark part of the night? >> This is immensely useful as is for visual observers, but it becomes very >> useful for imagers also if we add one more control -- how many hours before >> / after meridian is available without twilight/moon. This logic will be >> hard to code, since we should calculate interference from moon and >> astronomical twilight, but I think it will be extremely useful and >> therefore worth it. >> 3. Has a "Note" associated with it in the "Log" field (less important) >> >> This is everything I can think of. >> >> >>> -- >>> Best Regards, >>> Jasem Mutlaq >>> >>> >>> >>> On Sat, Oct 31, 2020 at 4:43 AM Akarsh Simha >>> wrote: >>> >>>> Hi all, >>>> >>>> I'm just trying to get a sense of how people use the Observation >>>> Planner. A long time ago, the "Observation Planner" was called "Observing >>>> List" and just carried: >>>> (a) A list of objects >>>> (b) Ability to add notes / observation logs >>>> can't remember what else -- there may have been Alt vs Time >>>> >>>> But anyway, in 2009, Prakash Mohan and I worked together as part of >>>> GSoC to add the DSS image download feature, and the bifurcation between a >>>> "Session Plan" and a "Wishlist". Back then, my idea was as follows: >>>> 1. Whenever you come across an object that interests you, add it to >>>> your Wish List >>>> 2. Before you go out observing, plan out your observing session by >>>> importing the objects you wish to see from your wish list into your >>>> "session plan". KStars automatically assigns the upper meridian transit >>>> time on the night of observation (I suspect this is currently broken) as >>>> the observing time for each object. >>>> 3. Before you go out to a dark place, while you still have fast >>>> internet access, download any reference images you need from the DSS >>>> 4. On the observing field, sort the observing session plan by >>>> observation time. A special sorting filter sorts it from evening to morning. >>>> 5. Work through your objects in order. >>>> I have a tendency to pack my night with as many objects as I can >>>> possibly observe, and I imagined that the above workflow would maximize my >>>> efficiency. >>>> >>>> After trying this out on the field a few times, though, I found it very >>>> sub-par for visual observing. The reason is, it is difficult to estimate >>>> correctly how long it takes to observe an object. And the above is not >>>> robust to, let's say, losing half an hour due to dinner. Moreover, for >>>> those of us who observe with Dobsonians, sometimes meridian transit is a >>>> very inconvenient time to observe an object because of the singular motion >>>> of azimuth around the zenith ("Dobson's Hole" / "the Dob hole"). >>>> >>>> So I later came up with the following workflow that only involves the >>>> "Wish List": >>>> 1. Whenever you come across an object that interests you, add it to >>>> your Wish List. >>>> 2. Before you go out to a dark place, while you still have fast >>>> internet access, download any reference images you need from the DSS >>>> 3. On the observing field, sort the _wish list_ by % of max. altitude >>>> achieved at current time, while demoting the objects that are in the hole. >>>> 4. At any given time, pick your favorites amongst the top 5 or 10 >>>> objects in the sorted list and observe it. >>>> >>>> I have found the above wishlist-based workflow exceptional for visual >>>> observing. So much so that I am tempted to deprecate the session plan >>>> workflow, but I do think: >>>> (a) The session plan workflow is much better suited for imaging / >>>> scientific observations >>>> (b) There might still be people out there using the session plan >>>> workflow >>>> so I do think we should still retain it. >>>> >>>> Now, finally, I have a few more itches to scratch with the wishlist >>>> workflow: >>>> 1. Wish list grows to unmanageable sizes very quickly. >>>> 2. There is no easy way of removing/demoting objects that you have >>>> already observed. >>>> 3. Plus, unlike with the session plan, there's no scope for targeted >>>> observing projects -- just one massive wishlist. >>>> >>>> I'm looking for other people's "itches" with these workflows, and >>>> trying to understand how others use the Observation Planner, if at all. I >>>> remember Jasem telling me it's also integrated somehow with the Ekos >>>> sequencer, so I'm curious to know how that works. Finally, I'm proposing >>>> the following changes: >>>> >>>> 1. Let the Wish List remain a massive "Wish List" >>>> 2. Move/replicate the "% of max. altitude" workflow into the session >>>> plan instead. >>>> 3. Provide flexible ways of selecting and adding objects from the Wish >>>> List into the session plan (eg: only objects in a certain constellation, >>>> only objects in a certain catalog, only objects matching a certain regex, >>>> only objects that will attain culmination tonight, only objects which are >>>> above a certain magnitude...) >>>> 4. Preserve the old workflow of "assigned times" for objects, and fix >>>> any bugs in it >>>> 5. Provide an "Observed" checkbox that will demote the object in the "% >>>> of max. altitude" workflow, if checked. >>>> 6. Provide a way of removing objects that have already been observed >>>> from the session plan / wishlist >>>> >>>> I believe these features will create a much smoother observation >>>> experience for anyone interested in using the Observation Planner for >>>> observing. Now, I don't have enough context on imagers' requirements, but >>>> if there are any allied requirements, I would be happy to incorporate them >>>> into a project. >>>> >>>> The plan for this project is either I will slowly do it over my >>>> weekends and evenings, or work with a GSoC / SoK student to achieve these >>>> goals. Anyone else interested is also welcome. >>>> >>>> Regards >>>> Akarsh >>>> >>>> > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > --0000000000007fe5f005b2ff2008 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
Am Sa., 31. Okt. 2020 um 15:31=C2=A0U= hr schrieb Valentin <valentin@b= oettcher.cf>:
Hi,
Just a quick remark without having read the whole thing y= et.

It would be great if the observation log and info wasn't sto= red in the sky object itself, because in the new catalog system it isn'= t guaranteed that there is a unique instance of that object.

Very valid point. Should be in a DB associated = in some way with a UUID.


On October 31,= 2020 11:16:51 PM GMT+01:00, Akarsh Simha <akarshsimha@gmail.com> wrote:


=
Am Sa., 31. Okt. 2020 um 05:30=C2=A0U= hr schrieb Jasem Mutlaq <mutlaqja@ikarustech.com>:
Hello Akarsh,

Hi Jasem

I agree that the obs= erving wizard is due for a revamp! The GUI itself could reuse some design a= s well. With one object occupying a single row that includes all informatio= n, including a view for Alt vs Time for EACh object (so you can compare a f= ew against each other). Furthermore, I think it should also cater now to as= trophotography use. For this to work, I have a couple of ideas:
=

I agree that the ability to compare a few = objects against each other is cool, but I think having the Alt vs Time in t= he row will make the rows too big, to the point where you will only be able= to display 5--6 objects in a smaller screen. This is not ideal if you want= to search for an object by just scanning the list. Instead, I propose that= you can select multiple objects, and just like in the actual Alt vs Time t= ool, the Alt vs Time widget that's in the observation planner should pl= ot multiple traces, one for each object that was selected -- does this serv= e the purpose instead? I'm a bit surprised we don't already do this= ...
=C2=A0
1. Filter by Band: Broadband vs Narrow band targ= ets.

This would be extremely us= eful! But this is somewhat challenging because we don't have a field in= KStars that describes whether something is a narrow-band target or not. We= could make some guesses by catalog (eg: Sh2, Abell PN), but it's hard = because I think we only have one designation in KStars ("Diffuse Nebul= a") for both emission and reflection nebulae. Any ideas on how to make= this happen?
=C2=A0
2. Filter by size: Show objects that f= it within say 60% to 120% of the FOV. So they're not too small or too b= ig to image. Of course, this is only applicable to extended objects with kn= own angular resolutions.

This i= s totally doable.
=C2=A0

Also, I think inste= ad of having to use the wizard page-by-page to produce this, these should a= lways be accessible to filter from. So no more wizard, but controls to filt= er ALL the objects. Since we're limited by memory, we only show the fir= st 100 or so applicable objects by default, and perhaps this can be increas= ed. So we can have these filters:

I agree. I think this is a major itch to scratch! The step-by-step wizar= d is a bit too "elaborate" and yet inflexible. I would add the ab= ility to search all objects in KStars (including CatalogComponent), as well= as apply the filters only on the existing Wish List (while adding to the S= ession Plan). This way, for those of us who have targeted wish lists that a= re getting "too large to manage", we can still filter from the wi= shlist to plan a given night easily.

Also, showing= 100 objects "live" might be hard because we'll need to filte= r a very large database repeatedly. So I think we can use a mechanism simil= ar to the present "Update Count" button, with an "Update Lis= t" button that can take 1--2 seconds to process the entire database (s= ince we cannot possibly have an index on every field). The other solution i= s to have an QTimer to update it whenever the user has not changed the filt= ers for a while. This feature is very easy to bring on top of the wish list= , but it's very hard to bring for the entire database, especially if we= are going to lazy-load things like PGC galaxies at a later stage (Valentin= 's merge request).

So for this project, I thin= k we should "leave out" the ability to filter the entire database= including custom catalogs, but we can readily provide the ability to filte= r the wishlist as well as the NGC/IC catalogs. I think we should merge Vale= ntin's work before we can do any filter on the entire database.
=C2=A0
1. Type: Stars, Planets, Nebulae..etc
2. Band: RG= B, Narrowband..etc
3. Size: arcmins steps? maybe have option for = "camera FOV" as well which is auto-calculated from INDI.
4. Region (Constellation or NSWE)
5. Magnitude
6. Alt= itude
7. ???

On the F= OV subject, we could also allow calculation from FOV symbols (and for the F= OV symbols, we have the ability to use various calculations while defining = them).

If I can add a few more here:
1. R= egex on the name (so someone can search a specific catalog from the long na= me etc.)
2. Does the object transit meridian during the dark part of th= e night? This is immensely useful as is for visual observers, but it become= s very useful for imagers also if we add one more control -- how many hours= before / after meridian is available without twilight/moon. This logic wil= l be hard to code, since we should calculate interference from moon and ast= ronomical twilight, but I think it will be extremely useful and therefore w= orth it.
3. Has a "Note" associated with it in the &quo= t;Log" field (less important)

This is eve= rything I can think of.
=C2=A0
--
Best Regards,=
Jasem Mutlaq


=

On Sat, Oct 31, 2020 at 4:43 AM Akarsh Simha <akarshsimha@gmail.com> wrote:=
Hi all,

I'm just trying to get a sense = of how people use the Observation Planner. A long time ago, the "Obser= vation Planner" was called "Observing List" and just carried= :
(a) A list of objects
(b) Ability to add notes / obse= rvation logs
can't remember what else -- there may have been = Alt vs Time

But anyway, in 2009, Prakash Mohan and I work= ed together as part of GSoC to add the DSS image download feature, and the = bifurcation between a "Session Plan" and a "Wishlist". = Back then, my idea was as follows:
1. Whenever you come across an= object that interests you, add it to your Wish List
2. Before yo= u go out observing, plan out your observing session by importing the object= s you wish to see from your wish list into your "session plan". K= Stars automatically assigns the upper meridian transit time on the night of= observation (I suspect this is currently broken) as the observing time for= each object.
3. Before you go out to a dark place, while you sti= ll have fast internet access, download any reference images you need from t= he DSS
4. On the observing field, sort the observing session plan= by observation time. A special sorting filter sorts it from evening to mor= ning.
5. Work through your objects in order.
I have a = tendency to pack my night with as many objects as I can possibly observe, a= nd I imagined that the above workflow would maximize my efficiency.

After trying this out on the field a few times, thoug= h, I found it very sub-par for visual observing. The reason is, it is diffi= cult to estimate correctly how long it takes to observe an object. And the = above is not robust to, let's say, losing half an hour due to dinner. M= oreover, for those of us who observe with Dobsonians, sometimes meridian tr= ansit is a very inconvenient time to observe an object because of the singu= lar motion of azimuth around the zenith ("Dobson's Hole" / &q= uot;the Dob hole").

So I later came up wi= th the following workflow that only involves the "Wish List":
=
1. Whenever you come across an object that interests you, add it= to your Wish List.
2. Before you go out to a dark place, while y= ou still have fast internet access, download any reference images you need = from the DSS
3. On the observing field, sort the _wish list_ by %= of max. altitude achieved at current time, while demoting the objects that= are in the hole.
4. At any given time, pick your favorites among= st the top 5 or 10 objects in the sorted list and observe it.
I have found the above wishlist-based workflow exceptional for = visual observing. So much so that I am tempted to deprecate the session pla= n workflow, but I do think:
(a) The session plan workflow is much= better suited for imaging / scientific observations
(b) There mi= ght still be people out there using the session plan workflow
so = I do think we should still retain it.

Now, finally= , I have a few more itches to scratch with the wishlist workflow:
1. Wish list grows to unmanageable sizes very quickly.
2. There = is no easy way of removing/demoting objects that you have already observed.=
3. Plus, unlike with the session plan, there's no scope for = targeted observing projects -- just one massive wishlist.

I'm looking for other people's "itches" with thes= e workflows, and trying to understand how others use the Observation Planne= r, if at all. I remember Jasem telling me it's also integrated somehow = with the Ekos sequencer, so I'm curious to know how that works. Finally= , I'm proposing the following changes:

1. Let the Wis= h List remain a massive "Wish List"
2. Move/replica= te the "% of max. altitude" workflow into the session plan instea= d.
3. Provide flexible ways of selecting and adding objects f= rom the Wish List into the session plan (eg: only objects in a certain cons= tellation, only objects in a certain catalog, only objects matching a certa= in regex, only objects that will attain culmination tonight, only objects w= hich are above a certain magnitude...)
4. Preserve the old workfl= ow of "assigned times" for objects, and fix any bugs in it
<= div>5. Provide an "Observed" checkbox that will demote the object= in the "% of max. altitude" workflow, if checked.
6. P= rovide a way of removing objects that have already been observed from the s= ession plan / wishlist

I believe these features wi= ll create a much smoother observation experience for anyone interested in u= sing the Observation Planner for observing. Now, I don't have enough co= ntext on imagers' requirements, but if there are any allied requirement= s, I would be happy to incorporate them into a project.

The plan for this project is either I will slowly do it over my weeke= nds and evenings, or work with a GSoC / SoK student to achieve these goals.= Anyone else interested is also welcome.

Regar= ds
Akarsh


--
Sent from my Android device with K-9 Mail. Pl= ease excuse my brevity. --0000000000007fe5f005b2ff2008--