[prev in list] [next in list] [prev in thread] [next in thread]
List: python-edu-sig
Subject: [Edu-sig] Re: ChatGPT for py teaching
From: Wes Turner <wes.turner () gmail ! com>
Date: 2023-01-06 17:03:21
Message-ID: CACfEFw917gkzj47wn4f4EhDyq40QHczstUbgVqa3Nd70VjM3=Q () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
On Wed, Jan 4, 2023 at 4:34 PM Wes Turner <wes.turner@gmail.com> wrote:
> What are the expected limitations of [ChatGPT]?
>
> What is "Prompt Engineering"?
> [Prompt engineering - Wikipedia](
> https://en.wikipedia.org/wiki/Prompt_engineering )
>
> What lessons about technology reliance could you teach, in regards to
> Clippy?
>
> - "What is ChatGPT? Wrong answers only"
> - Human_n: EDGES WITH REASONING
>
> - "Tell me IDK ("I don't know") when you don't know"
>
- What are "Truthiness", Confidence Intervals and Error Propagation?
- What is Convergence?
- What does it mean for algorithmic outputs to converge given additional
parametric noise?
>
> - "How certain are you that that is the correct answer?"
>
- How does [ChatGPT] handle known-to-be or presumed-to-be unsolved math and
physics problems?
- "How do we create room-temperature superconductivity?"
- "How will planetary orbital trajectories change in the n-body gravity
problem if another dense probably interstellar mass passes through our
local system?"
>
> - "Are static analysis code metrics sufficient for Safety Critical code?"
>
- Where in the DevOpSec software development lifecycle should human code
review for security best practices and common vulnerabilities and
weaknesses?
- Is the [ChatGPT] model trained from *only* Formally Verified code with
associated tests?
- Branch and line coverage metrics indicate which tests run which lines
of which functions. Code coverage typically implies dynamic analysis?
>
> - "Whose code is this based on?"
>
> - "Where and when did you learn this?"
>
- Explainability (XAI)
- How can models "Unlearn" or "Learn over" given a sufficient meta-analytic
procedure given the information available at that point in spacetime?
> - "Why would a US President abstain from using ChatGPT or similar to fill
> speeches 'just like what I said before'?"
>
> #Burgundy
>
- How can we use [ChatGPT] and other Prompt Engineering approaches to
perform Evidence-Based Policy with supporting computational analyses
prepared in form for meta-analysis given verification of data quality and
experimental controls?
>
> GPT or similar trained on only Formally-Verified code with associated tests
> and/or e.g. Lean Mathlib, or e.g. the Principia in SymPy & Cirq; that
> could probably eliminate my job, but maybe still not teaching
>
- "Q: ChatGPT etc. trained on mathlib and tests"
https://github.com/leanprover-community/mathlib/issues/17919
- "Port the QuantumQ game to Cirq (SymPy (Python))?"
- https://quantumai.google/cirq/start/start
-
https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwxyw?usp=sharing
- "How does [ChatGPT] compare to Critical Thinking, Reasoning, Logic, and
Rationality?
https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_rationality
> On Wed, Jan 4, 2023, 6:28 AM Christian Mascher <christian.mascher@gmx.de>
> wrote:
>
>> Hi,
>>
>> a student of mine was aware of this chatbot and asked it about a
>> class-assignment of his own accord. We program in Java with some extra
>> homemade library class used by some schools in our region.
>>
>> The bot came up with a "solution" which was flawed in several respects:
>> 1. It used some other (unimported) classes - solution doesn't work and
>> doesn't fit the assignment.
>> 2. It put all the code into the constructor, a typical (design and
>> style) error for students beginning with Java.
>>
>> When confronted with the problem number one above, it acknowledged the
>> fault and produced a different unrelated solution.
>>
>> Sooo....
>>
>> I was impressed how well the chatbot simulated a typical clueless human
>> who even thinks he is smart, while his code is basically bullshit.
>> (Probably a result of googling forums, where other learners posted their
>> solutions to assignments with the given school library classes.) The bot
>> clearly passed the Turing test ;-)
>>
>> But...
>>
>> I don't think the interaction was helpful for somebody who is learning
>> to program. It is probably less helpful than conversing with other also
>> not very knowledgeable students as they are at least reasoning humans.
>>
>> Talking to the bot might be fun to do in the last lesson before
>> christmas or so. Entertaining until you realise the software is
>> "simulating" intelligent conversation - not really talking with insight.
>> And that could turn out to be a waste of time.
>>
>> Happy new year
>>
>> Christian
>>
>> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis:
>> > Hi, happy NY!
>> >
>> > ChatGPT can create, fix and explain code
>> > https://openai.com/blog/chatgpt/#samples
>> > <https://openai.com/blog/chatgpt/#samples>
>> >
>> > Anyone tried to incorporate it into teaching process?
>> > Or have ideas/doubts how it ciuld help?
>> >
>> > _______________________________________________
>> > Edu-sig mailing list -- edu-sig@python.org
>> > To unsubscribe send an email to edu-sig-leave@python.org
>> > https://mail.python.org/mailman3/lists/edu-sig.python.org/
>> > Member address: christian.mascher@gmx.de
>> _______________________________________________
>> Edu-sig mailing list -- edu-sig@python.org
>> To unsubscribe send an email to edu-sig-leave@python.org
>> https://mail.python.org/mailman3/lists/edu-sig.python.org/
>> Member address: wes.turner@gmail.com
>>
>
[Attachment #5 (text/html)]
<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Wed, Jan 4, 2023 at 4:34 PM Wes Turner <<a \
href="mailto:wes.turner@gmail.com">wes.turner@gmail.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">What \
are the expected limitations of [ChatGPT]?<div dir="auto"><br></div><div \
dir="auto">What is "Prompt Engineering"?</div><div dir="auto">[Prompt \
engineering - Wikipedia]( <a href="https://en.wikipedia.org/wiki/Prompt_engineering" \
target="_blank">https://en.wikipedia.org/wiki/Prompt_engineering</a> )<br><div \
dir="auto"><br></div><div dir="auto">What lessons about technology reliance could you \
teach, in regards to Clippy?</div><div dir="auto"><br></div><div dir="auto">- \
"What is ChatGPT? Wrong answers only"</div><div dir="auto"> - Human_n: \
EDGES WITH REASONING</div><div dir="auto"><br></div><div dir="auto">- "Tell me \
IDK ("I don't know") when you don't \
know"</div></div></div></blockquote><div><br></div><div>- What are \
"Truthiness", Confidence Intervals and Error Propagation?</div><div>- What \
is Convergence?</div><div>- What does it mean for algorithmic outputs to converge \
given additional parametric noise?</div><div> </div><blockquote class="gmail_quote" \
style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="auto"><div \
dir="auto"><br></div><div dir="auto">- "How certain are you that that is the \
correct answer?"</div></div></div></blockquote><div><br></div><div>- How does \
[ChatGPT] handle known-to-be or presumed-to-be unsolved math and physics \
problems?</div><div><br></div><div>- "How do we create room-temperature \
superconductivity?"</div><div><br></div><div>- "How will planetary orbital \
trajectories change in the n-body gravity problem if another dense probably \
interstellar mass passes through our local system?"</div><div> \
</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px \
solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="auto"><div \
dir="auto"><br></div><div dir="auto">- "Are static analysis code metrics \
sufficient for Safety Critical \
code?"</div></div></div></blockquote><div><br></div><div>- Where in the DevOpSec \
software development lifecycle should human code review for security best practices \
and common vulnerabilities and weaknesses?</div><div> - Is the [ChatGPT] model \
trained from *only* Formally Verified code with associated tests?</div><div> - \
Branch and line coverage metrics indicate which tests run which lines of which \
functions. Code coverage typically implies dynamic analysis?</div><div> \
</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px \
solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="auto"><div \
dir="auto"><br></div><div dir="auto">- "Whose code is this based \
on?"</div><div dir="auto"><br></div><div dir="auto">- "Where and when did \
you learn this?"</div></div></div></blockquote><div><br></div><div>- \
Explainability (XAI)</div><div>- How can models "Unlearn" or "Learn \
over" given a sufficient meta-analytic procedure given the information available \
at that point in spacetime?</div><div><br></div><div> </div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="auto"><div \
dir="auto"></div><div dir="auto">- "Why would a US President abstain from using \
ChatGPT or similar to fill speeches 'just like what I said \
before'?"</div><div dir="auto"><br></div><div \
dir="auto">#Burgundy</div></div></div></blockquote><div><br></div><div>- How can we \
use [ChatGPT] and other Prompt Engineering approaches to perform Evidence-Based \
Policy with supporting computational analyses prepared in form for meta-analysis \
given verification of data quality and experimental controls?</div><div> \
</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px \
solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="auto"><div \
dir="auto"><br></div><div dir="auto">GPT or similar trained on only Formally-Verified \
code with associated tests</div><div dir="auto">and/or e.g. Lean Mathlib, or e.g. the \
Principia in SymPy & Cirq; that could probably eliminate my job, but maybe still \
not teaching </div></div></div></blockquote><div><br></div><div>- "Q: ChatGPT \
etc. trained on mathlib and tests"</div><div> <a \
href="https://github.com/leanprover-community/mathlib/issues/17919">https://github.com/leanprover-community/mathlib/issues/17919</a></div><div><br></div><div>- \
"Port the QuantumQ game to Cirq (SymPy (Python))?"</div><div> - <a \
href="https://quantumai.google/cirq/start/start">https://quantumai.google/cirq/start/start</a></div><div> \
- <a href="https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwxyw? \
usp=sharing">https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwxyw?usp=sharing</a></div><div><br></div><div>- \
"How does [ChatGPT] compare to Critical Thinking, Reasoning, Logic, and \
Rationality?</div><div> <a \
href="https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_rationality">https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_rationality</a></div><div> \
</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jan 4, 2023, 6:28 AM \
Christian Mascher <<a href="mailto:christian.mascher@gmx.de" rel="noreferrer" \
target="_blank">christian.mascher@gmx.de</a>> wrote:<br></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex">Hi,<br> <br>
a student of mine was aware of this chatbot and asked it about a<br>
class-assignment of his own accord. We program in Java with some extra<br>
homemade library class used by some schools in our region.<br>
<br>
The bot came up with a "solution" which was flawed in several respects:<br>
1. It used some other (unimported) classes - solution doesn't work and<br>
doesn't fit the assignment.<br>
2. It put all the code into the constructor, a typical (design and<br>
style) error for students beginning with Java.<br>
<br>
When confronted with the problem number one above, it acknowledged the<br>
fault and produced a different unrelated solution.<br>
<br>
Sooo....<br>
<br>
I was impressed how well the chatbot simulated a typical clueless human<br>
who even thinks he is smart, while his code is basically bullshit.<br>
(Probably a result of googling forums, where other learners posted their<br>
solutions to assignments with the given school library classes.) The bot<br>
clearly passed the Turing test ;-)<br>
<br>
But...<br>
<br>
I don't think the interaction was helpful for somebody who is learning<br>
to program. It is probably less helpful than conversing with other also<br>
not very knowledgeable students as they are at least reasoning humans.<br>
<br>
Talking to the bot might be fun to do in the last lesson before<br>
christmas or so. Entertaining until you realise the software is<br>
"simulating" intelligent conversation - not really talking with \
insight.<br> And that could turn out to be a waste of time.<br>
<br>
Happy new year<br>
<br>
Christian<br>
<br>
Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis:<br>
> Hi, happy NY!<br>
><br>
> ChatGPT can create, fix and explain code<br>
> <a href="https://openai.com/blog/chatgpt/#samples" rel="noreferrer noreferrer \
noreferrer" target="_blank">https://openai.com/blog/chatgpt/#samples</a><br> > \
<<a href="https://openai.com/blog/chatgpt/#samples" rel="noreferrer noreferrer \
noreferrer" target="_blank">https://openai.com/blog/chatgpt/#samples</a>><br> \
><br> > Anyone tried to incorporate it into teaching process?<br>
> Or have ideas/doubts how it ciuld help?<br>
><br>
> _______________________________________________<br>
> Edu-sig mailing list -- <a href="mailto:edu-sig@python.org" rel="noreferrer \
noreferrer" target="_blank">edu-sig@python.org</a><br> > To unsubscribe send an \
email to <a href="mailto:edu-sig-leave@python.org" rel="noreferrer noreferrer" \
target="_blank">edu-sig-leave@python.org</a><br> > <a \
href="https://mail.python.org/mailman3/lists/edu-sig.python.org/" rel="noreferrer \
noreferrer noreferrer" \
target="_blank">https://mail.python.org/mailman3/lists/edu-sig.python.org/</a><br> \
> Member address: <a href="mailto:christian.mascher@gmx.de" rel="noreferrer \
noreferrer" target="_blank">christian.mascher@gmx.de</a><br> \
_______________________________________________<br> Edu-sig mailing list -- <a \
href="mailto:edu-sig@python.org" rel="noreferrer noreferrer" \
target="_blank">edu-sig@python.org</a><br> To unsubscribe send an email to <a \
href="mailto:edu-sig-leave@python.org" rel="noreferrer noreferrer" \
target="_blank">edu-sig-leave@python.org</a><br> <a \
href="https://mail.python.org/mailman3/lists/edu-sig.python.org/" rel="noreferrer \
noreferrer noreferrer" \
target="_blank">https://mail.python.org/mailman3/lists/edu-sig.python.org/</a><br> \
Member address: <a href="mailto:wes.turner@gmail.com" rel="noreferrer noreferrer" \
target="_blank">wes.turner@gmail.com</a><br> </blockquote></div>
</blockquote></div></div>
_______________________________________________
Edu-sig mailing list -- edu-sig@python.org
To unsubscribe send an email to edu-sig-leave@python.org
https://mail.python.org/mailman3/lists/edu-sig.python.org/
Member address: python-edu-sig@progressive-comp.com
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic