[prev in list] [next in list] [prev in thread] [next in thread]
List: cassandra-user
Subject: Re: Cassandra & Spark
From: Affan Syed <asyed () an10 ! io>
Date: 2018-08-25 6:58:15
Message-ID: CAGhWrLM_hiSni1F=vkSPE0HFfX92ysEiYDgna==Wa6ubH+OWaQ () mail ! gmail ! com
[Download RAW message or body]
Nope, Spark cassandra connector leverages data locality and get tremendous
improvements due to localitty.
- Affan
On Sat, Aug 25, 2018 at 11:25 AM CharSyam <charsyam@gmail.com> wrote:
> Spark can read hdfs directly so locality is important but Spark can't read
> Cassandra data directly it can only connect by api. So I think you don't
> need to install them a same node
>
> 2018ë…„ 8ì›” 25ì¼ (í† ) 오후 3:16, Affan Syed <asyed@an10.io>ë‹˜ì´ ìž‘ì„±:
>
>> Tobias,
>>
>> This is very interesting. Can I inquire a bit more on why you have both
>> C* and Kudu in the system?
>>
>> Wouldnt keeping just Kudu work (that was its initial purpose?). Is there
>> something to do with its production readiness? I ask as we have a similar
>> concern as well.
>>
>> Finally, how are your dashboard apps talking to Kudu? Is there a backend
>> that talks via impala, or do you have some calls to bash level scripts
>> communicating over some file system?
>>
>>
>>
>> - Affan
>>
>>
>> On Thu, Jun 8, 2017 at 7:01 PM Tobias Eriksson <
>> tobias.eriksson@qvantel.com> wrote:
>>
>>> Hi
>>>
>>> What I wanted was a dashboard with graphs/diagrams and it should not
>>> take minutes for the page to load
>>>
>>> Thus, it was a problem to have Spark with Cassandra, and not solving the
>>> parallelization to such an extent that I could have the diagrams rendered
>>> in seconds.
>>>
>>> Now with Kudu we get some decent results rendering the diagrams/graphs
>>>
>>>
>>>
>>> The way we transfer data from Cassandra which is the Production system
>>> storage to Kudu, is through an Apache Kafka topic (or many topics actually)
>>> and then we have an application which ingests the data into Kudu
>>>
>>>
>>>
>>>
>>>
>>> Other Systems -- > Domain Storage App(s) -- > Cassandra -- > KAFKA -- >
>>> KuduIngestion App -- > Kudu < -- Dashboard App(s)
>>>
>>>
>>>
>>>
>>>
>>> If you want to play with really fast analytics then perhaps consider
>>> looking at Apache Ignite
>>>
>>> https://ignite.apache.org
>>>
>>> Which then act as a layer between Cassandra and your applications
>>> storing into Cassandra (memory datagrid I think it is called)
>>>
>>> Basically, think of it as a big cache
>>>
>>> It is an in-memory thingi ☺
>>>
>>> And then you can run some super fast queries
>>>
>>>
>>>
>>> -Tobias
>>>
>>>
>>>
>>> *From: *DuyHai Doan <doanduyhai@gmail.com>
>>> *Date: *Thursday, 8 June 2017 at 15:42
>>> *To: *Tobias Eriksson <tobias.eriksson@qvantel.com>
>>> *Cc: *한 승호 <shhan87@outlook.com>, "user@cassandra.apache.org" <
>>> user@cassandra.apache.org>
>>> *Subject: *Re: Cassandra & Spark
>>>
>>>
>>>
>>> Interesting
>>>
>>>
>>>
>>> Tobias, when you said "Instead we transferred the data to Apache Kudu",
>>> did you transfer all Cassandra data into Kudu from with a single migration
>>> and then tap into Kudo for aggregation or did you run data import every
>>> day/week/month from Cassandra into Kudu ?
>>>
>>>
>>>
>>> From my point of view, the difficulty is not to have a static set of
>>> data and run aggregation on it, there are a lot of alternatives out there.
>>> The difficulty is to be able to run analytics on a live/production/changing
>>> dataset with all the data movement & update that it implies.
>>>
>>>
>>>
>>> Regards
>>>
>>>
>>>
>>> On Thu, Jun 8, 2017 at 3:37 PM, Tobias Eriksson <
>>> tobias.eriksson@qvantel.com> wrote:
>>>
>>> Hi
>>>
>>> Something to consider before moving to Apache Spark and Cassandra
>>>
>>> I have a background where we have tons of data in Cassandra, and we
>>> wanted to use Apache Spark to run various jobs
>>>
>>> We loved what we could do with Spark, BUT….
>>>
>>> We realized soon that we wanted to run multiple jobs in parallel
>>>
>>> Some jobs would take 30 minutes and some 45 seconds
>>>
>>> Spark is by default arranged so that it will take up all the resources
>>> there is, this can be tweaked by using Mesos or Yarn
>>>
>>> But even with Mesos and Yarn we found it complicated to run multiple
>>> jobs in parallel.
>>>
>>> So eventually we ended up throwing out Spark,
>>>
>>> Instead we transferred the data to Apache Kudu, and then we ran our
>>> analysis on Kudu, and what a difference !
>>>
>>> "my two cents!"
>>>
>>> -Tobias
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *From: *한 승호 <shhan87@outlook.com>
>>> *Date: *Thursday, 8 June 2017 at 10:25
>>> *To: *"user@cassandra.apache.org" <user@cassandra.apache.org>
>>> *Subject: *Cassandra & Spark
>>>
>>>
>>>
>>> Hello,
>>>
>>>
>>>
>>> I am Seung-ho and I work as a Data Engineer in Korea. I need some advice.
>>>
>>>
>>>
>>> My company recently consider replacing RDMBS-based system with Cassandra
>>> and Hadoop.
>>>
>>> The purpose of this system is to analyze Cadssandra and HDFS data with
>>> Spark.
>>>
>>>
>>>
>>> It seems many user cases put emphasis on data locality, for instance,
>>> both Cassandra and Spark executor should be on the same node.
>>>
>>>
>>>
>>> The thing is, my company's data analyst team wants to analyze
>>> heterogeneous data source, Cassandra and HDFS, using Spark.
>>>
>>> So, I wonder what would be the best practices of using Cassandra and
>>> Hadoop in such case.
>>>
>>>
>>>
>>> Plan A: Both HDFS and Cassandra with NodeManager(Spark Executor) on the
>>> same node
>>>
>>>
>>>
>>> Plan B: Cassandra + Node Manager / HDFS + NodeManager in each node
>>> separately but the same cluster
>>>
>>>
>>>
>>>
>>>
>>> Which would be better or correct, or would be a better way?
>>>
>>>
>>>
>>> I appreciate your advice in advance :)
>>>
>>>
>>>
>>> Best Regards,
>>>
>>> Seung-Ho Han
>>>
>>>
>>>
>>>
>>>
>>> Windows 10ìš© ë©”ì¼ <https://go.microsoft.com/fwlink/?LinkId=550986>ì—ì„œ 보냄
>>>
>>>
>>>
>>>
>>>
>>
[Attachment #3 (text/html)]
<div dir="ltr">Nope, Spark cassandra connector leverages data locality and get \
tremendous improvements due to localitty. <div><br></div><div><br \
clear="all"><div><div dir="ltr" class="gmail_signature" \
data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">- \
Affan</div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div \
dir="ltr">On Sat, Aug 25, 2018 at 11:25 AM CharSyam <<a \
href="mailto:charsyam@gmail.com">charsyam@gmail.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Spark can read hdfs \
directly so locality is important but Spark can't read Cassandra data directly it \
can only connect by api. So I think you don't need to install them a same \
node</div><br><div class="gmail_quote"><div dir="ltr">2018ë…„ 8ì›” 25ì¼ (í† ) 오후 \
3:16, Affan Syed <<a href="mailto:asyed@an10.io" \
target="_blank">asyed@an10.io</a>>ë‹˜ì´ ìž‘ì„±:<br></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Tobias, <div><br></div><div>This is very \
interesting. Can I inquire a bit more on why you have both C* and Kudu in the system? \
</div><div><br></div><div><div><div dir="ltr" \
class="m_-2836222024595857481m_-4156022453326096597gmail_signature" \
data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Wouldnt keeping \
just Kudu work (that was its initial purpose?). Is there something to do with its \
production readiness? I ask as we have a similar concern as well. </div><div \
dir="ltr"><br></div><div>Finally, how are your dashboard apps talking to Kudu? Is \
there a backend that talks via impala, or do you have some calls to bash level \
scripts communicating over some file system?</div><div><br></div><div><br></div><div \
dir="ltr"><br></div><div dir="ltr">- \
Affan</div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div \
dir="ltr">On Thu, Jun 8, 2017 at 7:01 PM Tobias Eriksson <<a \
href="mailto:tobias.eriksson@qvantel.com" rel="noreferrer" \
target="_blank">tobias.eriksson@qvantel.com</a>> wrote:<br></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">
<div bgcolor="white" lang="EN-GB" link="blue" vlink="purple">
<div class="m_-2836222024595857481m_-4156022453326096597m_2928097255820453437WordSection1">
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">Hi<u></u><u></u></span></p> <p \
class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">What I wanted was a dashboard with \
graphs/diagrams and it should not take minutes for the page to \
load<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">Thus, it was a problem to have Spark \
with Cassandra, and not solving the parallelization to such an extent that I could \
have the diagrams rendered in seconds.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">Now with Kudu we get some decent results \
rendering the diagrams/graphs<u></u><u></u></span></p> <p class="MsoNormal"><span \
lang="EN-US" style="font-size:11.0pt;font-family:Calibri"><u></u> <u></u></span></p> \
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">The way we transfer data from Cassandra \
which is the Production system storage to Kudu, is through an Apache Kafka topic (or \
many topics actually) and then we have an application which ingests the data into \
Kudu<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri"><u></u> <u></u></span></p> <p \
class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri"><u></u> <u></u></span></p> <p \
class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Other Systems -- \
> Domain Storage App(s) -- > Cassandra -- > KAFKA -- > KuduIngestion App \
-- > Kudu < -- Dashboard App(s)<u></u><u></u></span></p> <p \
class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:Calibri"><u></u> <u></u></span></p> <p \
class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:Calibri">If \
you want to play with really fast analytics then perhaps consider looking at Apache \
Ignite<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri"><a href="https://ignite.apache.org" \
rel="noreferrer" target="_blank">https://ignite.apache.org</a><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">Which then act as a layer between \
Cassandra and your applications storing into Cassandra (memory datagrid I think it is \
called)<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">Basically, think of it as a big \
cache<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">It is an in-memory thingi </span><span \
lang="EN-US" style="font-size:11.0pt;font-family:"Apple Color \
Emoji"">☺</span><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri"><u></u><u></u></span></p> <p \
class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:Calibri">And \
then you can run some super fast queries <u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri"><u></u> <u></u></span></p> <p \
class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">-Tobias<u></u><u></u></span></p> <p \
class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri"><u></u> <u></u></span></p> <div \
style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p \
class="MsoNormal"><b><span style="font-family:Calibri;color:black">From: </span> \
</b><span style="font-family:Calibri;color:black">DuyHai Doan <<a \
href="mailto:doanduyhai@gmail.com" rel="noreferrer" \
target="_blank">doanduyhai@gmail.com</a>><br> <b>Date: </b>Thursday, 8 June 2017 \
at 15:42<br> <b>To: </b>Tobias Eriksson <<a \
href="mailto:tobias.eriksson@qvantel.com" rel="noreferrer" \
target="_blank">tobias.eriksson@qvantel.com</a>><br> <b>Cc: </b></span><span \
style="font-family:"Malgun Gothic";color:black">한</span><span \
style="font-family:Calibri;color:black"> </span><span style="font-family:"Malgun \
Gothic";color:black">승호</span><span style="font-family:Calibri;color:black"> \
<<a href="mailto:shhan87@outlook.com" rel="noreferrer" \
target="_blank">shhan87@outlook.com</a>>, "<a \
href="mailto:user@cassandra.apache.org" rel="noreferrer" \
target="_blank">user@cassandra.apache.org</a>" <<a \
href="mailto:user@cassandra.apache.org" rel="noreferrer" \
target="_blank">user@cassandra.apache.org</a>><br> <b>Subject: </b>Re: Cassandra \
& Spark<u></u><u></u></span></p> </div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Interesting <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Tobias, when you said "<span \
style="font-size:11.0pt;font-family:Calibri">Instead we transferred the data to \
Apache Kudu", did you transfer all Cassandra data into Kudu from with a single \
migration and then tap into Kudo for aggregation or did you run data import every \
day/week/month from Cassandra into Kudu ?</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">From my point \
of view, the difficulty is not to have a static set of data and run aggregation on \
it, there are a lot of alternatives out there. The difficulty is to be able to run \
analytics on a live/production/changing dataset with all the data movement & \
update that it implies.</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:Calibri">Regards</span><u></u><u></u></p> </div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Thu, Jun 8, 2017 at 3:37 PM, Tobias Eriksson <<a \
href="mailto:tobias.eriksson@qvantel.com" rel="noreferrer" \
target="_blank">tobias.eriksson@qvantel.com</a>> wrote:<u></u><u></u></p> \
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm \
6.0pt;margin-left:4.8pt;margin-right:0cm"> <div>
<div>
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">Hi</span><u></u><u></u></p> <p \
class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">Something to consider before moving to \
Apache Spark and Cassandra</span><u></u><u></u></p> <p class="MsoNormal"><span \
lang="EN-US" style="font-size:11.0pt;font-family:Calibri">I have a background where \
we have tons of data in Cassandra, and we wanted to use Apache Spark to run various \
jobs</span><u></u><u></u></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">We loved what we could do with Spark, \
BUT…. </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">We realized soon that we wanted to run \
multiple jobs in parallel</span><u></u><u></u></p> <p class="MsoNormal"><span \
lang="EN-US" style="font-size:11.0pt;font-family:Calibri">Some jobs would take 30 \
minutes and some 45 seconds</span><u></u><u></u></p> <p class="MsoNormal"><span \
lang="EN-US" style="font-size:11.0pt;font-family:Calibri">Spark is by default \
arranged so that it will take up all the resources there is, this can be tweaked by \
using Mesos or Yarn</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">But even with Mesos and Yarn we found it \
complicated to run multiple jobs in parallel.</span><u></u><u></u></p> <p \
class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:Calibri">So \
eventually we ended up throwing out Spark, </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">Instead we transferred the data to \
Apache Kudu, and then we ran our analysis on Kudu, and what a difference \
!</span><u></u><u></u></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">"my two cents!"</span><u></u><u></u></p> \
<p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri">-Tobias</span><u></u><u></u></p> <p \
class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:Calibri"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt;font-family:Calibri"> </span><u></u><u></u></p> <p \
class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:Calibri"> \
</span><u></u><u></u></p> <div style="border:none;border-top:solid #b5c4df \
1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal"><b><span \
style="font-family:Calibri;color:black">From: </span></b><span \
style="font-family:"Malgun Gothic";color:black">한</span><span \
style="font-family:Calibri;color:black"> </span><span style="font-family:"Malgun \
Gothic";color:black">승호</span><span style="font-family:Calibri;color:black"> \
<<a href="mailto:shhan87@outlook.com" rel="noreferrer" \
target="_blank">shhan87@outlook.com</a>><br> <b>Date: </b>Thursday, 8 June 2017 at \
10:25<br> <b>To: </b>"<a href="mailto:user@cassandra.apache.org" \
rel="noreferrer" target="_blank">user@cassandra.apache.org</a>" <<a \
href="mailto:user@cassandra.apache.org" rel="noreferrer" \
target="_blank">user@cassandra.apache.org</a>><br> <b>Subject: </b>Cassandra & \
Spark</span><u></u><u></u></p> </div>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal" style="background:white;text-autospace:ideograph-other">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">Hello,</span><u></u><u></u></p> <div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">I am Seung-ho and I work as a Data Engineer in Korea. I need \
some advice.</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">My company recently consider replacing RDMBS-based system \
with Cassandra and Hadoop. </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">The purpose of this system is to analyze Cadssandra and HDFS \
data with Spark.</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">It seems many user cases put emphasis on data locality, for \
instance, both Cassandra and Spark executor should be on the same \
node.</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">The thing is, my company's data analyst team wants to \
analyze heterogeneous data source, Cassandra and HDFS, using Spark. \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">So, I wonder what would be the best practices of using \
Cassandra and Hadoop in such case.</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">Plan A: Both HDFS and Cassandra with NodeManager(Spark \
Executor) on the same node</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">Plan B: Cassandra + Node Manager / HDFS + NodeManager in \
each node separately but the same cluster</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">Which would be better or correct, or would be a better \
way?</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">I appreciate your advice in advance \
:)</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222"> </span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">Best Regards,</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal" style="background:white">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Courier \
New";color:#222222">Seung-Ho Han</span><u></u><u></u></p> </div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-size:11.0pt"> </span><u></u><u></u></p> <p class="MsoNormal"><span \
lang="EN-US" style="font-size:11.0pt">Windows 10</span><span \
style="font-size:11.0pt;font-family:"Malgun Gothic"">ìš©</span><span \
style="font-size:11.0pt"> </span><span lang="EN-US" style="font-size:11.0pt"><a \
href="https://go.microsoft.com/fwlink/?LinkId=550986" rel="noreferrer" \
target="_blank"><span lang="EN-GB" style="font-family:"Malgun \
Gothic"">ë©”ì¼</span></a></span><span \
style="font-size:11.0pt;font-family:"Malgun Gothic"">ì—ì„œ</span><span \
style="font-size:11.0pt"> </span><span \
style="font-size:11.0pt;font-family:"Malgun \
Gothic"">보냄</span><u></u><u></u></p> <p class="MsoNormal"><span lang="EN-US" \
style="font-family:Helvetica"> </span><u></u><u></u></p> </div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic