[prev in list] [next in list] [prev in thread] [next in thread]
List: wsf-javascript-dev
Subject: Re: [Dev] [GSOC2016]Proposal 4: [ML] Ensemble Methods Support for WSO2 Machine Learner
From: Supun Sethunga <supuns () wso2 ! com>
Date: 2016-06-30 6:28:29
Message-ID: CAMUGTrrb8Hct7uR_toEvZHqPMCEcLg3cyYiATOHgmdzK0KYTbw () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/related)]
[Attachment #4 (multipart/alternative)]
Hi Misgana,
I tried to run you script with your forks of carbon-ml [1] and product-ml
[2], but noticed several issues.
1. There are some issues in the model-generation.sh. (A whitespace at
[3]. "./jq" should be "jq").
2. There's a typo in the h2.sql script [4] (missing comma), hence the DB
table is not getting created. So any step beyond creating hyper-parameters
aren't working.
3. In the SQL statement [5], you are setting only one parameter, while
the prepared statement expect two parameters.
Can you fix those please?
Are the two repos different from your working copy?
[1] https://github.com/zemoel/carbon-ml
[2] https://github.com/zemoel/product-ml
[3]
https://github.com/zemoel/product-ml/blob/master/modules/samples/tuned/stacking/model-generation.sh#L60
[4]
https://github.com/zemoel/carbon-ml/blob/master/features/ml/org.wso2.carbon.ml.database.feature/src/main/resources/dbscripts/h2.sql#L152
[5]
https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.datab \
ase/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2183
Regards,
Supun
On Wed, Jun 29, 2016 at 6:52 PM, Misgana Negassi <
negassim@tf.uni-freiburg.de> wrote:
> Hi Supun,
> I logged into the server using UI. As you can see from the screenshot
> below, the installed features are of the previous version of server. Do you
> think, this could be the problem?[image: screenshot]
>
> Best,
> Misgana
>
>
> On 29.06.2016 08:06, Supun Sethunga wrote:
>
> Hi,
>
> I will check and let you know
>
> Regards,
> Supun
>
> On Tue, Jun 28, 2016 at 5:33 AM, Misgana Negassi <
> negassim@tf.uni-freiburg.de> wrote:
>
> > Hi Supun,
> >
> > These are the links for the code I changed.
> >
> >
> > https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/DatabaseService.java#L422
> >
> > https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.d \
> > atabase/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2173
> >
> > https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.d \
> > atabase/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2506
> >
> > https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.commons/src/main/java/org/wso2/carbon/ml/commons/domain/Workflow.java#L267
> >
> > https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.commons/src/main/java/org/wso2/carbon/ml/commons/domain/Workflow.java#L280
> >
> > Best,
> > Misgana
> >
> >
> > On 27.06.2016 08:53, Supun Sethunga wrote:
> >
> > Hi Misgana,
> >
> > Can you commit all the changes to your fork of carbon-ml, and share us
> > the link. So that I can check whats happening.
> >
> > Also, do I need to manually edit the machinelearner.xml file(s) to
> > > include Stacking?
> >
> > Yes. No need to add hyper-parameters, since we can't predefine them.
> >
> > Thanks,
> > Supun
> >
> > On Mon, Jun 27, 2016 at 4:33 AM, Misgana Negassi <
> > <negassim@tf.uni-freiburg.de>negassim@tf.uni-freiburg.de> wrote:
> >
> > > Hello Supun,
> > > sorry to bother you again with the same problem but I am not making any
> > > progress right now :(
> > > I can't get the sample script running.
> > >
> > > I tried two approaches:
> > > A) Using the server provided and applying patches.
> > > Problem: I can not apply my patch. I suspect the problem is that my
> > > patch is of a newer version. The server's carbon ml is 1.1.1. and mine is
> > > 1.1.2. Can I somewhere change the version number when I build my project?
> > >
> > > B) Building carbon and then product-ml and then unpacking
> > > modules/distribution/target/wso2ml-2.0.0-SNAPSHOT.zip
> > > I can start the server with the right components except for one
> > > exception:
> > >
> > > log4j:ERROR Could not instantiate appender named "CARBON_MEMORY".
> > > log4j:ERROR Could not instantiate class
> > > [org.wso2.carbon.logging.service.appender.CarbonMemoryAppender].
> > > java.lang.ClassNotFoundException:
> > > org.wso2.carbon.logging.service.appender.CarbonMemoryAppender cannot be
> > > found by org.wso2.carbon.logging_4.4.3
> > > at
> > > org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:501)
> > >
> > >
> > > However, when I try to run my example script there seems to be nothing
> > > happening on the server side. No exceptions but also the debugger does not
> > > stop when I expect it would. It seems something is crashing but the
> > > exceptions are swallowed. Is there a way to increase the debug output?
> > >
> > > Also, do I need to manually edit the machinelearner.xml file(s) to
> > > include Stacking?
> > >
> > >
> > > On 21.06.2016 05:41, Supun Sethunga wrote:
> > >
> > > Hi Misgana,
> > >
> > > I have two issues:
> > > 1. Issue:
> > >
> > > Exception in thread "pool-17-thread-7" java.lang.IllegalArgumentException:
> > > > No enum constant org.wso2.carbon.ml.commons.con
> > > > stants.MLConstants.SUPERVISED_ALGORITHM.STACKING
> > > > at java.lang.Enum.valueOf(Enum.java:238)
> > > > at org.wso2.carbon.ml.commons.constants.MLConstants$SUPERVISED_
> > > > ALGORITHM.valueOf(MLConstants.java:225)
> > > > at org.wso2.carbon.ml.core.spark.algorithms.SupervisedSparkMode
> > > > lBuilder.build(SupervisedSparkModelBuilder.java:158)
> > >
> > > Which approach did you try? putting the jar as a patch (patch9999)? or
> > > build the carbon-ml and product ml respectively? If it was patching method,
> > > did you build org.wso2.carbon.ml.commons as well?
> > >
> > > Caused by: org.wso2.carbon.ml.database.exceptions.DatabaseHandlerException:
> > > > An error occurred while inserting hyper parameter to the database: Value
> > > > too long for column "VALUE VARCHAR(50)": "'rO0ABXNyABNqYXZhLnV0aWwuQXJy
> > > > YXlMaXN0eIHSHZnHYZ0DAAFJAARzaXpleHAAAAACdwQAAAACc3IAEWphdmEu
> > > > dXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2F... (372)"; SQL statement:
> > > > INSERT INTO ML_HYPER_PARAMETER(ANALYSIS_ID, ALGORITHM_NAME, NAME,
> > > > VALUE, LAST_MODIFIED_TIME) VALUES(?,?,?,?, CURRENT_TIMESTAMP()) [90005-140]
> > > > The Logic is: The hyperparameters of base-algorithms are serialized and
> > > > passed in the product-ml hyperparamters script file.This is a serialized
> > > > string of List of Maps, where each map contains hyperparameters of each
> > > > base-algorithm and will be deserialized in buildStackingModel() in
> > > > SupervisedSparkModelBuilder and fed to Stacking train().
> > > > Current idea to solve it is:
> > > > 1. Increase column capacity more than VALUE VARCHAR(50)
> > > > 2. Change serialization logic
> > > > What do you think?
> > >
> > >
> > > I believe we don't need to pass a serialized a map and put the string in
> > > DB. Can we store the hyperparamerets as follows, in the ML_HYPER_PARAMETER
> > > table?
> > >
> > > *ANALYSIS_ID* *ALGORITHM_NAME* *NAME* *VALUE* *LAST_MODIFIED_TIME*
> > > 123 Meta_Algorithm_1 Learning Rate 0.001 <some_timestamp>
> > > 123 Meta_Algorithm_1 Num_Trees 10 <some_timestamp>
> > > 123 Meta_Algorithm_1 Max_Depth 0.001 <some_timestamp>
> > > 123 Meta_Algorithm_1 Seed 4567 <some_timestamp>
> > > 123 Meta_Algorithm_2 Learning Rate 0.001 <some_timestamp>
> > > 123 Meta_Algorithm_2 Iterations 100 <some_timestamp>
> > > 123 Meta_Algorithm_2 Seed 789 <some_timestamp>
> > > … … … … …
> > > 123 Base_Algorithm Learning Rate 0.001 <some_timestamp>
> > > 123 Base_Algorithm Iterations 100 <some_timestamp>
> > > 123 Base_Algorithm Seed 6325 <some_timestamp>
> > >
> > > Do you see any complications/issues in doing so?
> > >
> > >
> > >
> > > Thanks,
> > > Supun
> > >
> > > On Mon, Jun 20, 2016 at 6:58 PM, Misgana Negassi <
> > > <negassim@tf.uni-freiburg.de>negassim@tf.uni-freiburg.de> wrote:
> > >
> > > > Hi Supun,
> > > >
> > > > I have two issues:
> > > > 1. Issue:
> > > >
> > > > Exception in thread "pool-17-thread-7"
> > > > java.lang.IllegalArgumentException: No enum constant
> > > > org.wso2.carbon.ml.commons.constants.MLConstants.SUPERVISED_ALGORITHM.STACKING
> > > > at java.lang.Enum.valueOf(Enum.java:238)
> > > > at
> > > > org.wso2.carbon.ml.commons.constants.MLConstants$SUPERVISED_ALGORITHM.valueOf(MLConstants.java:225)
> > > > at
> > > > org.wso2.carbon.ml.core.spark.algorithms.SupervisedSparkModelBuilder.build(SupervisedSparkModelBuilder.java:158)
> > > >
> > > >
> > > > It doesn't recognize STACKING although I have built carbonml as you
> > > > have described.
> > > >
> > > > 2. Issue
> > > >
> > > >
> > > > Caused by:
> > > > org.wso2.carbon.ml.database.exceptions.DatabaseHandlerException: An error
> > > > occurred while inserting hyper parameter to the database: Value too long
> > > > for column "VALUE VARCHAR(50)":
> > > > "'rO0ABXNyABNqYXZhLnV0aWwuQXJyYXlMaXN0eIHSHZnHYZ0DAAFJAARzaXpleHAAAAACdwQAAAACc3IAEWphdmEudXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2F...
> > > > (372)"; SQL statement:
> > > > INSERT INTO ML_HYPER_PARAMETER(ANALYSIS_ID, ALGORITHM_NAME, NAME,
> > > > VALUE, LAST_MODIFIED_TIME) VALUES(?,?,?,?, CURRENT_TIMESTAMP()) [90005-140]
> > > >
> > > > The Logic is: The hyperparameters of base-algorithms are serialized and
> > > > passed in the product-ml hyperparamters script file.This is a serialized
> > > > string of List of Maps, where each map contains hyperparameters of each
> > > > base-algorithm and will be deserialized in buildStackingModel() in
> > > > SupervisedSparkModelBuilder and fed to Stacking train().
> > > >
> > > > Current idea to solve it is:
> > > > 1. Increase column capacity more than VALUE VARCHAR(50)
> > > > 2. Change serialization logic
> > > >
> > > > What do you think?
> > > >
> > > > Regards,
> > > > Misgana
> > > >
> > > >
> > > >
> > > >
> > > > Can you specify why what was the error? Is there any error/stack trace?
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > *Supun Sethunga*
> > > Senior Software Engineer
> > > WSO2, Inc.
> > > <http://wso2.com/>http://wso2.com/
> > > lean | enterprise | middleware
> > > Mobile : +94 716546324 <%2B94%20716546324>
> > > Blog: <http://supunsetunga.blogspot.com>
> > > http://supunsetunga.blogspot.com
> > >
> > >
> > >
> >
> >
> > --
> > *Supun Sethunga*
> > Senior Software Engineer
> > WSO2, Inc.
> > <http://wso2.com/>http://wso2.com/
> > lean | enterprise | middleware
> > Mobile : +94 716546324 <%2B94%20716546324>
> > Blog: <http://supunsetunga.blogspot.com>http://supunsetunga.blogspot.com
> >
> >
> >
>
>
> --
> *Supun Sethunga*
> Senior Software Engineer
> WSO2, Inc.
> <http://wso2.com/>http://wso2.com/
> lean | enterprise | middleware
> Mobile : +94 716546324
> Blog: <http://supunsetunga.blogspot.com>http://supunsetunga.blogspot.com
>
>
>
--
*Supun Sethunga*
Senior Software Engineer
WSO2, Inc.
http://wso2.com/
lean | enterprise | middleware
Mobile : +94 716546324
Blog: http://supunsetunga.blogspot.com
[Attachment #7 (text/html)]
<div dir="ltr">Hi Misgana,<div><br></div><div>I tried to run you script with your \
forks of carbon-ml [1] and product-ml [2], but noticed several \
issues.</div><div><ol><li>There are some issues in the model-generation.sh. (A \
whitespace at [3]. "./jq" should be "jq").</li><li>There's a \
typo in the h2.sql script [4] (missing comma), hence the DB table is not getting \
created. So any step beyond creating hyper-parameters aren't working.</li><li>In \
the SQL statement [5], you are setting only one parameter, while the prepared \
statement expect two parameters.</li></ol><div>Can you fix those please? \
</div></div><div>Are the two repos different from your working \
copy?</div><div><br></div><div>[1] <a \
href="https://github.com/zemoel/carbon-ml">https://github.com/zemoel/carbon-ml</a></div><div>[2] \
<a href="https://github.com/zemoel/product-ml">https://github.com/zemoel/product-ml</a></div><div>[3] \
<a href="https://github.com/zemoel/product-ml/blob/master/modules/samples/tuned/stacki \
ng/model-generation.sh#L60">https://github.com/zemoel/product-ml/blob/master/modules/samples/tuned/stacking/model-generation.sh#L60</a></div><div>[4] \
<a href="https://github.com/zemoel/carbon-ml/blob/master/features/ml/org.wso2.carbon.m \
l.database.feature/src/main/resources/dbscripts/h2.sql#L152">https://github.com/zemoel \
/carbon-ml/blob/master/features/ml/org.wso2.carbon.ml.database.feature/src/main/resources/dbscripts/h2.sql#L152</a></div><div>[5] \
<a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon \
.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java \
#L2183">https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon. \
ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2183</a></div><div><br></div><div>Regards,</div><div>Supun</div></div><div \
class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 29, 2016 at 6:52 PM, \
Misgana Negassi <span dir="ltr"><<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
Hi Supun,<br>
I logged into the server using UI. As you can see from the
screenshot below, the installed features are of the previous version
of server. Do you think, this could be the problem?<img alt="screenshot" \
src="cid:part1.02060304.02070203@tf.uni-freiburg.de" height="144" width="586"><br> \
<br> Best,<br>
Misgana<div><div class="h5"><br>
<br>
<div>On 29.06.2016 08:06, Supun Sethunga
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>I will check and let you know</div>
<div><br>
</div>
<div>Regards,</div>
<div>Supun</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jun 28, 2016 at 5:33 AM,
Misgana Negassi <span dir="ltr"><<a \
href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex"> <div text="#000000" bgcolor="#FFFFFF"> Hi Supun,<br>
<br>
These are the links for the code I changed.<br>
<br>
<a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/o \
rg.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/DatabaseService.java#L422" \
target="_blank">https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2 \
.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/DatabaseService.java#L422</a><br>
<a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carb \
on.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2173" \
target="_blank">https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2 \
.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2173</a><br>
<a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carb \
on.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2506" \
target="_blank">https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2 \
.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2506</a><br>
<a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carb \
on.ml.commons/src/main/java/org/wso2/carbon/ml/commons/domain/Workflow.java#L267" \
target="_blank">https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2 \
.carbon.ml.commons/src/main/java/org/wso2/carbon/ml/commons/domain/Workflow.java#L267</a><br>
<a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carb \
on.ml.commons/src/main/java/org/wso2/carbon/ml/commons/domain/Workflow.java#L280" \
target="_blank">https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2 \
.carbon.ml.commons/src/main/java/org/wso2/carbon/ml/commons/domain/Workflow.java#L280</a><br>
<br>
Best,<br>
Misgana
<div>
<div><br>
<br>
<div>On 27.06.2016 08:53, Supun Sethunga wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Misgana,
<div><br>
</div>
<div>Can you commit all the changes to your fork
of carbon-ml, and share us the link. So that I
can check whats happening.</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span \
style="font-size:12.8px">Also, do I need to manually edit the machinelearner.xml \
file(s) to include Stacking?</span></blockquote>
<div>Yes. No need to add hyper-parameters, since
we can't predefine them.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Supun </div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Jun 27, 2016 at
4:33 AM, Misgana Negassi <span dir="ltr"><<a \
href="mailto:negassim@tf.uni-freiburg.de" target="_blank"></a><a \
href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex"> <div text="#000000" \
bgcolor="#FFFFFF"> Hello Supun,<br>
sorry to bother you again with the same
problem but I am not making any progress
right now :(<br>
I can't get the sample script running.<br>
<br>
I tried two approaches:<br>
A) Using the server provided and applying
patches.<br>
Problem: I can not apply my patch. I suspect
the problem is that my patch is of a newer
version. The server's carbon ml is 1.1.1.
and mine is 1.1.2. Can I somewhere change
the version number when I build my project?<br>
<br>
B) Building carbon and then product-ml and
then unpacking
modules/distribution/target/wso2ml-2.0.0-SNAPSHOT.zip<br>
I can start the server with the right
components except for one exception:<br>
<br>
log4j:ERROR Could not instantiate appender
named "CARBON_MEMORY".<br>
log4j:ERROR Could not instantiate class
\
[org.wso2.carbon.logging.service.appender.CarbonMemoryAppender].<br> \
java.lang.ClassNotFoundException:
\
org.wso2.carbon.logging.service.appender.CarbonMemoryAppender cannot be found by
org.wso2.carbon.logging_4.4.3<br>
at
org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:501)<br>
<br>
<br>
However, when I try to run my example script
there seems to be nothing happening on the
server side. No exceptions but also the
debugger does not stop when I expect it
would. It seems something is crashing but
the exceptions are swallowed. Is there a way
to increase the debug output?<br>
<br>
Also, do I need to manually edit the
machinelearner.xml file(s) to include
Stacking?
<div>
<div><br>
<br>
<div>On 21.06.2016 05:41, Supun Sethunga
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Misgana,
<div><br>
</div>
<div><span style="font-size:12.8px">I
have two issues:</span><br \
style="font-size:12.8px"> <span style="font-size:12.8px">1.
Issue:</span><br style="font-size:12.8px">
<br style="font-size:12.8px">
<blockquote class="gmail_quote" \
style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span \
style="font-size:12.8px">Exception
in thread "pool-17-thread-7"
java.lang.IllegalArgumentExcep</span><span \
style="font-size:12.8px">tion: No enum constant
org.wso2.carbon.ml.commons.con</span><span \
style="font-size:12.8px">stants.MLConstants.SUPERVISED_</span><span \
style="font-size:12.8px">ALGORITHM.STACKING<br>
</span><span style="font-size:12.8px">
at
java.lang.Enum.valueOf(Enum.ja</span><span \
style="font-size:12.8px">va:238)<br>
</span><span style="font-size:12.8px">
at
org.wso2.carbon.ml.commons.con</span><span \
style="font-size:12.8px">stants.MLConstants$SUPERVISED_</span><span \
style="font-size:12.8px">ALGORITHM.valueOf(MLConstants.</span><span \
style="font-size:12.8px">java:225)<br>
</span><span style="font-size:12.8px">
at
org.wso2.carbon.ml.core.spark.</span><span \
style="font-size:12.8px">algorithms.SupervisedSparkMode</span><span \
style="font-size:12.8px">lBuilder.build(SupervisedSpark</span><span \
style="font-size:12.8px">ModelBuilder.java:158)</span></blockquote> </div>
<div><span style="font-size:12.8px">Which
approach did you try? putting
the jar as a patch (patch9999)?
or build the carbon-ml and
product ml respectively? If it
was patching method, did you
build org.wso2.carbon.ml.commons
as well?</span></div>
<div>
<div><br>
</div>
<blockquote class="gmail_quote" \
style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span \
style="font-size:12.8px">Caused by:
org.wso2.carbon.ml.database.ex</span><span \
style="font-size:12.8px">ceptions.DatabaseHandlerExcept</span><span \
style="font-size:12.8px">ion: An error occurred while
inserting hyper parameter to
the database: Value too long
for column "VALUE
VARCHAR(50)":
\
"'rO0ABXNyABNqYXZhLnV0aWwuQXJy</span><span \
style="font-size:12.8px">YXlMaXN0eIHSHZnHYZ0DAAFJAARzaX</span><span \
style="font-size:12.8px">pleHAAAAACdwQAAAACc3IAEWphdmEu</span><span \
style="font-size:12.8px">dXRpbC5IYXNoTWFwBQfawcMWYNEDAA</span><span \
style="font-size:12.8px">JGAApsb2F...
(372)"; SQL statement:<br>
</span><span style="font-size:12.8px">INSERT
INTO
ML_HYPER_PARAMETER(ANALYSIS_ID</span><span \
style="font-size:12.8px">, ALGORITHM_NAME, NAME, VALUE,
LAST_MODIFIED_TIME)
VALUES(?,?,?,?,
CURRENT_TIMESTAMP())
[90005-140]</span><br \
style="font-size:12.8px"> <span style="font-size:12.8px">The
Logic is: The hyperparameters
of base-algorithms are
serialized and passed in the
product-ml hyperparamters
script file.This is a
serialized string of List of
Maps, where each map contains
hyperparameters of each
base-algorithm and will be
deserialized in
buildStackingModel() in
SupervisedSparkModelBuilder
and fed to Stacking train().</span><br \
style="font-size:12.8px">
<span style="font-size:12.8px">Current
idea to solve it is:<br>
</span><span style="font-size:12.8px"> 1.
Increase column capacity more
than VALUE VARCHAR(50)<br>
</span><span style="font-size:12.8px"> 2.
Change serialization logic</span><br \
style="font-size:12.8px">
<span style="font-size:12.8px">What
do you think?</span></blockquote>
<div><br>
</div>
<div>I believe we don't need to
pass a serialized a map and put
the string in DB. Can we store
the hyperparamerets as follows,
in the ML_HYPER_PARAMETER table?</div>
<div><br>
</div>
<div>
<table cellspacing="0" border="0">
<colgroup width="118"></colgroup>
<colgroup width="164"></colgroup>
<colgroup width="112"></colgroup>
<colgroup width="85"></colgroup>
<colgroup width="155"></colgroup>
<tbody>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center"><b>ANALYSIS_ID</b></td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><b>ALGORITHM_NAME</b></td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><b>NAME</b></td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><b>VALUE</b></td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><b>LAST_MODIFIED_TIME</b></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Meta_Algorithm_1</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Learning Rate</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">0.001</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Meta_Algorithm_1</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Num_Trees</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">10</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Meta_Algorithm_1</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Max_Depth</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">0.001</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Meta_Algorithm_1</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Seed</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">4567</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Meta_Algorithm_2</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Learning Rate</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">0.001</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Meta_Algorithm_2</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Iterations</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">100</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Meta_Algorithm_2</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Seed</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">789</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">…</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">…</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">…</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">…</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">…</td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Base_Algorithm</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Learning Rate</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">0.001</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Base_Algorithm</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Iterations</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">100</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
<tr>
<td style="border:1px solid rgb(0,0,0)" \
height="17" align="center">123</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Base_Algorithm</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">Seed</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center">6325</td>
<td style="border:1px solid rgb(0,0,0)" \
align="center"><some_timestamp></td> </tr>
</tbody>
</table>
</div>
<div><br>
</div>
<div>Do you see any
complications/issues in doing
so?</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Supun </div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Jun
20, 2016 at 6:58 PM, Misgana
Negassi <span dir="ltr"><<a \
href="mailto:negassim@tf.uni-freiburg.de" target="_blank"></a><a \
href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 \
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Supun,<br>
<br>
I have two issues:<br>
1. Issue:<br>
<br>
Exception in thread
"pool-17-thread-7"
java.lang.IllegalArgumentException:
No enum constant
org.wso2.carbon.ml.commons.constants.MLConstants.SUPERVISED_ALGORITHM.STACKING<br>
at
java.lang.Enum.valueOf(Enum.java:238)<br>
at
org.wso2.carbon.ml.commons.constants.MLConstants$SUPERVISED_ALGORITHM.valueOf(MLConstants.java:225)<br>
at
org.wso2.carbon.ml.core.spark.algorithms.SupervisedSparkModelBuilder.build(SupervisedSparkModelBuilder.java:158)<br>
<br>
<br>
It doesn't recognize STACKING
although I have built carbonml
as you have described.<br>
<br>
2. Issue<br>
<br>
<br>
Caused by:
\
org.wso2.carbon.ml.database.exceptions.DatabaseHandlerException: An error occurred \
while inserting hyper parameter to
the database: Value too long for
column "VALUE VARCHAR(50)":
\
"'rO0ABXNyABNqYXZhLnV0aWwuQXJyYXlMaXN0eIHSHZnHYZ0DAAFJAARzaXpleHAAAAACdwQAAAACc3IAEWphdmEudXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2F...
(372)"; SQL statement:<br>
INSERT INTO
ML_HYPER_PARAMETER(ANALYSIS_ID,
ALGORITHM_NAME, NAME, VALUE,
LAST_MODIFIED_TIME)
VALUES(?,?,?,?,
CURRENT_TIMESTAMP()) [90005-140]<br>
<br>
The Logic is: The
hyperparameters of
base-algorithms are serialized
and passed in the product-ml
hyperparamters script file.This
is a serialized string of List
of Maps, where each map contains
hyperparameters of each
base-algorithm and will be
deserialized in
buildStackingModel() in
SupervisedSparkModelBuilder and
fed to Stacking train().<br>
<br>
Current idea to solve it is:<br>
1. Increase column capacity
more than VALUE VARCHAR(50)<br>
2. Change serialization logic<br>
<br>
What do you think?<br>
<br>
Regards,<br>
Misgana
<div>
<div><br>
<br>
<br>
<br>
<blockquote class="gmail_quote" \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Can you \
specify why what was the error? Is there
any error/stack trace?<br>
</blockquote>
<br>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><b><font \
face="verdana,
sans-serif" size="1">Supun
Sethunga</font></b>
<div><font face="verdana,
sans-serif" size="1">Senior
Software
Engineer<br>
</font></div>
<div><font face="verdana,
sans-serif" size="1">WSO2,
Inc.</font></div>
<div><font face="verdana,
sans-serif" size="1"><a \
href="http://wso2.com/" target="_blank"></a><a href="http://wso2.com/" \
target="_blank">http://wso2.com/</a><br> </font></div>
<div><font face="verdana,
sans-serif" size="1">lean
| enterprise |
middleware</font></div>
<div><font face="verdana,
sans-serif" size="1">Mobile
: <a \
href="tel:%2B94%20716546324" value="+94716546324" target="_blank">+94
716546324</a></font></div>
<div><font face="verdana,
sans-serif" size="1">Blog: \
<a href="http://supunsetunga.blogspot.com" target="_blank"></a><a \
href="http://supunsetunga.blogspot.com" \
target="_blank">http://supunsetunga.blogspot.com</a></font></div> </div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><b><font \
face="verdana,
sans-serif" size="1">Supun
Sethunga</font></b>
<div><font face="verdana,
sans-serif" size="1">Senior
Software Engineer<br>
</font></div>
<div><font face="verdana,
sans-serif" size="1">WSO2,
Inc.</font></div>
<div><font face="verdana,
sans-serif" size="1"><a \
href="http://wso2.com/" target="_blank"></a><a href="http://wso2.com/" \
target="_blank">http://wso2.com/</a><br> </font></div>
<div><font face="verdana,
sans-serif" size="1">lean |
enterprise |
middleware</font></div>
<div><font face="verdana,
sans-serif" size="1">Mobile :
<a href="tel:%2B94%20716546324" \
value="+94716546324" target="_blank">+94
716546324</a></font></div>
<div><font face="verdana,
sans-serif" size="1">Blog: <a \
href="http://supunsetunga.blogspot.com" target="_blank"></a><a \
href="http://supunsetunga.blogspot.com" \
target="_blank">http://supunsetunga.blogspot.com</a></font></div> </div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><b><font face="verdana,
sans-serif" size="1">Supun
Sethunga</font></b>
<div><font face="verdana,
sans-serif" size="1">Senior
Software Engineer<br>
</font></div>
<div><font face="verdana,
sans-serif" size="1">WSO2, Inc.</font></div>
<div><font face="verdana,
sans-serif" size="1"><a \
href="http://wso2.com/" target="_blank"></a><a href="http://wso2.com/" \
target="_blank">http://wso2.com/</a><br> </font></div>
<div><font face="verdana,
sans-serif" size="1">lean |
enterprise | middleware</font></div>
<div><font face="verdana,
sans-serif" size="1">Mobile :
<a href="tel:%2B94%20716546324" \
value="+94716546324" target="_blank">+94 716546324</a></font></div> <div><font \
face="verdana, sans-serif" size="1">Blog: <a \
href="http://supunsetunga.blogspot.com" target="_blank"></a><a \
href="http://supunsetunga.blogspot.com" \
target="_blank">http://supunsetunga.blogspot.com</a></font></div> </div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div \
class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><b><font face="verdana, sans-serif" size="1">Supun \
Sethunga</font></b><div><font face="verdana, sans-serif" size="1">Senior Software \
Engineer<br></font></div><div><font face="verdana, sans-serif" size="1">WSO2, \
Inc.</font></div><div><font face="verdana, sans-serif" size="1"><a \
href="http://wso2.com/" \
target="_blank">http://wso2.com/</a><br></font></div><div><font face="verdana, \
sans-serif" size="1">lean | enterprise | middleware</font></div><div><font \
face="verdana, sans-serif" size="1">Mobile : +94 716546324</font></div><div><font \
face="verdana, sans-serif" size="1">Blog: <a href="http://supunsetunga.blogspot.com" \
target="_blank">http://supunsetunga.blogspot.com</a></font></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
--94eb2c070ac0a7b09c053678cfb1--
["2016-06-29-151723_1479x311_scrot.png" (image/png)]
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic