[prev in list] [next in list] [prev in thread] [next in thread] 

List:       esb-java-dev
Subject:    Re: [Dev] [GSOC2016]Proposal 4: [ML] Ensemble Methods Support for WSO2 Machine Learner
From:       Nirmal Fernando <nirmal () wso2 ! com>
Date:       2016-08-30 6:38:24
Message-ID: CAJ+w7F=1rRAbsV4Lo0Ma0Eb5J1Rddajr7xf9ij+00X_u7cMRCw () mail ! gmail ! com
[Download RAW message or body]

[Attachment #2 (multipart/related)]

[Attachment #4 (multipart/alternative)]


Congratulations Misgana!! Thanks for all the hard work and we look forward
to work with you in future too.

All the very best !

On Mon, Aug 22, 2016 at 9:32 AM, Supun Sethunga <supuns@wso2.com> wrote:

> Hi Misgana,
>
> I believe you have completed all the tasks. Please make sure you submit
> everything to GSoC on time :)
>
> Regards,
> Supun
>
> On Fri, Aug 19, 2016 at 9:17 AM, Supun Sethunga <supuns@wso2.com> wrote:
>
>> Hi,
>>
>> High-level info should be enough...
>>
>> Thanks,
>> Supun
>>
>> On Thu, Aug 18, 2016 at 5:57 PM, Misgana Negassi <
>> negassim@tf.uni-freiburg.de> wrote:
>>
>>> Hi Supun,
>>>
>>> I have a question about how technical the documentation should be.
>>> Should I just explain high-level changes and how it is done,  or go deeper
>>> and talk about technical details such as which functionality are added
>>> and/or changed and how?
>>>
>>> Thank you!
>>>
>>> Misgana
>>>
>>> On 18.08.2016 12:03, Supun Sethunga wrote:
>>>
>>> Hi Misgana,
>>>
>>> Currently, your documentation [1] says "Performance Comparison". I would
>>> suggest to make it the Final Documentation for the project (rather than
>>> just the perf comparison, but including those too). So you will have to add
>>> a brief introduction to the project, the objectives, the solution, and then
>>> finally the performance comparison. In that doc, you can add the two PR's
>>> as references, for the solution section. And you can then send the link of
>>> the documentation for the final evaluation. Would that work?
>>>
>>> [1] https://docs.google.com/document/d/1bpTMiSciOqt7vfDgYglS
>>> BQucQReDs-btEa5xFjBZkDU/edit
>>>
>>> Regards,
>>> Supun
>>>
>>> On Thu, Aug 18, 2016 at 2:51 PM, Misgana Negassi <
>>> negassim@tf.uni-freiburg.de> wrote:
>>>
>>>> Dear Supun,
>>>>
>>>> How would you suggest I send link of my work for final evaluation since
>>>> I contributed to two repositories? Do you have any comments on my work
>>>> before I publish the link?
>>>>
>>>> Thank you,
>>>>
>>>> Misgana
>>>>
>>>> On 15.08.2016 18:01, Misgana Negassi wrote:
>>>>
>>>> Hi Supun,
>>>>
>>>> I updated the PR as per your comments. And also updated the
>>>> documentation.
>>>>
>>>> I'm grateful for your feedback.
>>>>
>>>> Thanks,
>>>>
>>>> Misgana
>>>>
>>>> On 15.08.2016 05:59, Supun Sethunga wrote:
>>>>
>>>> Hi Misgana,
>>>>
>>>> Added couple of comments on the PR. Can you please check them and
>>>> update?
>>>> Also in the documentation, can you add some brief description of the
>>>> project, the objectives, and the brief explanation about your solution,
>>>> apart from the accuracy numbers?
>>>>
>>>> Once you fix those, please share them both in this thread.
>>>>
>>>> Thanks,
>>>> Supun
>>>>
>>>> On Mon, Aug 15, 2016 at 9:17 AM, Misgana Negassi <
>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>
>>>>> Hi Supun,
>>>>>
>>>>> I have done all three tasks you mentioned. I would be grateful for
>>>>> your comments.
>>>>>
>>>>> Best,
>>>>> Misgana
>>>>>
>>>>> On 10.08.2016 12:52, Supun Sethunga wrote:
>>>>>
>>>>> Hi Misgana,
>>>>>
>>>>> As the GSoC Pencil down deadline is approaching, shall we wrap things
>>>>> up by:
>>>>>
>>>>>    - Finishing and sharing the final documentation (including
>>>>>    accuracy comparison and etc),
>>>>>    - Cleaning-up / Formating the code,
>>>>>    - Sending git pull request(s) to wso2/carbon-ml and
>>>>>    wso2/product-ml.
>>>>>
>>>>> Regards,
>>>>> Supun
>>>>>
>>>>> On Mon, Jul 25, 2016 at 9:24 AM, Supun Sethunga <supuns@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Misgana,
>>>>>>
>>>>>> Thats great!
>>>>>>
>>>>>> [1] The data that was passed to [2] was already encoded so when
>>>>>>> passed to the BasicEncoder, it was considered as unknown String and
>>>>>>> defaulted to 0th mapping. So, I decoded the data before passing it to
>>>>>>> getVectors() method.
>>>>>>
>>>>>> Does that means, it first decodes, before passing into the method,
>>>>>> and again encodes inside that method? If so, it's better to pass a flag
>>>>>> (boolean) to that getVectors() method along with the data, to
>>>>>> indicate whether to encode or not. That way you can avoid unnecessary
>>>>>> decoding and encoding..
>>>>>>
>>>>>> Yes, its OK to setup the demo any day at 10AM IST. Please invite
>>>>>> Nirmal (cc'ed) too.
>>>>>>
>>>>>> Thanks,
>>>>>> Supun
>>>>>>
>>>>>> On Sat, Jul 23, 2016 at 4:42 PM, Misgana Negassi <
>>>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>>>
>>>>>>> Hallo Supun,
>>>>>>>
>>>>>>> Sorry for the late response.
>>>>>>>
>>>>>>> The issue with the encoding values is resolved. Please see [1].
>>>>>>>
>>>>>>> I am now cleaning up code and preparing for the presentation. As I
>>>>>>> have mentioned in my proposal, I am attending a summer school next week. If
>>>>>>> the presentation will be next week, would be it be possible to do it early
>>>>>>> 10AM IST, any day of the week except Wednesday? If not, the other week. Are
>>>>>>> the aforementioned appointments ok?
>>>>>>>
>>>>>>> Thank you!
>>>>>>> Misgana
>>>>>>>
>>>>>>>
>>>>>>> [1] The data that was passed to [2] was already encoded so when
>>>>>>> passed to the BasicEncoder, it was considered as unknown String and
>>>>>>> defaulted to 0th mapping. So, I decoded the data before passing it to
>>>>>>> getVectors() method.
>>>>>>>
>>>>>>> [2] https://github.com/zemoel/carbon-ml/blob/master/componen
>>>>>>> ts/ml/org.wso2.carbon.ml.core/src/main/java/org/wso2/carbon/
>>>>>>> ml/core/impl/Predictor.java#L313
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 20.07.2016 06:43, Supun Sethunga wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Can you remote debug and see what are the encoding values you get at
>>>>>>> [1] ?
>>>>>>>
>>>>>>> [1] https://github.com/zemoel/carbon-ml/blob/master/componen
>>>>>>> ts/ml/org.wso2.carbon.ml.core/src/main/java/org/wso2/carbon/
>>>>>>> ml/core/impl/Predictor.java#L313
>>>>>>>
>>>>>>> On Tue, Jul 19, 2016 at 9:11 PM, Misgana Negassi <
>>>>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>>>>
>>>>>>>> Hi Supun,
>>>>>>>>
>>>>>>>> I can track the weird results down to the encodings. Here [1] it
>>>>>>>> encodes the test dataset to all zeros excluding the first column.
>>>>>>>> Do you have time in the next days for a hangout session to help me
>>>>>>>> on this issue?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>> Misgana
>>>>>>>>
>>>>>>>> [1] https://github.com/zemoel/carbon-ml/blob/master/components/m
>>>>>>>> l/org.wso2.carbon.ml.core/src/main/java/org/wso2/carbon/ml/c
>>>>>>>> ore/impl/Predictor.java#L75
>>>>>>>>
>>>>>>>>
>>>>>>>> On 15.07.2016 10:34, Misgana Negassi wrote:
>>>>>>>>
>>>>>>>> Hi Supun,
>>>>>>>>
>>>>>>>> sorry it was not a clear question. Actually, after working on it
>>>>>>>> more, I found out the problem lies when we set the attributes for the
>>>>>>>> mlmodel in BaseModelBuilder eg. includedFeatures, encodings and
>>>>>>>> responseIndex. I used to get it from the workflow which is not compatible
>>>>>>>> to the levelOneDataset.
>>>>>>>>
>>>>>>>> Currently I am trying to understand what these are and how I can
>>>>>>>> populate them to be compatible to LevelOneDataset.
>>>>>>>> My idea is to create helper method to setFeatures(name,
>>>>>>>> ImputeOption, ..) and get them, and create an empty map for encodings. For
>>>>>>>> responseIndex, I dont have idea yet.
>>>>>>>>
>>>>>>>> Can you give me a hint?
>>>>>>>> Thanks,
>>>>>>>> Misgana
>>>>>>>>
>>>>>>>> On 15.07.2016 05:57, Supun Sethunga wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Sorry I didn't get the question. can you explain it a bit?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Supun
>>>>>>>>
>>>>>>>> On Thu, Jul 14, 2016 at 6:22 PM, Misgana Negassi <
>>>>>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>>>>>
>>>>>>>>> Hi Supun,
>>>>>>>>>
>>>>>>>>> I have a question regarding the issue with the Stacking
>>>>>>>>> LevelOneDataset.
>>>>>>>>> I envoke sparkContext parallelize method to get
>>>>>>>>> JavaRDD<LabeledPoint>. But here is what I get:
>>>>>>>>> LevelOneDataset : ParallelCollectionRDD at parallelize . Do we
>>>>>>>>> expect rather MapPartitionRDD at randomSplit? If so, can you give me a hint
>>>>>>>>> how to cast/get it?
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Misgana
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 14.07.2016 08:58, Supun Sethunga wrote:
>>>>>>>>>
>>>>>>>>> Hi Misgana,
>>>>>>>>>
>>>>>>>>> That is great! Keep it up!
>>>>>>>>> Yes, let's try to solve all the issues of the existing methods
>>>>>>>>> first, before moving on to the other one. Also do not worry about the
>>>>>>>>> accuracy, because it can be increased by tuning the hyper-parameters, which
>>>>>>>>> is up to the user.. :)
>>>>>>>>>
>>>>>>>>> Once you solve the current issues, can you setup a small  demo
>>>>>>>>> with the ML team, to show how the two methods work from end to end, to some
>>>>>>>>> sample dataset?
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Supun
>>>>>>>>>
>>>>>>>>> On Thu, Jul 14, 2016 at 11:55 AM, Misgana Negassi <
>>>>>>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Supun,
>>>>>>>>>>
>>>>>>>>>> This is what I have done:
>>>>>>>>>> 1. Stacking
>>>>>>>>>>  Issue with getting the Hyperparameters in a Map of Maps  is
>>>>>>>>>> resolved now. Basically Stacking is implemented end-to-end. I am now
>>>>>>>>>> tackling a error which arises while training the Metamodel. My hypothesis
>>>>>>>>>> is that the problem lies while stacking the predictions to form the
>>>>>>>>>> levelOneDataset.
>>>>>>>>>>
>>>>>>>>>> 2. Bagging
>>>>>>>>>> Bagging is also implemented end-to-end. But the accuracy is low:
>>>>>>>>>> 67.5%. I am trying to figure out where the problem lies.
>>>>>>>>>>
>>>>>>>>>> I have commited my latest code. Once I am done with those Issues,
>>>>>>>>>> I will implement Boosting.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Misgana
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 14.07.2016 07:03, Supun Sethunga wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Can you please give us a brief update on the current status?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Supun
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 6, 2016 at 9:41 AM, Supun Sethunga <supuns@wso2.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Misgana,
>>>>>>>>>>>
>>>>>>>>>>> Can you give us a brief update?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Supun
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jul 1, 2016 at 12:40 PM, Supun Sethunga <supuns@wso2.com
>>>>>>>>>>> > wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Also, the server I am using is the one created from building
>>>>>>>>>>>>> the product-ml and then unpacking modules/distribution/target/
>>>>>>>>>>>>> wso2ml-2.0.0-SNAPSHOT.zip. The server I downloaded from WSO2
>>>>>>>>>>>>> website and patches still doesn't recognize my changes.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Does your changes take effect in the pack built from source? if
>>>>>>>>>>>> so, please proceed with it.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jun 30, 2016 at 6:13 PM, Misgana Negassi <
>>>>>>>>>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Supun,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you for pointing out the missing comma in h2.script has
>>>>>>>>>>>>> solved the problem while creating the hyper-parameters table.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have now included algorithmName as second hyperparameter and
>>>>>>>>>>>>> trying to debug the NullPointerException. I am currently trying to retrace
>>>>>>>>>>>>> my steps and see of any logical flaw in creating the
>>>>>>>>>>>>> Map<String, Map<String, String>> Hyperparameters.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, the server I am using is the one created from building
>>>>>>>>>>>>> the product-ml and then unpacking modules/distribution/target/wso2ml-2.0.0-SNAPSHOT.zip.
>>>>>>>>>>>>> The server I downloaded from WSO2 website and patches still doesn't
>>>>>>>>>>>>> recognize my changes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Misgana
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 30.06.2016 08:16, Supun Sethunga wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Misgana,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I tried to run you script with your forks of carbon-ml [1] and
>>>>>>>>>>>>> product-ml [2], but noticed several issues.
>>>>>>>>>>>>>
>>>>>>>>>>>>>    1. There are some issues in the model-generation.sh. (A
>>>>>>>>>>>>>    whitespace at [3]. "./jq" should be "jq").
>>>>>>>>>>>>>    2. There's a typo in the h2.sql script [4] (missing
>>>>>>>>>>>>>    comma), hence the DB table is not getting created. So any step beyond
>>>>>>>>>>>>>    creating hyper-parameters aren't working.
>>>>>>>>>>>>>    3. In the SQL statement [5], you are setting only one
>>>>>>>>>>>>>    parameter, while the prepared statement expect two parameters.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can you fix those please?
>>>>>>>>>>>>> Are the two repos different from your working copy?
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://github.com/zemoel/carbon-ml
>>>>>>>>>>>>> [2] https://github.com/zemoel/product-ml
>>>>>>>>>>>>> [3] https://github.com/zemoel/product-ml/blob/master/modules
>>>>>>>>>>>>> /samples/tuned/stacking/model-generation.sh#L60
>>>>>>>>>>>>> [4] https://github.com/zemoel/carbon-ml/blob/master/features
>>>>>>>>>>>>> /ml/org.wso2.carbon.ml.database.feature/src/main/resources/d
>>>>>>>>>>>>> bscripts/h2.sql#L152
>>>>>>>>>>>>> [5] https://github.com/zemoel/carbon-ml/blob/master/componen
>>>>>>>>>>>>> ts/ml/org.wso2.carbon.ml.database/src/main/java/org/wso2/car
>>>>>>>>>>>>> bon/ml/database/internal/MLDatabaseService.java#L2183
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Supun
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 29, 2016 at 6:52 PM, Misgana Negassi <
>>>>>>>>>>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Supun,
>>>>>>>>>>>>>> I logged into the server using UI. As you can see from the
>>>>>>>>>>>>>> screenshot below, the installed features are of the previous version of
>>>>>>>>>>>>>> server. Do you think, this could be the problem?[image:
>>>>>>>>>>>>>> screenshot]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>> Misgana
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 29.06.2016 08:06, Supun Sethunga wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I will check and let you know
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Supun
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 28, 2016 at 5:33 AM, Misgana Negassi <
>>>>>>>>>>>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Supun,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> These are the links for the code I changed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://github.com/zemoel/carbon-ml/blob/master/components/m
>>>>>>>>>>>>>>> l/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/
>>>>>>>>>>>>>>> ml/database/DatabaseService.java#L422
>>>>>>>>>>>>>>> https://github.com/zemoel/carbon-ml/blob/master/components/m
>>>>>>>>>>>>>>> l/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/
>>>>>>>>>>>>>>> ml/database/internal/MLDatabaseService.java#L2173
>>>>>>>>>>>>>>> https://github.com/zemoel/carbon-ml/blob/master/components/m
>>>>>>>>>>>>>>> l/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/
>>>>>>>>>>>>>>> ml/database/internal/MLDatabaseService.java#L2506
>>>>>>>>>>>>>>> https://github.com/zemoel/carbon-ml/blob/master/components/m
>>>>>>>>>>>>>>> l/org.wso2.carbon.ml.commons/src/main/java/org/wso2/carbon/m
>>>>>>>>>>>>>>> l/commons/domain/Workflow.java#L267
>>>>>>>>>>>>>>> https://github.com/zemoel/carbon-ml/blob/master/components/m
>>>>>>>>>>>>>>> l/org.wso2.carbon.ml.commons/src/main/java/org/wso2/carbon/m
>>>>>>>>>>>>>>> l/commons/domain/Workflow.java#L280
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>> Misgana
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 27.06.2016 08:53, Supun Sethunga wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Misgana,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you commit all the changes to your fork of carbon-ml,
>>>>>>>>>>>>>>> and share us the link. So that I can check whats happening.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also, do I need to manually edit the machinelearner.xml
>>>>>>>>>>>>>>>> file(s) to include Stacking?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes. No need to add hyper-parameters, since we can't
>>>>>>>>>>>>>>> predefine them.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Supun
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 27, 2016 at 4:33 AM, Misgana Negassi <
>>>>>>>>>>>>>>> negassim@tf.uni-freiburg.de> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello Supun,
>>>>>>>>>>>>>>>> sorry to bother you again with the same problem but I am
>>>>>>>>>>>>>>>> not making any progress right now :(
>>>>>>>>>>>>>>>> I can't get the sample script running.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I tried two approaches:
>>>>>>>>>>>>>>>> A) Using the server provided and applying patches.
>>>>>>>>>>>>>>>> Problem: I can not apply my patch. I suspect the problem is
>>>>>>>>>>>>>>>> that my patch is of a newer version. The server's carbon ml is 1.1.1. and
>>>>>>>>>>>>>>>> mine is 1.1.2. Can I somewhere change the version number when I build my
>>>>>>>>>>>>>>>> project?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> B) Building carbon and then product-ml and then unpacking
>>>>>>>>>>>>>>>> modules/distribution/target/wso2ml-2.0.0-SNAPSHOT.zip
>>>>>>>>>>>>>>>> I can start the server with the right components except for
>>>>>>>>>>>>>>>> one exception:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> log4j:ERROR Could not instantiate appender named
>>>>>>>>>>>>>>>> "CARBON_MEMORY".
>>>>>>>>>>>>>>>> log4j:ERROR Could not instantiate class
>>>>>>>>>>>>>>>> [org.wso2.carbon.logging.servi
>>>>>>>>>>>>>>>> ce.appender.CarbonMemoryAppender].
>>>>>>>>>>>>>>>> java.lang.ClassNotFoundException:
>>>>>>>>>>>>>>>> org.wso2.carbon.logging.servic
>>>>>>>>>>>>>>>> e.appender.CarbonMemoryAppender cannot be found by
>>>>>>>>>>>>>>>> org.wso2.carbon.logging_4.4.3
>>>>>>>>>>>>>>>>     at org.eclipse.osgi.internal.load
>>>>>>>>>>>>>>>> er.BundleLoader.findClassInternal(BundleLoader.java:501)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> However, when I try to run my example script there seems to
>>>>>>>>>>>>>>>> be nothing happening on the server side. No exceptions but also the
>>>>>>>>>>>>>>>> debugger does not stop when I expect it would. It seems something is
>>>>>>>>>>>>>>>> crashing but the exceptions are swallowed. Is there a way to increase the
>>>>>>>>>>>>>>>> debug output?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, do I need to manually edit the machinelearner.xml
>>>>>>>>>>>>>>>> file(s) to include Stacking?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 21.06.2016 05:41, Supun Sethunga wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Misgana,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have two issues:
>>>>>>>>>>>>>>>> 1. Issue:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Exception in thread "pool-17-thread-7"
>>>>>>>>>>>>>>>>> java.lang.IllegalArgumentException: No enum constant
>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.commons.con
>>>>>>>>>>>>>>>>> stants.MLConstants.SUPERVISED_ALGORITHM.STACKING
>>>>>>>>>>>>>>>>>     at java.lang.Enum.valueOf(Enum.java:238)
>>>>>>>>>>>>>>>>>     at org.wso2.carbon.ml.commons.con
>>>>>>>>>>>>>>>>> stants.MLConstants$SUPERVISED_
>>>>>>>>>>>>>>>>> ALGORITHM.valueOf(MLConstants.java:225)
>>>>>>>>>>>>>>>>>     at org.wso2.carbon.ml.core.spark.
>>>>>>>>>>>>>>>>> algorithms.SupervisedSparkMode
>>>>>>>>>>>>>>>>> lBuilder.build(SupervisedSparkModelBuilder.java:158)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Which approach did you try? putting the jar as a patch
>>>>>>>>>>>>>>>> (patch9999)? or build the carbon-ml and product ml respectively? If it was
>>>>>>>>>>>>>>>> patching method, did you build org.wso2.carbon.ml.commons
>>>>>>>>>>>>>>>> as well?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Caused by: org.wso2.carbon.ml.database.ex
>>>>>>>>>>>>>>>>> ceptions.DatabaseHandlerException: An error occurred
>>>>>>>>>>>>>>>>> while inserting hyper parameter  to the database: Value too long for column
>>>>>>>>>>>>>>>>> "VALUE VARCHAR(50)": "'rO0ABXNyABNqYXZhLnV0aWwuQXJy
>>>>>>>>>>>>>>>>> YXlMaXN0eIHSHZnHYZ0DAAFJAARzaX
>>>>>>>>>>>>>>>>> pleHAAAAACdwQAAAACc3IAEWphdmEu
>>>>>>>>>>>>>>>>> dXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2F... (372)"; SQL
>>>>>>>>>>>>>>>>> statement:
>>>>>>>>>>>>>>>>> INSERT INTO ML_HYPER_PARAMETER(ANALYSIS_ID,
>>>>>>>>>>>>>>>>> ALGORITHM_NAME, NAME, VALUE, LAST_MODIFIED_TIME) VALUES(?,?,?,?,
>>>>>>>>>>>>>>>>> CURRENT_TIMESTAMP()) [90005-140]
>>>>>>>>>>>>>>>>> The Logic is: The hyperparameters of base-algorithms are
>>>>>>>>>>>>>>>>> serialized and passed in the product-ml hyperparamters script file.This is
>>>>>>>>>>>>>>>>> a serialized string of List of Maps, where each map contains
>>>>>>>>>>>>>>>>> hyperparameters of each base-algorithm and will be deserialized in
>>>>>>>>>>>>>>>>> buildStackingModel() in SupervisedSparkModelBuilder and fed to Stacking
>>>>>>>>>>>>>>>>> train().
>>>>>>>>>>>>>>>>> Current idea to solve it is:
>>>>>>>>>>>>>>>>>  1. Increase column capacity more than VALUE VARCHAR(50)
>>>>>>>>>>>>>>>>>  2. Change serialization logic
>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I believe we don't need to pass a serialized a map and put
>>>>>>>>>>>>>>>> the string in DB. Can we store the hyperparamerets as follows, in the
>>>>>>>>>>>>>>>> ML_HYPER_PARAMETER table?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *ANALYSIS_ID* *ALGORITHM_NAME* *NAME* *VALUE*
>>>>>>>>>>>>>>>> *LAST_MODIFIED_TIME*
>>>>>>>>>>>>>>>> 123 Meta_Algorithm_1 Learning Rate 0.001 <some_timestamp>
>>>>>>>>>>>>>>>> 123 Meta_Algorithm_1 Num_Trees 10 <some_timestamp>
>>>>>>>>>>>>>>>> 123 Meta_Algorithm_1 Max_Depth 0.001 <some_timestamp>
>>>>>>>>>>>>>>>> 123 Meta_Algorithm_1 Seed 4567 <some_timestamp>
>>>>>>>>>>>>>>>> 123 Meta_Algorithm_2 Learning Rate 0.001 <some_timestamp>
>>>>>>>>>>>>>>>> 123 Meta_Algorithm_2 Iterations 100 <some_timestamp>
>>>>>>>>>>>>>>>> 123 Meta_Algorithm_2 Seed 789 <some_timestamp>
>>>>>>>>>>>>>>>> … … …
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ...
>>>>
>>>> [Message clipped]
>>>
>>>
>>>
>>>
>>> --
>>> *Supun Sethunga*
>>> Senior Software Engineer
>>> WSO2, Inc.
>>> http://wso2.com/
>>> lean | enterprise | middleware
>>> Mobile : +94 716546324
>>> Blog: http://supunsetunga.blogspot.com
>>>
>>>
>>>
>>
>>
>> --
>> *Supun Sethunga*
>> Senior Software Engineer
>> WSO2, Inc.
>> http://wso2.com/
>> lean | enterprise | middleware
>> Mobile : +94 716546324
>> Blog: http://supunsetunga.blogspot.com
>>
>
>
>
> --
> *Supun Sethunga*
> Senior Software Engineer
> WSO2, Inc.
> http://wso2.com/
> lean | enterprise | middleware
> Mobile : +94 716546324
> Blog: http://supunsetunga.blogspot.com
>
> _______________________________________________
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 

Thanks & regards,
Nirmal

Team Lead - WSO2 Machine Learner
Associate Technical Lead - Data Technologies Team, WSO2 Inc.
Mobile: +94715779733
Blog: http://nirmalfdo.blogspot.com/

[Attachment #7 (text/html)]

<div dir="ltr">Congratulations Misgana!! Thanks for all the hard work and we look \
forward to work with you in future too.  <div><br></div><div>All the very best \
!</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 22, \
2016 at 9:32 AM, Supun Sethunga <span dir="ltr">&lt;<a href="mailto:supuns@wso2.com" \
target="_blank">supuns@wso2.com</a>&gt;</span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Hi Misgana,<div><br></div><div>I believe you \
have completed all the tasks. Please make sure you submit everything to GSoC on time \
:)</div><div><br></div><div>Regards,</div><div>Supun</div></div><div \
class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 19, 2016 at 9:17 AM, \
Supun Sethunga <span dir="ltr">&lt;<a href="mailto:supuns@wso2.com" \
target="_blank">supuns@wso2.com</a>&gt;</span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>High-level  info should \
be enough...</div><div><br></div><div>Thanks,</div><div>Supun</div></div><div \
class="gmail_extra"><br><div class="gmail_quote"><div><div>On Thu, Aug 18, 2016 at \
5:57 PM, Misgana Negassi <span dir="ltr">&lt;<a \
href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span> \
wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex">  
    
  
  <div text="#000000" bgcolor="#FFFFFF"><div><div>
    <p>Hi Supun,</p>
    <p>I have a question about how technical the documentation should
      be. Should I just explain high-level changes and how it is done,  
      or go deeper and talk about technical details such as which
      functionality are added and/or changed and how?</p>
    <p>Thank you!</p>
    <p>Misgana<br>
    </p>
    <br>
    <div>On 18.08.2016 12:03, Supun Sethunga
      wrote:<br>
    </div>
    </div></div><blockquote type="cite"><div><div>
      <div dir="ltr">Hi Misgana,
        <div><br>
        </div>
        <div>Currently, your documentation [1] says &quot;Performance
          Comparison&quot;. I would suggest to make it the Final
          Documentation for the project (rather than just the perf
          comparison, but including those too). So you will have to add
          a brief introduction to the project, the objectives, the
          solution, and then finally the performance comparison. In that
          doc, you can add the two PR&#39;s as references, for the solution
          section. And you can then send the link of the documentation
          for the final evaluation. Would that work?</div>
        <div><br>
        </div>
        <div>[1]  <a \
href="https://docs.google.com/document/d/1bpTMiSciOqt7vfDgYglSBQucQReDs-btEa5xFjBZkDU/edit" \
target="_blank">https://docs.google.com/do<wbr>cument/d/1bpTMiSciOqt7vfDgYglS<wbr>BQucQReDs-btEa5xFjBZkDU/edit</a></div>
  <div><br>
        </div>
        <div>Regards,</div>
        <div>Supun</div>
      </div>
      </div></div><div class="gmail_extra"><br>
        <div class="gmail_quote"><div><div>On Thu, Aug 18, 2016 at 2:51 PM,
          Misgana Negassi <span dir="ltr">&lt;<a \
href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
          </div></div><blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex">  <div text="#000000" \
bgcolor="#FFFFFF"><div><div>  <p>Dear Supun,</p>
              <p>How would you suggest I send link of my work for final
                evaluation since I contributed to two repositories? Do
                you have any comments on my work before I publish the
                link?</p>
              <p>Thank you,</p>
              <p>Misgana <br>
              </p>
              </div></div><div>
                <div><div><div> <br>
                  <div>On 15.08.2016 18:01, Misgana Negassi wrote:<br>
                  </div>
                  </div></div><blockquote type="cite"><div><div>
                    <p>Hi Supun,</p>
                    <p>I updated the PR as per your comments. And also
                      updated the documentation.</p>
                    <p>I&#39;m grateful for your feedback.<br>
                    </p>
                    <p>Thanks,</p>
                    <p>Misgana<br>
                    </p>
                    <br>
                    <div>On 15.08.2016 05:59, Supun Sethunga wrote:<br>
                    </div>
                    </div></div><blockquote type="cite"><div><div>
                      <div dir="ltr">Hi Misgana,
                        <div><br>
                        </div>
                        <div>Added couple of comments on the PR. Can you
                          please check them and update?</div>
                        <div>Also in the documentation, can you add some
                          brief description of the project, the
                          objectives, and the brief explanation about
                          your solution, apart from the accuracy
                          numbers?</div>
                        <div><br>
                        </div>
                        <div>Once you fix those, please share them both
                          in this thread.</div>
                        <div><br>
                        </div>
                        <div>Thanks,</div>
                        <div>Supun</div>
                      </div>
                      </div></div><div class="gmail_extra"><br>
                        <div class="gmail_quote"><div><div>On Mon, Aug 15, 2016 at
                          9:17 AM, Misgana Negassi <span dir="ltr">&lt;<a \
href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                          </div></div><blockquote class="gmail_quote" style="margin:0 \
                0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                            <div text="#000000" bgcolor="#FFFFFF"><div><div>
                              <div>
                                <div>
                                  <div>Hi Supun,<br>
                                    <br>
                                  </div>
                                  I have done all three tasks you
                                  mentioned. I would be grateful for
                                  your comments.<br>
                                  <br>
                                </div>
                                Best,<br>
                              </div>
                              Misgana<br>
                              <br>
                              <div>On 10.08.2016 12:52, Supun Sethunga
                                wrote:<br>
                              </div>
                              </div></div><blockquote type="cite"><div><div>
                                <div dir="ltr">Hi Misgana,
                                  <div><br>
                                  </div>
                                  <div>As the GSoC Pencil down
                                    deadline  is approaching, shall we
                                    wrap things up by:</div>
                                  <div>
                                    <ul>
                                      <li>Finishing and sharing the
                                        final documentation (including
                                        accuracy comparison  and etc),</li>
                                      <li>Cleaning-up / Formating the
                                        code,  <br>
                                      </li>
                                      <li>Sending git pull request(s) to
                                        wso2/carbon-ml and
                                        wso2/product-ml.</li>
                                    </ul>
                                    <div>Regards,</div>
                                  </div>
                                  <div>Supun</div>
                                </div>
                                </div></div><div class="gmail_extra"><br>
                                  <div class="gmail_quote"><div><div>On Mon, Jul
                                    25, 2016 at 9:24 AM, Supun Sethunga
                                    <span dir="ltr">&lt;<a \
href="mailto:supuns@wso2.com" target="_blank">supuns@wso2.com</a>&gt;</span>  \
                wrote:<br>
                                    </div></div><blockquote class="gmail_quote" \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>  \
<div dir="ltr">Hi Misgana,  <div><br>
                                        </div>
                                        <div>Thats great!</div>
                                        <div><br>
                                        </div>
                                        <div><span>
                                            <blockquote class="gmail_quote" \
style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span \
style="font-size:12.8px">[1]  The data that was passed
                                                to [2] was already
                                                encoded so when passed
                                                to the BasicEncoder, it
                                                was considered as
                                                unknown String and
                                                defaulted to 0th
                                                mapping. So, I decoded
                                                the data before passing
                                                it to getVectors()
                                                method.</span></blockquote>
                                          </span>Does that means, it
                                          first decodes, before passing
                                          into the method, and again
                                          encodes inside that method? If
                                          so, it&#39;s better to pass a flag
                                          (boolean) to that  <span \
style="font-size:12.8px">getVectors()  method along with the data,
                                            to indicate whether to
                                            encode or not. That way you
                                            can avoid unnecessary
                                            decoding and encoding..</span></div>
                                        <div><br>
                                        </div>
                                        <div>Yes, its OK to setup the
                                          demo any day at 10AM IST.
                                          Please invite Nirmal (cc&#39;ed)
                                          too.</div>
                                        <div><br>
                                        </div>
                                        <div>Thanks,</div>
                                        <div>Supun</div>
                                      </div>
                                      </div></div><div class="gmail_extra"><br>
                                        <div class="gmail_quote"><div><div>
                                          <div>
                                            <div>On Sat, Jul 23, 2016 at
                                              4:42 PM, Misgana Negassi <span \
dir="ltr">&lt;<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                                            </div>
                                          </div>
                                          </div></div><blockquote class="gmail_quote" \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">  <div>
                                              <div>
                                                <div text="#000000" \
bgcolor="#FFFFFF"><div><div>  Hallo Supun,<br>
                                                  <br>
                                                  Sorry for the late
                                                  response.<br>
                                                  <br>
                                                  The issue with the
                                                  encoding values is
                                                  resolved. Please see
                                                  [1]. <br>
                                                  <br>
                                                  I am now cleaning up
                                                  code and preparing for
                                                  the presentation. As I
                                                  have mentioned in my
                                                  proposal, I am
                                                  attending a summer
                                                  school next week. If
                                                  the presentation will
                                                  be next week, would be
                                                  it be possible to do
                                                  it early 10AM IST, any
                                                  day of the week except
                                                  Wednesday? If not, the
                                                  other week. Are the
                                                  aforementioned
                                                  appointments ok?<br>
                                                  <br>
                                                  Thank you!<br>
                                                  Misgana<br>
                                                  <br>
                                                  <br>
                                                  [1] The data that was
                                                  passed to [2] was
                                                  already encoded so
                                                  when passed to the
                                                  BasicEncoder, it was
                                                  considered as unknown
                                                  String and defaulted
                                                  to 0th mapping. So, I
                                                  decoded the data
                                                  before passing it to
                                                  getVectors() method.<br>
                                                  <div><br>
                                                  </div>
                                                  <div>[2]  <a \
href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.core/src/main/java/org/wso2/carbon/ml/core/impl/Predictor.java#L313" \
target="_blank">https://github.com/zemoel/<wbr>carbon-ml/blob/master/componen<wbr>ts/m \
l/org.wso2.carbon.ml.core/<wbr>src/main/java/org/wso2/carbon/<wbr>ml/core/impl/Predictor.java#L3<wbr>13</a></div>
  </div></div><div>
                                                    <div><div><div> <br>
                                                      <br>
                                                      <br>
                                                      <br>
                                                      <div>On 20.07.2016
                                                        06:43, Supun
                                                        Sethunga wrote:<br>
                                                      </div>
                                                      </div></div><blockquote \
type="cite"><div><div>  <div dir="ltr">Hi,
                                                          <div><br>
                                                          </div>
                                                          <div>Can you
                                                          remote debug
                                                          and see what
                                                          are the
                                                          encoding
                                                          values you get
                                                          at [1] ?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>[1]  <a \
href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.core/src/main/java/org/wso2/carbon/ml/core/impl/Predictor.java#L313" \
target="_blank">https://github.com/zemoel/<wbr>carbon-ml/blob/master/componen<wbr>ts/m \
l/org.wso2.carbon.ml.core/<wbr>src/main/java/org/wso2/carbon/<wbr>ml/core/impl/Predictor.java#L3<wbr>13</a></div>
  </div>
                                                        </div></div><div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Tue, Jul 19,
                                                          2016 at 9:11
                                                          PM, Misgana
                                                          Negassi <span \
dir="ltr">&lt;<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
                solid;padding-left:1ex">
                                                          <div text="#000000" \
bgcolor="#FFFFFF"><div><div> Hi Supun, <br>  <br>
                                                          I can track
                                                          the weird
                                                          results down
                                                          to the
                                                          encodings.
                                                          Here [1] it
                                                          encodes the
                                                          test dataset
                                                          to all zeros
                                                          excluding the
                                                          first column.<br>
                                                          Do you have
                                                          time in the
                                                          next days for
                                                          a hangout
                                                          session to
                                                          help me on
                                                          this issue?<br>
                                                          <br>
                                                          Thanks!<br>
                                                          Misgana<br>
                                                          <br>
                                                          [1] <a \
href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.core/src/main/java/org/wso2/carbon/ml/core/impl/Predictor.java#L75" \
target="_blank">https://github.com/zemoel/carb<wbr>on-ml/blob/master/components/m<wbr> \
l/org.wso2.carbon.ml.core/src/<wbr>main/java/org/wso2/carbon/ml/c<wbr>ore/impl/Predictor.java#L75</a><br>
  <br>
                                                          <br>
                                                          <div>On
                                                          15.07.2016
                                                          10:34, Misgana
                                                          Negassi wrote:<br>
                                                          </div>
                                                          </div></div><blockquote \
type="cite"><div><div>  Hi Supun,<br>
                                                          <br>
                                                          sorry it was
                                                          not a clear
                                                          question.
                                                          Actually,
                                                          after working
                                                          on it more, I
                                                          found out the
                                                          problem lies
                                                          when we set
                                                          the attributes
                                                          for the
                                                          mlmodel in
                                                          BaseModelBuilder
                                                          eg.
                                                          includedFeatures,
                                                          encodings and
                                                          responseIndex.
                                                          I used to get
                                                          it from the
                                                          workflow which
                                                          is not
                                                          compatible to
                                                          the
                                                          levelOneDataset.
                                                          <br>
                                                          <br>
                                                          Currently I am
                                                          trying to
                                                          understand
                                                          what these are
                                                          and how I can
                                                          populate them
                                                          to be
                                                          compatible to
LevelOneDataset. <br>
                                                          My idea is to
                                                          create helper
                                                          method to
                                                          setFeatures(name,
                                                          ImputeOption,
                                                          ..) and get
                                                          them, and
                                                          create an
                                                          empty map for
                                                          encodings. For
                                                          responseIndex,
                                                          I dont have
                                                          idea yet.<br>
                                                          <br>
                                                          Can you give
                                                          me a hint?<br>
                                                          Thanks,<br>
                                                          Misgana<br>
                                                          <br>
                                                          <div>On
                                                          15.07.2016
                                                          05:57, Supun
                                                          Sethunga
                                                          wrote:<br>
                                                          </div>
                                                          </div></div><blockquote \
                type="cite"><div><div>
                                                          <div dir="ltr">Hi,
                                                          <div><br>
                                                          </div>
                                                          <div>Sorry I
                                                          didn&#39;t get the
                                                          question. can
                                                          you explain it
                                                          a bit?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Thanks,</div>
                                                          <div>Supun  </div>
                                                          </div>
                                                          </div></div><div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Thu, Jul 14,
                                                          2016 at 6:22
                                                          PM, Misgana
                                                          Negassi <span \
dir="ltr">&lt;<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
                solid;padding-left:1ex">
                                                          <div text="#000000" \
bgcolor="#FFFFFF"><div><div> Hi Supun,<br>  <br>
                                                          I have a
                                                          question
                                                          regarding the
                                                          issue with the
                                                          Stacking
                                                          LevelOneDataset.<br>
                                                          I envoke
                                                          sparkContext
                                                          parallelize
                                                          method to get
JavaRDD&lt;LabeledPoint&gt;. But here is what I get:<br>
LevelOneDataset : ParallelCollectionRDD at parallelize . Do we expect
                                                          rather
                                                          MapPartitionRDD
                                                          at
                                                          randomSplit?
                                                          If so, can you
                                                          give me a hint
                                                          how to
                                                          cast/get it?<br>
                                                          <br>
                                                          Best,<br>
                                                          Misgana<br>
                                                          <br>
                                                          <br>
                                                          <div>On
                                                          14.07.2016
                                                          08:58, Supun
                                                          Sethunga
                                                          wrote:<br>
                                                          </div>
                                                          </div></div><blockquote \
                type="cite"><div><div>
                                                          <div dir="ltr">Hi
                                                          Misgana,
                                                          <div><br>
                                                          </div>
                                                          <div>That is
                                                          great! Keep it
                                                          up!</div>
                                                          <div>Yes,
                                                          let&#39;s try to
                                                          solve all the
                                                          issues of the
                                                          existing
                                                          methods first,
                                                          before moving
                                                          on to the
                                                          other one.
                                                          Also do not
                                                          worry about
                                                          the accuracy,
                                                          because it can
                                                          be increased
                                                          by tuning the
hyper-parameters, which is up to the user..  :)</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Once you
                                                          solve the
                                                          current
                                                          issues, can
                                                          you setup a
                                                          small   demo
                                                          with the ML
                                                          team, to show
                                                          how the two
                                                          methods
                                                          work  from end
                                                          to end, to
                                                          some sample
                                                          dataset?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Regards,</div>
                                                          <div>Supun</div>
                                                          </div>
                                                          </div></div><div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Thu, Jul 14,
                                                          2016 at 11:55
                                                          AM, Misgana
                                                          Negassi <span \
dir="ltr">&lt;<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
                solid;padding-left:1ex">
                                                          <div text="#000000" \
bgcolor="#FFFFFF"><div><div> Hi Supun,<br>  <br>
                                                          This is what I
                                                          have done:<br>
                                                          1. Stacking<br>
                                                            Issue with
                                                          getting the
                                                          Hyperparameters
                                                          in a Map of
                                                          Maps   is
                                                          resolved now.
                                                          Basically
                                                          Stacking is
                                                          implemented
                                                          end-to-end. I
                                                          am now
                                                          tackling a
                                                          error which
                                                          arises while
                                                          training the
                                                          Metamodel. My
                                                          hypothesis is
                                                          that the
                                                          problem lies
                                                          while stacking
                                                          the
                                                          predictions to
                                                          form the
                                                          levelOneDataset.<br>
                                                          <br>
                                                          2. Bagging<br>
                                                          Bagging is
                                                          also
                                                          implemented
                                                          end-to-end.
                                                          But the
                                                          accuracy is
                                                          low: 67.5%. I
                                                          am trying to
                                                          figure out
                                                          where the
                                                          problem lies.<br>
                                                          <br>
                                                          I have
                                                          commited my
                                                          latest code.
                                                          Once I am done
                                                          with those
                                                          Issues, I will
                                                          implement
                                                          Boosting.<br>
                                                          <br>
                                                          Best,<br>
                                                          Misgana
                                                          </div></div><div>
                                                          <div><div><div><br>
                                                          <br>
                                                          <br>
                                                          <br>
                                                          <div>On
                                                          14.07.2016
                                                          07:03, Supun
                                                          Sethunga
                                                          wrote:<br>
                                                          </div>
                                                          </div></div><blockquote \
                type="cite"><div><div>
                                                          <div dir="ltr">Hi,
                                                          <div><br>
                                                          </div>
                                                          <div>Can you
                                                          please give us
                                                          a brief update
                                                          on the current
                                                          status?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Thanks,</div>
                                                          <div>Supun</div>
                                                          </div>
                                                          </div></div><div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Wed, Jul 6,
                                                          2016 at 9:41
                                                          AM, Supun
                                                          Sethunga <span \
dir="ltr">&lt;<a href="mailto:supuns@wso2.com" \
target="_blank">supuns@wso2.com</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
                solid;padding-left:1ex"><div><div>
                                                          <div dir="ltr">Hi
                                                          Misgana,
                                                          <div><br>
                                                          </div>
                                                          <div>Can you
                                                          give us a
                                                          brief  update?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Thanks,</div>
                                                          <div>Supun</div>
                                                          </div>
                                                          </div></div><div>
                                                          <div>
                                                          <div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Fri, Jul 1,
                                                          2016 at 12:40
                                                          PM, Supun
                                                          Sethunga <span \
dir="ltr">&lt;<a href="mailto:supuns@wso2.com" \
target="_blank">supuns@wso2.com</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div><div>  <div dir="ltr">
                                                          <div>Hi,</div>
                                                          <span>
                                                          <div>  </div>
                                                          <blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span \
style="font-size:12.8px">Also, the server I am using is the one created  from \
building  the product-ml
                                                          and then
                                                          unpacking
                                                          \
modules/distribution/target/</span><span \
style="font-size:12.8px">ws<wbr>o2ml-2.0.0-SNAPSHOT.zip. The server I  downloaded
                                                          from WSO2
                                                          website and
                                                          patches still
                                                          doesn&#39;t
                                                          recognize my
                                                          \
changes.</span></blockquote>  <div><br>
                                                          </div>
                                                          </span>
                                                          <div>Does your
                                                          changes take
                                                          effect in the
                                                          pack built
                                                          from source?
                                                          if so, please
                                                          proceed with
                                                          it.   </div>
                                                          </div>
                                                          </div></div><div>
                                                          <div>
                                                          <div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Thu, Jun 30,
                                                          2016 at 6:13
                                                          PM, Misgana
                                                          Negassi <span \
dir="ltr">&lt;<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
                solid;padding-left:1ex">
                                                          <div text="#000000" \
bgcolor="#FFFFFF"><div><div> Hi Supun,<br>  <br>
                                                          Thank you for
                                                          pointing out
                                                          the missing
                                                          comma in
                                                          h2.script has
                                                          solved the
                                                          problem while
                                                          creating the
                                                          hyper-parameters
                                                          table. <br>
                                                          <br>
                                                          I have now
                                                          included
                                                          algorithmName
                                                          as second
                                                          hyperparameter
                                                          and trying to
                                                          debug the
                                                          NullPointerException.
                                                          I am currently
                                                          trying to
                                                          retrace my
                                                          steps and see
                                                          of any logical
                                                          flaw in
                                                          creating the <br>
                                                          Map&lt;String,
                                                          Map&lt;String,
                                                          String&gt;&gt;
Hyperparameters.<br>
                                                          <br>
                                                          Also, the
                                                          server I am
                                                          using is the
                                                          one created
                                                          from building
                                                          the product-ml
                                                          and then
                                                          unpacking
                                                          \
modules/distribution/target/ws<wbr>o2ml-2.0.0-SNAPSHOT.zip.  The server I
                                                          downloaded
                                                          from WSO2
                                                          website and
                                                          patches still
                                                          doesn&#39;t
                                                          recognize my
                                                          changes.<br>
                                                          <br>
                                                          Best,<br>
                                                          Misgana
                                                          </div></div><div>
                                                          <div><div><div><br>
                                                          <br>
                                                          <br>
                                                          <br>
                                                          <br>
                                                          <div>On
                                                          30.06.2016
                                                          08:16, Supun
                                                          Sethunga
                                                          wrote:<br>
                                                          </div>
                                                          </div></div><blockquote \
                type="cite"><div><div>
                                                          <div dir="ltr">Hi
                                                          Misgana,
                                                          <div><br>
                                                          </div>
                                                          <div>I tried
                                                          to run you
                                                          script with
                                                          your forks of
                                                          carbon-ml [1]
                                                          and product-ml
                                                          [2], but
                                                          noticed
                                                          several
                                                          issues.</div>
                                                          <div>
                                                          <ol>
                                                          <li>There are
                                                          some issues in
                                                          the
                                                          model-generation.sh.
                                                          (A whitespace
                                                          at [3]. &quot;./jq&quot;
                                                          should be
                                                          &quot;jq&quot;).</li>
                                                          <li>There&#39;s a
                                                          typo in the
                                                          h2.sql script
                                                          [4] (missing
                                                          comma), hence
                                                          the DB table
                                                          is not getting
                                                          created. So
                                                          any step
                                                          beyond
                                                          creating
                                                          hyper-parameters
                                                          aren&#39;t
                                                          working.</li>
                                                          <li>In the SQL
                                                          statement [5],
                                                          you are
                                                          setting only
                                                          one parameter,
                                                          while the
                                                          prepared
                                                          statement
                                                          expect two
                                                          parameters.</li>
                                                          </ol>
                                                          <div>Can you
                                                          fix those
                                                          please?  </div>
                                                          </div>
                                                          <div>Are the
                                                          two repos
                                                          different from
                                                          your working
                                                          copy?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>[1]  <a \
href="https://github.com/zemoel/carbon-ml" \
target="_blank">https://github.com/zemoel/<wbr>carbon-ml</a></div>  <div>[2]  <a \
href="https://github.com/zemoel/product-ml" \
target="_blank">https://github.com/zemoel/<wbr>product-ml</a></div>  <div>[3]  <a \
href="https://github.com/zemoel/product-ml/blob/master/modules/samples/tuned/stacking/model-generation.sh#L60" \
target="_blank">https://github.com/zemoel/<wbr>product-ml/blob/master/modules<wbr>/samples/tuned/stacking/model-<wbr>generation.sh#L60</a></div>
  <div>[4]  <a href="https://github.com/zemoel/carbon-ml/blob/master/features/ml/org.wso2.carbon.ml.database.feature/src/main/resources/dbscripts/h2.sql#L152" \
target="_blank">https://github.com/zemoel/<wbr>carbon-ml/blob/master/features<wbr>/ml/ \
org.wso2.carbon.ml.databas<wbr>e.feature/src/main/resources/d<wbr>bscripts/h2.sql#L152</a></div>
  <div>[5]  <a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org \
.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2183" \
target="_blank">https://github.com/zemoel/<wbr>carbon-ml/blob/master/componen<wbr>ts/m \
l/org.wso2.carbon.ml.datab<wbr>ase/src/main/java/org/wso2/car<wbr>bon/ml/database/internal/MLDat<wbr>abaseService.java#L2183</a></div>
  <div><br>
                                                          </div>
                                                          <div>Regards,</div>
                                                          <div>Supun</div>
                                                          </div>
                                                          </div></div><div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Wed, Jun 29,
                                                          2016 at 6:52
                                                          PM, Misgana
                                                          Negassi <span \
dir="ltr">&lt;<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
                solid;padding-left:1ex">
                                                          <div text="#000000" \
bgcolor="#FFFFFF"><div><div> Hi Supun,<br>  I logged into
                                                          the server
                                                          using UI. As
                                                          you can see
                                                          from the
                                                          screenshot
                                                          below, the
                                                          installed
                                                          features are
                                                          of the
                                                          previous
                                                          version of
                                                          server. Do you
                                                          think, this
                                                          could be the
                                                          problem?<img \
alt="screenshot" src="cid:part21.BB6A07D7.D85B9C3E@tf.uni-freiburg.de" width="586" \
height="144"><br>  <br>
                                                          Best,<br>
                                                          Misgana
                                                          </div></div><div>
                                                          <div><div><div><br>
                                                          <br>
                                                          <div>On
                                                          29.06.2016
                                                          08:06, Supun
                                                          Sethunga
                                                          wrote:<br>
                                                          </div>
                                                          </div></div><blockquote \
                type="cite"><div><div>
                                                          <div dir="ltr">Hi,
                                                          <div><br>
                                                          </div>
                                                          <div>I will
                                                          check and let
                                                          you know</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Regards,</div>
                                                          <div>Supun</div>
                                                          </div>
                                                          </div></div><div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Tue, Jun 28,
                                                          2016 at 5:33
                                                          AM, Misgana
                                                          Negassi <span \
dir="ltr">&lt;<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
                solid;padding-left:1ex">
                                                          <div text="#000000" \
bgcolor="#FFFFFF"><div><div> Hi Supun,<br>  <br>
                                                          These are the
                                                          links for the
                                                          code I
                                                          changed.<br>
                                                          <br>
                                                          <a \
href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/DatabaseService.java#L422" \
target="_blank">https://github.com/zemoel/carb<wbr>on-ml/blob/master/components/m<wbr> \
l/org.wso2.carbon.ml.database/<wbr>src/main/java/org/wso2/carbon/<wbr>ml/database/DatabaseService.ja<wbr>va#L422</a><br>
  <a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carb \
on.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2173" \
target="_blank">https://github.com/zemoel/carb<wbr>on-ml/blob/master/components/m<wbr> \
l/org.wso2.carbon.ml.database/<wbr>src/main/java/org/wso2/carbon/<wbr>ml/database/internal/MLDatabas<wbr>eService.java#L2173</a><br>
  <a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carb \
on.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/MLDatabaseService.java#L2506" \
target="_blank">https://github.com/zemoel/carb<wbr>on-ml/blob/master/components/m<wbr> \
l/org.wso2.carbon.ml.database/<wbr>src/main/java/org/wso2/carbon/<wbr>ml/database/internal/MLDatabas<wbr>eService.java#L2506</a><br>
  <a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carb \
on.ml.commons/src/main/java/org/wso2/carbon/ml/commons/domain/Workflow.java#L267" \
target="_blank">https://github.com/zemoel/carb<wbr>on-ml/blob/master/components/m<wbr> \
l/org.wso2.carbon.ml.commons/s<wbr>rc/main/java/org/wso2/carbon/m<wbr>l/commons/domain/Workflow.java<wbr>#L267</a><br>
  <a href="https://github.com/zemoel/carbon-ml/blob/master/components/ml/org.wso2.carb \
on.ml.commons/src/main/java/org/wso2/carbon/ml/commons/domain/Workflow.java#L280" \
target="_blank">https://github.com/zemoel/carb<wbr>on-ml/blob/master/components/m<wbr> \
l/org.wso2.carbon.ml.commons/s<wbr>rc/main/java/org/wso2/carbon/m<wbr>l/commons/domain/Workflow.java<wbr>#L280</a><br>
  <br>
                                                          Best,<br>
                                                          Misgana
                                                          </div></div><div>
                                                          <div><div><div><br>
                                                          <br>
                                                          <div>On
                                                          27.06.2016
                                                          08:53, Supun
                                                          Sethunga
                                                          wrote:<br>
                                                          </div>
                                                          </div></div><blockquote \
                type="cite"><div><div>
                                                          <div dir="ltr">Hi
                                                          Misgana,
                                                          <div><br>
                                                          </div>
                                                          <div>Can you
                                                          commit all the
                                                          changes to
                                                          your fork of
                                                          carbon-ml, and
                                                          share us the
                                                          link. So that
                                                          I can check
                                                          whats
                                                          happening.</div>
                                                          <div><br>
                                                          </div>
                                                          <blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span \
                style="font-size:12.8px">Also, do I need to manually edit the
                                                          machinelearner.xml
                                                          file(s) to
                                                          include
                                                          \
Stacking?</span></blockquote>  <div>Yes. No
                                                          need to add
                                                          hyper-parameters,
                                                          since we can&#39;t
                                                          predefine
                                                          them.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Thanks,</div>
                                                          <div>Supun  </div>
                                                          </div>
                                                          </div></div><div \
                class="gmail_extra"><br>
                                                          <div \
class="gmail_quote"><div><div>On  Mon, Jun 27,
                                                          2016 at 4:33
                                                          AM, Misgana
                                                          Negassi <span \
dir="ltr">&lt;<a href="mailto:negassim@tf.uni-freiburg.de" \
target="_blank">negassim@tf.uni-freiburg.de</a>&gt;</span>  wrote:<br>
                                                          </div></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
                solid;padding-left:1ex">
                                                          <div text="#000000" \
bgcolor="#FFFFFF"><div><div> Hello Supun,<br>  sorry to
                                                          bother you
                                                          again with the
                                                          same problem
                                                          but I am not
                                                          making any
                                                          progress right
                                                          now :(<br>
                                                          I can&#39;t get
                                                          the sample
                                                          script
                                                          running.<br>
                                                          <br>
                                                          I tried two
                                                          approaches:<br>
                                                          A) Using the
                                                          server
                                                          provided and
                                                          applying
                                                          patches.<br>
                                                          Problem: I can
                                                          not apply my
                                                          patch. I
                                                          suspect the
                                                          problem is
                                                          that my patch
                                                          is of a newer
                                                          version. The
                                                          server&#39;s
                                                          carbon ml is
                                                          1.1.1. and
                                                          mine is 1.1.2.
                                                          Can I
                                                          somewhere
                                                          change the
                                                          version number
                                                          when I build
                                                          my project?<br>
                                                          <br>
                                                          B) Building
                                                          carbon and
                                                          then
                                                          product-ml and
                                                          then unpacking
modules/distribution/target/ws<wbr>o2ml-2.0.0-SNAPSHOT.zip<br>
                                                          I can start
                                                          the server
                                                          with the right
                                                          components
                                                          except for one
                                                          exception:<br>
                                                          <br>
                                                          log4j:ERROR
                                                          Could not
                                                          instantiate
                                                          appender named
&quot;CARBON_MEMORY&quot;.<br>
                                                          log4j:ERROR
                                                          Could not
                                                          instantiate
                                                          class
                                                          \
[org.wso2.carbon.logging.servi<wbr>ce.appender.CarbonMemoryAppend<wbr>er].<br> \
java.lang.ClassNotFoundExcepti<wbr>on: \
org.wso2.carbon.logging.servic<wbr>e.appender.CarbonMemoryAppende<wbr>r  cannot be
                                                          found by
                                                          \
org.wso2.carbon.logging_4.4.3<br>  at
org.eclipse.osgi.internal.load<wbr>er.BundleLoader.findClassInter<wbr>nal(BundleLoader.java:501)<br>
  <br>
                                                          <br>
                                                          However, when
                                                          I try to run
                                                          my example
                                                          script there
                                                          seems to be
                                                          nothing
                                                          happening on
                                                          the server
                                                          side. No
                                                          exceptions but
                                                          also the
                                                          debugger does
                                                          not stop when
                                                          I expect it
                                                          would. It
                                                          seems
                                                          something is
                                                          crashing but
                                                          the exceptions
                                                          are swallowed.
                                                          Is there a way
                                                          to increase
                                                          the debug
                                                          output?<br>
                                                          <br>
                                                          Also, do I
                                                          need to
                                                          manually edit
                                                          the
                                                          machinelearner.xml
                                                          file(s) to
                                                          include
                                                          Stacking?
                                                          </div></div><div>
                                                          <div><div><div><br>
                                                          <br>
                                                          <div>On
                                                          21.06.2016
                                                          05:41, Supun
                                                          Sethunga
                                                          wrote:<br>
                                                          </div>
                                                          </div></div><blockquote \
                type="cite">
                                                          <div dir="ltr"><div><div>Hi
                                                          Misgana,
                                                          <div><br>
                                                          </div>
                                                          <div><span \
style="font-size:12.8px">I  have two
                                                          issues:</span><br \
                style="font-size:12.8px">
                                                          <span \
                style="font-size:12.8px">1.
                                                          Issue:</span><br \
                style="font-size:12.8px">
                                                          <br \
style="font-size:12.8px">  <blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span \
                style="font-size:12.8px">Exception in thread \
                &quot;pool-17-thread-7&quot;
                                                          \
java.lang.IllegalArgumentExcep</span><span style="font-size:12.8px"><wbr>tion: No \
enum constant  org.wso2.carbon.ml.commons.con</span><span \
style="font-size:12.8px"><wbr>stants.MLConstants.SUPERVISED_</span><span \
                style="font-size:12.8px"><wbr>ALGORITHM.STACKING<br>
                                                          </span><span \
style="font-size:12.8px">    at
                                                          \
                java.lang.Enum.valueOf(Enum.ja</span><span \
                style="font-size:12.8px"><wbr>va:238)<br>
                                                          </span><span \
style="font-size:12.8px">    at
                                                          \
org.wso2.carbon.ml.commons.con</span><span \
style="font-size:12.8px"><wbr>stants.MLConstants$SUPERVISED_</span><span \
style="font-size:12.8px"><wbr>ALGORITHM.valueOf(MLConstants.</span><span \
                style="font-size:12.8px"><wbr>java:225)<br>
                                                          </span><span \
style="font-size:12.8px">    at
                                                          \
org.wso2.carbon.ml.core.spark.</span><span \
style="font-size:12.8px"><wbr>algorithms.SupervisedSparkMode</span><span \
style="font-size:12.8px"><wbr>lBuilder.build(SupervisedSpark</span><span \
style="font-size:12.8px"><wbr>ModelBuilder.java:158)</span></blockquote>  </div>
                                                          <div><span \
style="font-size:12.8px">Which  approach did
                                                          you try?
                                                          putting the
                                                          jar as a patch
                                                          (patch9999)?
                                                          or build the
                                                          carbon-ml and
                                                          product ml
                                                          respectively?
                                                          If it was
                                                          patching
                                                          method, did
                                                          you build  <a \
                href="http://org.wso2.carbon.ml" \
                target="_blank">org.wso2.carbon.ml</a>.commo<wbr>ns
                                                          as well?</span></div>
                                                          </div></div><div><div><div>
                                                          <div><br>
                                                          </div>
                                                          <blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span \
style="font-size:12.8px">Caused by: org.wso2.carbon.ml.database.ex</span><span \
style="font-size:12.8px"><wbr>ceptions.DatabaseHandlerExcept</span><span \
style="font-size:12.8px"><wbr>ion: An error occurred while inserting  hyper
                                                          parameter   to
                                                          the database:
                                                          Value too long
                                                          for column
                                                          &quot;VALUE
                                                          VARCHAR(50)&quot;:
&quot;&#39;rO0ABXNyABNqYXZhLnV0aWwuQXJy</span><span \
style="font-size:12.8px"><wbr>YXlMaXN0eIHSHZnHYZ0DAAFJAARzaX</span><span \
style="font-size:12.8px"><wbr>pleHAAAAACdwQAAAACc3IAEWphdmEu</span><span \
style="font-size:12.8px"><wbr>dXRpbC5IYXNoTWFwBQfawcMWYNEDAA</span><span \
                style="font-size:12.8px"><wbr>JGAApsb2F... (372)&quot;; SQL \
                statement:<br>
                                                          </span><span \
style="font-size:12.8px">INSERT  INTO
                                                          \
ML_HYPER_PARAMETER(ANALYSIS_ID</span><span style="font-size:12.8px"><wbr>, \
                ALGORITHM_NAME, NAME, VALUE,
                                                          LAST_MODIFIED_TIME)
VALUES(?,?,?,?, CURRENT_TIMESTAMP()) [90005-140]</span><br style="font-size:12.8px">
                                                          <span \
style="font-size:12.8px">The  Logic is: The
hyperparameters of base-algorithms are serialized and passed in the
                                                          product-ml
                                                          hyperparamters
                                                          script
                                                          file.This is a
                                                          serialized
                                                          string of List
                                                          of Maps, where
                                                          each map
                                                          contains
                                                          hyperparameters
                                                          of each
                                                          base-algorithm
                                                          and will be
                                                          deserialized
                                                          in
                                                          buildStackingModel()
                                                          in
                                                          SupervisedSparkModelBuilder
                                                          and fed to
                                                          Stacking
                                                          train().</span><br \
                style="font-size:12.8px">
                                                          <span \
style="font-size:12.8px">Current  idea to solve
                                                          it is:<br>
                                                          </span><span \
style="font-size:12.8px">  1.  Increase
                                                          column
                                                          capacity more
                                                          than VALUE
                                                          VARCHAR(50)<br>
                                                          </span><span \
style="font-size:12.8px">  2.  Change
                                                          serialization
                                                          logic</span><br \
                style="font-size:12.8px">
                                                          <span \
                style="font-size:12.8px">What
                                                          do you \
think?</span></blockquote>  <div><br>
                                                          </div>
                                                          <div>I believe
                                                          we don&#39;t need
                                                          to pass a
                                                          serialized a
                                                          map and put
                                                          the string in
                                                          DB. Can we
                                                          store the
                                                          hyperparamerets  as
                                                          follows, in
                                                          the
                                                          ML_HYPER_PARAMETER
                                                          table?</div>
                                                          <div><br>
                                                          </div>
                                                          </div></div><div>
                                                          <table border="0" \
                cellspacing="0">
                                                          <colgroup \
                width="118"></colgroup>
                                                          <colgroup \
                width="164"></colgroup>
                                                          <colgroup \
                width="112"></colgroup>
                                                          <colgroup \
                width="85"></colgroup>
                                                          <colgroup \
width="155"></colgroup>  <tbody>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17"><b>ANALYSIS_ID</b></td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center"><b>ALGORITHM_NAME</b></td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center"><b>NAME</b></td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center"><b>VALUE</b></td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center"><b>LAST_MODIFIED_TIME</b></td>  </tr>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17">123</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Meta_Algorithm_1</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">Learning  Rate</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">0.001</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">&lt;some_timestamp&gt;</td>  </tr>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17">123</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Meta_Algorithm_1</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Num_Trees</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">10</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">&lt;some_timestamp&gt;</td>  </tr>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17">123</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Meta_Algorithm_1</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Max_Depth</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">0.001</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">&lt;some_timestamp&gt;</td>  </tr>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17">123</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Meta_Algorithm_1</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Seed</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">4567</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">&lt;some_timestamp&gt;</td>  </tr>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17">123</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Meta_Algorithm_2</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">Learning  Rate</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">0.001</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">&lt;some_timestamp&gt;</td>  </tr>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17">123</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Meta_Algorithm_2</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Iterations</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">100</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">&lt;some_timestamp&gt;</td>  </tr>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17">123</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Meta_Algorithm_2</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">Seed</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">789</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">&lt;some_timestamp&gt;</td>  </tr>
                                                          <tr>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center" height="17">…</td>
                                                          <td style="border:1px solid \
                rgb(0,0,0)" align="center">…</td>
                                                          <td style="border:1px solid \
rgb(0,0,0)" align="center">…</td>  </tr>
                                                          </tbody>
                                                          </table>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                        </div>
                                                      </blockquote>
                                                    </div>
                                                  </div>
                                                </div>
                                              </div>
                                            </div>
                                          </blockquote>
                                        </div>
                                      </div>
                                    </blockquote>
                                  </div>
                                </div>
                              </blockquote>
                            </div>
                          </blockquote>
                        </div>
                      </div>
                    </blockquote>
                  </blockquote>
                </div>
              </div>
            </div>
            ...<br>
            <br>
            [Message clipped]    </blockquote>
        </div><div><div>
        <br>
        <br clear="all"><span><font color="#888888">
        <div><br>
        </div>
        -- <br>
        <div data-smartmail="gmail_signature">
          <div dir="ltr">
            <div>
              <div dir="ltr">
                <div>
                  <div dir="ltr">
                    <div>
                      <div dir="ltr">
                        <div>
                          <div dir="ltr">
                            <div>
                              <div dir="ltr">
                                <div>
                                  <div dir="ltr"><b><font size="1" face="verdana, \
sans-serif">Supun  Sethunga</font></b>
                                    <div><font size="1" face="verdana,
                                        sans-serif">Senior Software
                                        Engineer<br>
                                      </font></div>
                                    <div><font size="1" face="verdana,
                                        sans-serif">WSO2, Inc.</font></div>
                                    <div><font size="1" face="verdana,
                                        sans-serif"><a href="http://wso2.com/" \
target="_blank">http://wso2.com/</a><br>  </font></div>
                                    <div><font size="1" face="verdana,
                                        sans-serif">lean | enterprise |
                                        middleware</font></div>
                                    <div><font size="1" face="verdana,
                                        sans-serif">Mobile : +94
                                        716546324</font></div>
                                    <div><font size="1" face="verdana,
                                        sans-serif">Blog:  <a \
href="http://supunsetunga.blogspot.com" \
target="_blank">http://supunsetunga.blog<wbr>spot.com</a></font></div>  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </font></span></div></div></div>
    </blockquote>
    <br>
  </div>

</blockquote></div><div><div><br><br clear="all"><span class="HOEnZb"><font \
color="#888888"><div><br></div>-- <br><div data-smartmail="gmail_signature"><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><b><font face="verdana, sans-serif" \
size="1">Supun Sethunga</font></b><div><font face="verdana, sans-serif" \
size="1">Senior Software Engineer<br></font></div><div><font face="verdana, \
sans-serif" size="1">WSO2, Inc.</font></div><div><font face="verdana, sans-serif" \
size="1"><a href="http://wso2.com/" \
target="_blank">http://wso2.com/</a><br></font></div><div><font face="verdana, \
sans-serif" size="1">lean | enterprise | middleware</font></div><div><font \
face="verdana, sans-serif" size="1">Mobile : <a href="tel:%2B94%20716546324" \
value="+94716546324" target="_blank">+94 716546324</a></font></div><div><font \
face="verdana, sans-serif" size="1">Blog:  <a href="http://supunsetunga.blogspot.com" \
target="_blank">http://supunsetunga.blog<wbr>spot.com</a></font></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
 </font></span></div></div></div><span class="HOEnZb"><font color="#888888">
</font></span></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br \
clear="all"><div><br></div>-- <br><div data-smartmail="gmail_signature"><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><b><font face="verdana, sans-serif" \
size="1">Supun Sethunga</font></b><div><font face="verdana, sans-serif" \
size="1">Senior Software Engineer<br></font></div><div><font face="verdana, \
sans-serif" size="1">WSO2, Inc.</font></div><div><font face="verdana, sans-serif" \
size="1"><a href="http://wso2.com/" \
target="_blank">http://wso2.com/</a><br></font></div><div><font face="verdana, \
sans-serif" size="1">lean | enterprise | middleware</font></div><div><font \
face="verdana, sans-serif" size="1">Mobile : <a href="tel:%2B94%20716546324" \
value="+94716546324" target="_blank">+94 716546324</a></font></div><div><font \
face="verdana, sans-serif" size="1">Blog:  <a href="http://supunsetunga.blogspot.com" \
target="_blank">http://supunsetunga.<wbr>blogspot.com</a></font></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
 </font></span></div>
<br>______________________________<wbr>_________________<br>
Dev mailing list<br>
<a href="mailto:Dev@wso2.org">Dev@wso2.org</a><br>
<a href="http://wso2.org/cgi-bin/mailman/listinfo/dev" rel="noreferrer" \
target="_blank">http://wso2.org/cgi-bin/<wbr>mailman/listinfo/dev</a><br> \
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div \
class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><br>Thanks &amp; regards,<br>Nirmal<br><br>Team Lead - \
WSO2 Machine Learner<br>Associate Technical Lead - Data Technologies Team, WSO2 \
Inc.<br>Mobile: +94715779733<br>Blog: <a href="http://nirmalfdo.blogspot.com/" \
target="_blank">http://nirmalfdo.blogspot.com/</a><br><br><img \
src="http://c.content.wso2.com/signatures/general.png" width="420" \
height="115"><br></div></div></div></div></div></div> </div>

--089e013d18a8a1bccf053b44106f--


[Attachment #8 (image/png)]

_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev


[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic