[prev in list] [next in list] [prev in thread] [next in thread]
List: esb-java-dev
Subject: Re: [Dev] Configuration for SHARD_ID column length in siddhi
From: Fazlan Nazeem <fazlann () wso2 ! com>
Date: 2019-11-23 11:54:35
Message-ID: CAPJf1mdqZbsAxmTq-WqaKR+QXgxsngpiCxwBVRe99ihoT_HkZg () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
One related and good to have feature is to set a single character encoding
by default for the tables when siddhi creates them on the database. The
primary key length issue is mostly due to having collations which take more
bytes to represent a single character. If we can default to a character
encoding of a single byte per character for each database type, then we
could avoid these issues. Just a thought.
On Fri, Nov 22, 2019 at 5:31 PM Rukshan Premathunga <rukshan@wso2.com>
wrote:
> Hi All,
>
> When we configure partitionById in SP, siddhi automatically adds
> a SHARD_ID column to all the aggregated tables in RDBMS. But we are having
> a "too long key issue" in the database. As a solution, we need to properly
> set the column length for each attribute in the aggregated event stream.
>
> Limit of columns in the aggregated streams can be defined when we
> implement the siddhi app. But SHARD_ID is used only when partitionById is
> configured. So we cannot provide and initial column length for that.
>
> So it will be an ideal solution for us to have a configuration for this.
> Otherwise, users need to alter the database or siddhi applications to
> define these values. So can you please check the possibility to support
> this?
>
> ex:
> siddhi:
> properties:
> partitionById: true
> shardId: dc1
> shardId_size: 20 # or derive length from 'shardId'
>
> Thanks and Regards
>
> --
> Rukshan C. Premathunga | Associate Technical Lead | WSO2 Inc.
> (m) +94711822074 | (w) +94112145345 | Email: rukshan@wso2.com
> GET INTEGRATION AGILE
> Integration Agility for Digitally Driven Business
> _______________________________________________
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
--
Thanks & Regards,
*Fazlan Nazeem | *Associate Technical Lead | WSO2 Inc
Mobile : +94772338839 | fazlann@wso2.com
[Attachment #5 (text/html)]
<div dir="ltr">One related and good to have feature is to set a single character \
encoding by default for the tables when siddhi creates them on the database. The \
primary key length issue is mostly due to having collations which take more bytes to \
represent a single character. If we can default to a character encoding of a single \
byte per character for each database type, then we could avoid these issues. Just a \
thought. </div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On \
Fri, Nov 22, 2019 at 5:31 PM Rukshan Premathunga <<a \
href="mailto:rukshan@wso2.com">rukshan@wso2.com</a>> wrote:<br></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi All,<div><br></div><div>When we \
configure partitionById in SP, siddhi automatically adds a SHARD_ID column to all \
the aggregated tables in RDBMS. But we are having a "too long key issue" in \
the database. As a solution, we need to properly set the column length for each \
attribute in the aggregated event stream. </div><div><br></div><div>Limit of \
columns in the aggregated streams can be defined when we implement the siddhi app. \
But SHARD_ID is used only when partitionById is configured. So we cannot provide \
and initial column length for that. </div><div><br></div><div>So it will be an ideal \
solution for us to have a configuration for this. Otherwise, users need to alter the \
database or siddhi applications to define these values. So can you please check the \
possibility to support this?</div><div><br></div><div>ex:</div><div>siddhi:<br> \
properties:<br> partitionById: true<br> shardId: dc1</div><div> \
shardId_size: 20 # or derive length from \
'shardId'</div><div><br></div><div>Thanks and Regards<br \
clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div \
style="color:rgb(136,136,136)"><div dir="ltr">Rukshan C. Premathunga | Associate \
Technical Lead | WSO2 Inc.<br></div></div><div dir="ltr" \
style="color:rgb(136,136,136)"><div>(m) +94711822074 | (w) +94112145345 | Email: <a \
href="mailto:rukshan@wso2.com" style="color:rgb(17,85,204)" \
target="_blank">rukshan@wso2.com</a></div><div><div><font color="#999999">GET \
INTEGRATION AGILE</font></div><div><font color="#999999">Integration Agility for \
Digitally Driven Business</font></div></div></div></div></div></div></div></div></div></div></div></div></div>
_______________________________________________<br>
Dev mailing list<br>
<a href="mailto:Dev@wso2.org" target="_blank">Dev@wso2.org</a><br>
<a href="http://wso2.org/cgi-bin/mailman/listinfo/dev" rel="noreferrer" \
target="_blank">http://wso2.org/cgi-bin/mailman/listinfo/dev</a><br> \
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" \
class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span><font \
color="#888888">Thanks & Regards,</font></span></div><div><span><font \
color="#888888"><br></font></span></div><div dir="ltr"><span><font \
color="#888888"><b>Fazlan Nazeem | </b></font></span><span \
style="color:rgb(136,136,136)">Associate Technical Lead | </span><span \
style="color:rgb(136,136,136)">WSO2 Inc</span></div><div dir="ltr"><span><font \
color="#888888"><div>Mobile : +94772338839 | <a href="mailto:fazlann@wso2.com" \
target="_blank">fazlann@wso2.com</a></div><div><a><img \
src="https://c.content.wso2.com/signatures/wso2-signature-general.png" width="200" \
height="35"><br></a><br></div></font></span></div></div></div></div></div></div></div></div></div></div>
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic