[prev in list] [next in list] [prev in thread] [next in thread]
List: sr-dev
Subject: Re: [Serdev] SER footprint
From: Jiri Kuthan <jiri () iptel ! org>
Date: 2008-04-30 20:06:09
Message-ID: 4818D131.5050301 () iptel ! org
[Download RAW message or body]
That's great, why don't you actually post it on the SER webpage? -jiri
William Zhang wrote:
> Finally I got some time to put everything together to produce the
> small footprint SER based on SER 0.9.6 for embedded system. Thanks a
> lot to Andrei for providing very useful suggestion. Here is the list
> of my changes:
>
> 1. Reduce code size
>
> a. Remove the USE_IPV6, USE_MCAST, USE_TCP, DISABLE_NAGLE and NO_DEBUG
> definitions
> b. Remove unix socket server support in the core and modules. Add
> NO_UNIXSOCKET definition in twenty some files in core and other
> modules for conditional compilation.
> c. Remove -g compiler option
> d. Add with -Os option
>
> 2. Reduce data size
>
> a. Decrease the shared Memory Pool from 32MB to 192KB (SHM_MEM_SIZE)
> b. Decrease the Memory Pool from 1MB to 64KB (PKG_MEM_POOL_SIZE)
> c. Decrease socket buffer from 64KB to 4KB (BUF_SIZE)
> d. Reduce the TM hash table from 64K entries to 1K entries (T_TABLE_POWER)
> e. Reduce the MySql buffer size from 64KB to 4KB (SQL_BUF_LEN)
>
> The data memory reduction is quite aggressive so it might not works if
> you have some load and but you can easily increase the constant value
> to meet your requirement. I tested it with SJPhone and Quintum Tenor
> AF PSTN gateway, it works fine.
>
> Below is the result for ser core on x86 target with gcc 4.2.1
> file size(KB) .text(KB)
> .bss(KB) .data(KB) .rodata(KB) shared mem(KB)
> Original 1430.4 490.4 1257.2 4.1
> 98.0 32768
> Modified 350.4 216.2 106.3 3.9
> 66.6 192
>
> I can post the files I modified for removing the unix socket server if
> anybody is interested. Just drop me a line. Also if you have any
> other idea to further reduce the footprint for embedded application,
> please let me know.
>
> Thanks!
> William
>
> On Thu, Apr 24, 2008 at 11:52 PM, William Zhang <espzzh@gmail.com> wrote:
>> Thanks again for the help. It look like I need to look into more
>> detail of the code to understand the design and further reduce the
>> code/data size:) Sure I'll make a list of all my changes and I agree
>> it would be beneficial to other ppl who like to use SER in the
>> embedded system. There are a few changes in the header file const
>> definition, compilation defines and options, and also about 10 files
>> for adding conditional compilation for unixsocket support. I am going
>> to look further into the source code and see any other code can be
>> removed and tune the time out setting as you suggested so that i can
>> relax the memory pool a little bit. I should be able to put everything
>> together sometime next week and keep everybody posted.
>>
>> On Thu, Apr 24, 2008 at 7:01 AM, Andrei Pelinescu-Onciul
>>
>>
>> <andrei@iptel.org> wrote:
>> > On Apr 24, 2008 at 02:33, William Zhang <espzzh@gmail.com> wrote:
>> > > Hi Andrei,
>> > >
>> > > Thank you very much for the information and detail explanation!
>> > >
>> > > The project requirement is to reduce the total code and data memory
>> > > for SER and necessary modules(I listed in my previous message) to 1MB
>> > > or less. I am working on SER 0.9.6 and I removed the the TCP, IPV6 and
>> > > UDP MULTICAST define and manually remove the unixsocket support in the
>> > > core and modules that use it. With -Os and debug strip, I am getting
>> > > 233.7KB for core and 225KB for my modules.
>> >
>> > That's great!
>> >
>> >
>> > > But base on your
>> > > compilation result on mipsel, I probably need to take consider a code
>> > > size increase of 40% when build for MIPS system.
>> >
>> > Hopefully that had a lot to do with the compiler.
>> >
>> >
>> > >
>> > > By removing the unixsocket modules, I saved 128KB for the socket
>> > > buffer. And I slashed the pool memory to 64KB, and reduced the tm hash
>> > > table from 64K entry to 1K entry( 24 bytes each entry so use only 24KB
>> > > in total now) so I can reduced the shared memory to 192KB, basically
>> > > 168KB for any other shared memory usage. (What the tm hash table used
>> > > for? Is 1024 entries too less?)
>> >
>> > The tm hash table is used to store and quickly find a transaction.
>> > A call involves at least 2 transactions: one for the original INVITE
>> > and one for the BYE. So if you are aiming for 1cps that means at most 2
>> > transactions/s. However transactions are kept in memory another 5 s
>> > after the completion (to catch retransmissions) so at a 1 cps constant
>> > rate => you could have 10 transaction in memory in ideal conditions (in
>> > non ideal conditions, like call attempts with nobody answering =>
>> > invite transaction timeout 120s + 5s => maximum 625 transaction in
>> > memory in the same time). So you could go much lower then 1024 (in
>> > theory, in practice we haven't checked if the hash function
>> > distribution is still good with very low hash table sizes). If you
>> > don't care about DOS or corner cases (like having only unanswered
>> > calls) you'll have less then 50 transaction in memory at any time =>
>> > you could even try a hast table with only 1 entry. The worst
>> > thing that could happen is to have very high cpu usage (if this happens
>> > try to increase the hash table).
>> >
>> > I'm a little worried about the pool memory. 64Kb seems a little to less
>> > to me. I don't think there would be a problem if you use a higher
>> > number. The unused part should not be paged-in by the OS so the
>> > effective used memory should not greater if you use a larger pkg_mem
>> > pool size. For example, if I start ser 2.1 with a basic stateless
>> > forwarding config, it uses only 556Kb of memory (RSS in top or ps axl)
>> > although it is configured with 4Mb pool size and started with 2 Mb
>> > shared memory.
>> > So by limiting the POOL size or the shared memory size, you are in fact
>> > limiting the maximum ser memory usage. If ser doesn't need the whole
>> > memory it won't access it and the memory will still be free for other
>> > programs.
>> >
>> >
>> > > This way I end up consume less than
>> > > 1MB code and data size. I can run a few calls successfully. I have
>> > > not run any significant load yet but based on your comment, I
>> > > definitely can not support 1cps and I have to either reduce cps or
>> > > increase the memory. You mentioned 1 cps with default timeouts you
>> > > need ~200k. What does the timeout do with the memory? Is it from the
>> > > memory pool or shared memory?
>> >
>> > It's from the shared memory. The non-shared memory (memory pool) it's
>> > used either on startup/init. or for temporary stuff.
>> > The timeouts influences how much a transaction will be kept in memory.
>> > For example even if a transaction is complete it still has to be kept in
>> > memory for 5s to catch possible delayed retransmissions (according to
>> > the rfc). It's worse if you are trying to call somebody who doesn't
>> > answer. In this case the timeout is 120s by default in ser (IIRC in the
>> > rfc is 180s).
>> > You could save some memory also by changing some of the tm timeouts.
>> > In general if you have a simple setup with low probability of delayed
>> > retransmissions (e.g. local net.) you can decrease the final wait timer to 1s
>> > without problems ( in ser.cfg modparam("tm", "wt_timer" ,1)).
>> >
>> > You could also tune the no-response timeouts (e.g. fr_timer set to 10s
>> > instead of the default 30 and fr_inv_timer to 90s instead of 120s).
>> >
>> >
>> > Could you make a list with all the changes you've made? I think it would
>> > be very usefull for anynone trying to run ser on some embedded system
>> > and we could even add and emebedded option in the makefile that would
>> > automatically build a low memory optimized ser version.
>> >
>> > Andrei
>> >
>> >
>> >
>> > >
>> > > Thanks,
>> > > William
>> > >
>> > > On Wed, Apr 23, 2008 at 5:49 AM, Andrei Pelinescu-Onciul
>> > > <andrei@iptel.org> wrote:
>> > > > On Apr 20, 2008 at 20:04, William Zhang <espzzh@gmail.com> wrote:
>> > > > > Hi All,
>> > > > >
>> > > > > I am a newbie to the SIP Express Router and the project I am currently
>> > > > > working on requires me to port SER to a MIPS embedded system with
>> > > > > limited memory space. So the code size and data memory size are my
>> > > > > primary concerns.
>> > > >
>> > > > ser already works and compiles for MIPS, so you'll need only to optimize
>> > > > it for lower memory usage.
>> > > >
>> > > > Example cross-compile for openwrt (gcc 3.4.4):
>> > > >
>> > > > make ARCH=mips2 CC=${OPENWRT}/staging_dir_mipsel/bin/mipsel-linux-gcc \
>> > > > OS=linux OSREL=2.4.35 CC_EXTRA_OPTS=-Os extra_defs=-DNO_DEBUG
>> > > >
>> > > > After runing strip on it:
>> > > > ls -l ser
>> > > > -rwxr-xr-x 1 andrei andrei 526652 Apr 23 14:10 ser
>> > > >
>> > > > On x86, compiling with similar options, gcc 4.2.3 and stripping, I get:
>> > > > -rwxr-xr-x 1 andrei andrei 379876 Apr 23 14:25 ser
>> > > >
>> > > >
>> > > > (for ser 0.9.7, 2.0 & 2.1 are ~ 1Mb for mipsel/gcc 3.4.4 and ~640k
>> > > > x86/gcc 4.2.3)
>> > > >
>> > > >
>> > > >
>> > > > > Since I am still having some issue with my MIPS cross compilation, I
>> > > > > just worked the SER 0.9.6 on my SUSE PC. With all the default compiler
>> > > > > option but removing the debug info (without the -g option) and i386
>> > > > > release target, the SER
>> > > > > is about 490KB in code size (the .text section in the map file). I
>> > > > > disabled the TCP, IPV6 and UDP MULTICAST support, the code size
>> > > > > decreased to 423KB. The SER website mentions that footprint size:
>> > > > > 300k core, all common modules
>> > > > > (optional) up to 630k (http://www.iptel.org/ser). Am I missing
>> > > > > something here or maybe the website is not up to date with the
>> > > > > footprint?
>> > > >
>> > > > It's not up to date (IIRC this was ser 0.8.10).
>> > > >
>> > > >
>> > > > > I only need the basic SER with authentication and PSNT
>> > > > > gateway support. I loaded these moduels:
>> > > > > mysql.so, sl.so, tm.so, rr.so, maxfwd.so, usrloc.so, registrar.so,
>> > > > > auth.so, auth_db.so, uri.so, uri_db.so, domain.so, avpops.so and
>> > > > > permissions.so. And these modules adds up to 432KB with tm.so as the
>> > > > > biggest one(200KB). Is there any
>> > > > > way to further reduce code size for the core and modules?
>> > > >
>> > > > Compile with -Os (CC_EXTRA_OPTS=-Os to the make command line),
>> > > > extra_defs=-DNO_DEBUG.
>> > > > There are lots of othe different defines you can play with (like turning
>> > > > down support for poll methods you don't use), but I don't remember all
>> > > > of them.
>> > > >
>> > > > >
>> > > > > Of course, this is only the size for PC platform. I am hoping the MIPS
>> > > > > should not differ too much. If anybody compile SER in MIPS, could you
>> > > > > let me know what the code size is? Even for IPAQ compile result, I
>> > > > > also would like to know it.
>> > > >
>> > > > It's bigger (see above) :-(
>> > > > Maybe a newer gcc would help.
>> > > >
>> > > >
>> > > > >
>> > > > > As for the data memory, it seems the SER use the 1MB pool memory as
>> > > > > default for dynamic memory allocation and 32MB shared memory. If I
>> > > > > only need to support 3600 call per hour, about 1 call per second, and
>> > > > > maximum 80 concurrent calls,
>> > > >
>> > > > For 1 cps with default timeouts you need ~200k. If you want to plan for
>> > > > a DOS: 2.5Mb - 5Mb.
>> > > > For maximum 80 concurent calls 1.6Mb.
>> > > > (rounding up transaction usage to 20k, in reality it's <10k on a 32 bit
>> > > > cpu and a simple no-avps config)
>> > > >
>> > > > Note however that there is some memory needed for internal structures
>> > > > (hashes a.s.o) so you can't probably start ser with less then 2Mb shared
>> > > > memory (at least not 2.1).
>> > > > Try starting with 3Mb (ser -m 3) if you use 0.9.7 or 4 Mb if you use 2.0
>> > > > or 2.1.
>> > > >
>> > > > There are also some compile option that could reduce dramatically the
>> > > > memory usage. For example the size of various hashes (tm hash, dns
>> > > > cache, blacklists a..s.o).
>> > > > I always wanted to introduce a make target or option for embedded
>> > > > systems, that would minimize memory usage at the cost of performance,
>> > > > but I never got to doing it :-(
>> > > >
>> > > >
>> > > > > is the default memory allocation too large? How far can I reduce from
>> > > > > the default? Is there way I can monitor the memory usage for an active
>> > > > > call?
>> > > >
>> > > > You can reduce default shared memory usage by starting ser with a
>> > > > different value: ser -m <value_in_MB>.
>> > > > The pool memory can be reduced only by modifying config.h
>> > > > PKG_MEM_POOL_SIZE and re-compiling. You could porbably get away with 512
>> > > > Kb of pkg mem. (or maybe even less). Just watch out the log for out of
>> > > > memory errors.
>> > > >
>> > > > You can't monitor memory usage per call (in fact ser doesn't even have a
>> > > > "call" notion). On 2.0 or 2.1 you could see the current shared memory
>> > > > usage using sercmd: sercmd core.shmmem.
>> > > > For 0.9.x you could try looking at top output for the "RES" value, or
>> > > > you could use a high log level and look at the log output on exit or
>> > > > after kill -SIGUSR1 <pid_of_ser>.
>> > > >
>> > > >
>> > > > Andrei
>> > > >
>> >
>>
> _______________________________________________
> Serdev mailing list
> Serdev@lists.iptel.org
> http://lists.iptel.org/mailman/listinfo/serdev
>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic