'[Filesystem-dev] CR: Enabling HD Streaming through http filesystem'

[prev in list] [next in list] [prev in thread] [next in thread] 

List: helix-filesystem-dev
Subject: [Filesystem-dev] CR: Enabling HD Streaming through http filesystem
From: <girish.shetty () nokia ! com>
Date: 2011-04-14 0:18:49
Message-ID: 16EDD0D433A1D0498F4A33532517B6B201170E () 008-AM1MPN1-015 ! mgdnok ! nokia ! com
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]


"Nokia submits this code under the terms of a commercial contribution agree=
ment with RealNetworks, and I am authorized to contribute this code under s=
aid agreement."

Modified by: Shubham.Baheti@nokia.com<mailto:girish.shetty@nokia.com>

Reviewed by:

Date: 04/13/2011

Project: SymbianMmf_wm, Symbian_client_apps

Error Id: NA

Synopsis: Enabling HD Streaming through http filesystem


Overview:



Currently, when we stream HD contents through http, we get continuous buffe=
ring after every 2-3 seconds of data.

This is caused by not prerolling/buffering enough data by the filesystem be=
fore giving the data back to the player.

And also, HTTP Buffer Ahead Amount and HTTP Memory Cache Size defined curre=
ntly is not suffeciant enough for reading and buffering enough data from th=
e server.



There is also a bug in the HTTP parser, for the Key Value pair which comes =
as part of the HTTP response.

When the client select HD content, server redirects with a new URI (for the=
 HD content), which contains ',' as part of the URI and current parser uses=
 ',' as the delimiter for the next Value.





Solution:



Increased TCP read buffer size from 1.5K to 64K (which is the MAX size of I=
P Packet)

And also increased MAX BUFFER before Process Idle to 1 MB compare to earlie=
r value of 64 KB



All these changes are protected under SYMBIAN flag, so that it won't break =
the functionality of other platforms.

Also, taken care of a possible memory leak in case of error case in base co=
ntroller.

Files added:
NONE


Files modified:



/cvsroot/clientapps/symbiancommon/config/R1_Mobile_4_0_Factory.cfg

/cvsroot/clientapps/symbianMmf/common/hxmmfbasectrl.cpp

/cvsroot/common/include/hxnet.h

/cvsroot/filesystem/http/httpfsys.cpp

/cvsroot/protocol/http/httppars.cpp


Image size and heap use impact: Negligible

Module release testing (STIF):

Test case(s) added: No

Memory leak check performed: NA

Platforms and profiles build verified: helix-client-s60-52-mmf-mdf-dsp, he=
lix-client-symbian-4

Platforms and profiles functionality verified: armv5

Branch: 210CayS, 420Brizo, HEAD

Diff: <attached>



[Attachment #5 (text/html)]

<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:x="urn:schemas-microsoft-com:office:excel" \
xmlns:p="urn:schemas-microsoft-com:office:powerpoint" \
xmlns:a="urn:schemas-microsoft-com:office:access" \
xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" \
xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" \
xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" \
xmlns:b="urn:schemas-microsoft-com:office:publisher" \
xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" \
xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" \
xmlns:odc="urn:schemas-microsoft-com:office:odc" \
xmlns:oa="urn:schemas-microsoft-com:office:activation" \
xmlns:html="http://www.w3.org/TR/REC-html40" \
xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" \
xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" \
xmlns:Repl="http://schemas.microsoft.com/repl/" \
xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" \
xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" \
xmlns:ppda="http://www.passport.com/NameSpace.xsd" \
xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" \
xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" \
xmlns:ds="http://www.w3.org/2000/09/xmldsig#" \
xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" \
xmlns:udc="http://schemas.microsoft.com/data/udc" \
xmlns:xsd="http://www.w3.org/2001/XMLSchema" \
xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" \
xmlns:ec="http://www.w3.org/2001/04/xmlenc#" \
xmlns:sp="http://schemas.microsoft.com/sharepoint/" \
xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" \
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" \
xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" \
xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" \
xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" \
xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" \
xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" \
xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" \
xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" \
xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" \
xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" \
xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" \
xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" \
xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" \
xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" \
xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="&#1;" \
xmlns="http://www.w3.org/TR/REC-html40"> <head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
&quot;Nokia submits this code under the terms of a commercial contribution \
agreement with RealNetworks, and I am authorized to contribute this code under said \
agreement.&quot;<o:p></o:p> &nbsp;<o:p></o:p> Modified by:&nbsp; \
Shubham.Baheti<a href="mailto:girish.shetty@nokia.com">@nokia.com</a><o:p></o:p> &nbsp;<o:p></o:p> Reviewed by:&nbsp; \
<o:p></o:p> &nbsp;<o:p></o:p> Date:&nbsp; 04/13/2011<o:p></o:p> &nbsp;<o:p></o:p> Project:&nbsp; \
SymbianMmf_wm, Symbian_client_apps<o:p></o:p> &nbsp;<o:p></o:p> Error Id:&nbsp; \
NA<o:p></o:p> &nbsp;<o:p></o:p> Synopsis:&nbsp; Enabling \
HD Streaming through http filesystem<o:p></o:p> &nbsp;<o:p></o:p> <pre>Overview:&nbsp; <o:p></o:p></pre>
<pre><o:p>&nbsp;</o:p></pre>
<pre>Currently, when we stream HD contents through http, we get continuous buffering \
after every 2-3 seconds of data.<o:p></o:p></pre> <pre>This is caused by not \
prerolling/buffering enough data by the filesystem before giving the data back to the \
player.<o:p></o:p></pre> <pre>And also, HTTP Buffer Ahead Amount and HTTP Memory \
Cache Size defined currently is not suffeciant enough for reading and buffering \
enough data from the server.<o:p></o:p></pre> <pre><o:p>&nbsp;</o:p></pre>
<pre>There is also a bug in the HTTP parser, for the Key Value pair which comes as \
part of the HTTP response.<o:p></o:p></pre> <pre>When the client select HD content, \
server redirects with a new URI (for the HD content), which contains &#8216;,&#8217; \
as part of the URI and current parser uses &#8216;,&#8217; as the delimiter for the \
next Value.<o:p></o:p></pre> <pre><o:p>&nbsp;</o:p></pre>
<pre><o:p>&nbsp;</o:p></pre>
<pre>Solution:&nbsp; <o:p></o:p></pre>
<pre><o:p>&nbsp;</o:p></pre>
<pre>Increased TCP read buffer size from 1.5K to 64K (which is the MAX size of IP \
Packet)<o:p></o:p></pre> <pre>And also increased MAX BUFFER before Process Idle to 1 \
MB compare to earlier value of 64 KB<o:p></o:p></pre> <pre><o:p>&nbsp;</o:p></pre>
<pre>All these changes are protected under SYMBIAN flag, so that it won&#8217;t break \
the functionality of other platforms.<o:p></o:p></pre> <o:p>&nbsp;</o:p> Also, taken care of a \
possible memory leak in case of error case in base controller.<o:p></o:p> \
<o:p>&nbsp;</o:p> Files added:&nbsp; \
<o:p></o:p> NONE<o:p></o:p> &nbsp;<o:p></o:p> <pre>Files modified:&nbsp; <o:p></o:p></pre>
<pre><o:p>&nbsp;</o:p></pre>
<pre>/cvsroot/clientapps/symbiancommon/config/R1_Mobile_4_0_Factory.cfg \
<o:p></o:p></pre> <pre>/cvsroot/clientapps/symbianMmf/common/hxmmfbasectrl.cpp \
<o:p></o:p></pre> <pre>/cvsroot/common/include/hxnet.h<o:p></o:p></pre>
<pre>/cvsroot/filesystem/http/httpfsys.cpp<o:p></o:p></pre>
<pre>/cvsroot/protocol/http/httppars.cpp<o:p></o:p></pre>
<pre>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
&nbsp;<o:p></o:p></pre> Image size and heap use \
impact: &nbsp;Negligible<o:p></o:p> &nbsp;<o:p></o:p> Module release testing \
(STIF):&nbsp;<o:p></o:p> &nbsp;<o:p></o:p> Test case(s) \
added:&nbsp; No&nbsp; <o:p></o:p>
&nbsp;<o:p></o:p> Memory leak check \
performed: &nbsp;NA<o:p></o:p> &nbsp;<o:p></o:p> Platforms and profiles \
build verified: &nbsp;helix-client-s60-52-mmf-mdf-dsp, \
helix-client-symbian-4<o:p></o:p> &nbsp;<o:p></o:p> Platforms and profiles \
functionality verified: &nbsp;armv5 <o:p></o:p>
&nbsp;<o:p></o:p> Branch: &nbsp;210CayS, \
420Brizo, HEAD <o:p></o:p>
<o:p>&nbsp;</o:p> Diff: \
&lt;attached&gt;<o:p></o:p> <o:p>&nbsp;</o:p> <o:p>&nbsp;</o:p>
</div>
</body>
</html>


["http_hd_streaming_diff.txt" (text/plain)]

Index: config/R1_Mobile_4_0_Factory.cfg
===================================================================
RCS file: /cvsroot/clientapps/symbiancommon/config/R1_Mobile_4_0_Factory.cfg,v
retrieving revision 1.26.2.54
diff -u -b -r1.26.2.54 R1_Mobile_4_0_Factory.cfg

--- config/R1_Mobile_4_0_Factory.cfg	27 Oct 2010 07:25:36 -0000	1.26.2.54
+++ config/R1_Mobile_4_0_Factory.cfg	12 Apr 2011 20:32:20 -0000
@@ -24,8 +24,8 @@
 HTTPProxySupport=0
 HTTPShoutcastSupportOnly=0
 HTTPReuseResolver=1
-HTTPBufferAheadAmount=32768
-HTTPMemCacheSize=524288
+HTTPBufferAheadAmount=1048576
+HTTPMemCacheSize=10485760
 LocalMediaFolder=c:\nokia\videos\
 LogSink=console
 LoopMode=0
@@ -111,8 +111,8 @@
 
 WCDMAMaximumBandwidth=3000000
 WCDMASustainBandwidth=2100000
-WCDMAMaximumBWPresets={ 64000, 128000 ,192000, 256000, 320000, 384000, 3000000}
-WCDMASustainBWPresets={ 44800,  89600, 134400, 179200, 224000, 268800, 2100000}
+WCDMAMaximumBWPresets={ 64000, 128000 ,192000, 256000, 320000, 2800000, 3000000}
+WCDMASustainBWPresets={ 44800,  89600, 134400, 179200, 224000, 1800000, 2100000}
 
 HSDPAMaximumBandwidth=14000000
 HSDPASustainBandwidth=9800000

Index: common/hxmmfbasectrl.cpp
===================================================================
RCS file: /cvsroot/clientapps/symbianMmf/common/hxmmfbasectrl.cpp,v
retrieving revision 1.1.2.71
diff -u -b -r1.1.2.71 hxmmfbasectrl.cpp
--- common/hxmmfbasectrl.cpp	9 Mar 2011 22:24:15 -0000	1.1.2.71
+++ common/hxmmfbasectrl.cpp	12 Apr 2011 20:33:18 -0000
@@ -2248,10 +2248,10 @@
                 pCookie2Buf->Set((const unsigned char*)"$Version=1\0", 12);
                 pHeaderVal->SetPropertyCString("Cookie2", pCookie2Buf);
                 HXLOGL2(HXLOG_SMMF, "Setting up Request Header with Cookie: %s", (const char*)m_cookie);
+            }
                 HX_RELEASE(pCookieBuf);
                 HX_RELEASE(pCookie2Buf);
             }
-        }
         len = m_referer.GetLength() + 1;
         if(len > 1)
         {

Index: hxnet.h
===================================================================
RCS file: /cvsroot/common/include/hxnet.h,v
retrieving revision 1.29.8.1
diff -u -b -r1.29.8.1 hxnet.h
--- hxnet.h	17 Oct 2007 17:05:53 -0000	1.29.8.1
+++ hxnet.h	12 Apr 2011 20:34:21 -0000
@@ -81,8 +81,12 @@
 #define MAX_IP_PACKET 0x10000
 
 // Default read size for TCP sockets when MSS is unavailable
+#ifdef SYMBIAN
+//TCP/IP can have up to 64K of data, so keep MAX as the read size
+#define DEFAULT_TCP_READ_SIZE 0x10000
+#else
 #define DEFAULT_TCP_READ_SIZE 1500
-
+#endif
 
 // Default read size for all other sockets when MSS is unavailable
 #if defined SYMBIAN

Index: httpfsys.cpp
===================================================================
RCS file: /cvsroot/filesystem/http/httpfsys.cpp,v
retrieving revision 1.88.2.18
diff -u -b -r1.88.2.18 httpfsys.cpp
--- httpfsys.cpp	9 Mar 2011 22:34:28 -0000	1.88.2.18
+++ httpfsys.cpp	14 Apr 2011 00:07:56 -0000
@@ -184,7 +184,11 @@
 #define DEF_CACHE_DB            "helix_http_cache_db"
 #endif // USE_TEMP_CACHE_LOCATION
 
+#ifdef SYMBIAN
+#define HTTP_MAX_BUFFER_BEFORE_PROCESSIDLE (1024 * 1024)
+#else
 #define HTTP_MAX_BUFFER_BEFORE_PROCESSIDLE (64 * 1024)
+#endif
 
 #if !defined(HELIX_CONFIG_NOSTATICS) 
 CCacheEntry*   g_pCacheEntry = NULL;
@@ -200,7 +204,11 @@
 
 // threshhold to see if it's better to do a seek or
 // just wait for old-fashioned reading to catch up
+#ifdef SYMBIAN
+#define BYTE_RANGE_SEEK_THRESHHOLD (1024 * 1024)
+#else
 #define BYTE_RANGE_SEEK_THRESHHOLD (4 * 1024)
+#endif
 
 // #define HELIX_FEATURE_HTTP_MEMCACHE
 
@@ -3139,7 +3147,9 @@
             {
                 // Read directly into the buffer
                 ULONG32 ulActual = 0;
-
+                // Copy the data only if we have some data read
+                if(ulReadCount)
+                {
                 HX_RESULT hxr = m_pChunkyRes->GetData(m_ulCurrentReadPosition,
                                     (char*) pBuffer->GetBuffer(),
                                     ulReadCount,
@@ -3147,6 +3157,7 @@
 
                 HX_ASSERT(ulActual == ulReadCount);
                 _SetCurrentReadPos(m_ulCurrentReadPosition + ulActual);
+                }
 
                 /* Remove from pending list */
                 m_PendingReadList.RemoveHead();

Index: httppars.cpp
===================================================================
RCS file: /cvsroot/protocol/http/httppars.cpp,v
retrieving revision 1.7
diff -u -b -r1.7 httppars.cpp
--- httppars.cpp	2 Aug 2005 18:00:40 -0000	1.7
+++ httppars.cpp	12 Apr 2011 20:36:14 -0000
@@ -262,7 +262,15 @@
     MIMEInputStream input(pValue, strlen(pValue));
     MIMEScanner scanner(input);
     
+#ifdef SYMBIAN
+    /*
+	 * Few tokens such as redirected URL has ',' as part of the Value
+	 * So, using '\r' as the Token Separator instead of ','
+	 */
+    MIMEToken nextTok = scanner.nextToken("\r");
+#else
     MIMEToken nextTok = scanner.nextToken(",");
+#endif
     while(nextTok.hasValue())
     {
 	parseHeaderValue(nextTok.value(), pHeader);


_______________________________________________
Filesystem-dev mailing list
Filesystem-dev@helixcommunity.org
http://lists.helixcommunity.org/mailman/listinfo/filesystem-dev


[prev in list] [next in list] [prev in thread] [next in thread] 
