[prev in list] [next in list] [prev in thread] [next in thread]
List: helix-server-cvs
Subject: [Server-cvs] protocol/http http_demux.cpp, 1.59,
From: dcollins () helixcommunity ! org
Date: 2010-10-21 21:46:02
Message-ID: 201010212146.o9LLk5tS001677 () mailer ! progressive-comp ! com
[Download RAW message or body]
Update of /cvsroot/server/protocol/http
In directory cvs01.internal.helixcommunity.org:/tmp/cvs-serv17651
Modified Files:
http_demux.cpp httpbaseproto.cpp httpprot.cpp servhttppost.cpp
Log Message:
Synopsis
========
Large-File Support updates for mp4ff, server mmapmgr and server http
Branches: HEAD
Reviewers: Chytanya
Description
===========
This contains various updates related to Large-File Support in 64-bit
server builds. The primary modules/objects updated are mp4ff,
server memory-mapped I/O, and server HTTP protocol support.
In most cases, 32-bit offsets were changed to HX_OFF_T. This allows
64-bit server builds to serve large (>4GB) MP4/3GP files via RTSP
streaming and HTTP progressive download.
The main exception was the server's HTTP byte-range code which was
updated to use UINT64's for all server builds, 32-bit and 64-bit.
While a 32-bit build should now be able to parse 64-bit Range headers
without generating bogus offsets, it would still be limited by the 32-bit
file offset limitation of the typical 32-bit builds.
The 64-bit MP4 support in the hxclient_2_1_0_cayenne and
hxclient_4_2_0_brizo branches were reviewed as part of this. Aside from
being incomplete for the server's needs, these were not very practical to
merge to the HEAD since these broke binary compatability for 32-bit builds
by changing numerous UINT32's to UINT64's. This is why the approach
used was to leverage the HX_OFF_T type. It will ease the transition and
provide more flexibility for various builds. These branches have also
diverged from the HEAD somewhat and a direct merge was not going to work
very well.
Supporting 64-bit large files with 32-bit builds of the server (or
other products) should be able to leverage most of this, though it is
not specifically part of this project so it will not work with these
changes alone.
Files Affected
==============
common/fileio/fdbufdataf.cpp
common/fileio/mmapmgr.cpp
common/fileio/mmapmgr.h
common/fileio/platform/unix/mmapdatf.cpp
common/fileio/platform/unix/mmapdatf.h
common/include/hxprdnld.h
common/runtime/pub/hlxclib/stdlib.h
datatype/mp4/fileformat/atomizer.cpp
datatype/mp4/fileformat/fswtchr.cpp
datatype/mp4/fileformat/pub/atomizer.h
datatype/mp4/fileformat/pub/fswtchr.h
datatype/mp4/fileformat/pub/hxfswch.h
datatype/mp4/fileformat/pub/qtatmmgs.h
datatype/mp4/fileformat/pub/qtatoms.h
datatype/mp4/fileformat/pub/qtatoms_inline.h
datatype/mp4/fileformat/pub/qtbatom.h
datatype/mp4/fileformat/pub/qtffplin.h
datatype/mp4/fileformat/pub/qtmcache.h
datatype/mp4/fileformat/pub/qttrack.h
datatype/mp4/fileformat/qtatoms.cpp
datatype/mp4/fileformat/qtbatom.cpp
datatype/mp4/fileformat/qtffplin.cpp
datatype/mp4/fileformat/qttrack.cpp
datatype_rn/mp4/fileformat/hinttrack/qthinttrack.cpp
datatype_rn/mp4/fileformat/hinttrack/qtsample.h
filesystem/local/full/smplfsys.cpp
protocol/http/httpmsg.cpp
protocol/http/httppars.cpp
protocol/http/pub/httpmsg.h
server/engine/core/dispatchq.cpp
server/engine/dataflow/http.cpp
server/engine/dataflow/pub/http.h
server/protocol/http/httpbaseproto.cpp
server/protocol/http/http_demux.cpp
server/protocol/http/httpprot.cpp
server/protocol/http/pub/httpprot.h
server/protocol/http/servhttppost.cpp
server-restricted/protocol/http/httpallow.cpp
server-restricted/protocol/http/httprange.cpp
server-restricted/protocol/http/pub/httprange.h
Testing Performed
=================
Unit Tests:
- N/A
Integration Tests:
- Tested with Memory-mapped I/O enabled and disabled, using RTSP/UDP and
HTTP progressive download. Tested with Mac Quicktime, Windows VLC,
and RealPlayer SP players with varying degrees of player functionality.
Also tested byte-ranges retrieval with curl.
Leak Tests:
- N/A
Performance Tests:
- N/A
Platforms Tested: linux-rhel5-x86_64
Builds Verified: linux-rhel5-x86_64, sunos-5.10-sparc64, win-x86_64-vc10
QA Hints
========
- N/A
Index: servhttppost.cpp
===================================================================
RCS file: /cvsroot/server/protocol/http/servhttppost.cpp,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -d -r1.2 -r1.3
--- servhttppost.cpp 12 May 2006 01:48:59 -0000 1.2
+++ servhttppost.cpp 21 Oct 2010 21:46:00 -0000 1.3
@@ -376,7 +376,6 @@
STDMETHODIMP
CHXHTTPPostObject::SetRequest(IHXRequest* pRequest)
{
- char* pTemp = NULL;
const char* pURL;
HX_RELEASE(m_pRequest);
@@ -458,9 +457,6 @@
{
LOG ("Init()");
HX_RESULT theErr = HXR_OK;
- HX_RESULT lResult = HXR_OK;
- char* pTemp = NULL;
- char* pRes = NULL;
if (m_pFileResponse)
{
@@ -536,7 +532,6 @@
HX_RESULT CHXHTTPPostObject::_OpenFile(const char* url, ULONG32 ulFlags)
{
HX_RESULT theErr = HXR_OK;
- HX_RESULT lResult = HXR_OK;
char* pTemp = NULL;
LOG("_OpenFile");
@@ -648,11 +643,6 @@
HX_RESULT theErr = HXR_OK;
HX_RESULT lResult = HXR_OK;
- CHXURL* pHXURL = NULL;
- IHXValues* pURLProperties = NULL;
- IHXBuffer* pHost = NULL;
- IHXBuffer* pPath = NULL;
- IHXBuffer* pCookies = NULL;
IHXNetworkServices* pNetworkServices = 0;
// header labels
@@ -665,13 +655,12 @@
const char szAcceptLang[] = "\r\nAccept-Language: ";
const char* pResource = m_strResource;
- ULONG32 ulPlatformData = 0;
IHXValues* pHeaders = 0;
char* pOutBuffer = 0;
#if defined (_WINDOWS) || defined (_WIN32)
- ulPlatformData = (ULONG32) GetModuleHandle(NULL);
+ ULONG32 ulPlatformData = (ULONG32) GetModuleHandle(NULL);
#endif // defined (_WINDOWS) || defined (_WIN32)
if (!((const char*)m_strHost))
@@ -1475,8 +1464,6 @@
pHeader = pMessage->getNextHeader();
}
- IHXBuffer* pCookie = NULL;
-
if (m_pRequest)
{
IHXValues* pOldResponseHeaders = NULL;
@@ -1692,10 +1679,10 @@
{
// Find the content length to support percent done
// handling...
- UINT32 ulValue = 0;
- if (pMessage->getHeaderValue("content-length", ulValue))
+ UINT64 ullValue = 0;
+ if (pMessage->getHeaderValue("content-length", ullValue))
{
- m_nContentSize = ulValue;
+ m_nContentSize = ullValue;
m_bKnowContentSize = TRUE;
}
Index: http_demux.cpp
===================================================================
RCS file: /cvsroot/server/protocol/http/http_demux.cpp,v
retrieving revision 1.59
retrieving revision 1.60
diff -u -d -r1.59 -r1.60
--- http_demux.cpp 13 Mar 2010 08:31:01 -0000 1.59
+++ http_demux.cpp 21 Oct 2010 21:46:00 -0000 1.60
@@ -368,7 +368,7 @@
pBuf->SetSize(uBufLen);
int msgLen = 0;
pMsg->asBuffer(pBuf->GetBuffer(), msgLen, uBufLen);
- if (msgLen > 0 && msgLen <= uBufLen && m_pSock)
+ if (msgLen > 0 && (UINT32)msgLen <= uBufLen && m_pSock)
{
pBuf->SetSize(msgLen);
hxr = m_pSock->Write(pBuf);
@@ -389,7 +389,7 @@
pBuf->SetSize(uBufLen);
int msgLen = 0;
pMsg->asBuffer(pBuf->GetBuffer(), msgLen, uBufLen);
- if (msgLen > 0 && msgLen <= uBufLen && m_pSock)
+ if (msgLen > 0 && (UINT32)msgLen <= uBufLen && m_pSock)
{
pBuf->SetSize(msgLen);
IHXBuffer** pSendVec = new IHXBuffer*[nVecLen + 1];
Index: httpprot.cpp
===================================================================
RCS file: /cvsroot/server/protocol/http/httpprot.cpp,v
retrieving revision 1.107
retrieving revision 1.108
diff -u -d -r1.107 -r1.108
--- httpprot.cpp 14 Sep 2010 16:57:29 -0000 1.107
+++ httpprot.cpp 21 Oct 2010 21:46:00 -0000 1.108
@@ -141,8 +141,8 @@
, m_tCreationTime(0)
, m_pszMimeType(NULL)
, m_nBytesSent(0)
- , m_nBytesSentPerRange(0)
- , m_ulByteRangeIndex(0)
+ , m_ullBytesSentPerRange(0)
+ , m_ullByteRangeIndex(0)
, m_allowanceStatus(HXR_UNEXPECTED)
, m_ulRegIndex(0)
, m_pURL(NULL)
@@ -335,7 +335,7 @@
}
DPRINTF(0x04000000, ("%6.1lu: 2. HTTPProt for client(%p)\n",
- pClient->m_ulConnId, pClient));
+ (unsigned long)pClient->m_ulConnId, pClient));
HX_RELEASE(pSock);
/*
@@ -417,7 +417,7 @@
m_nFileSize = 0;
m_tCreationTime = 0;
m_nBytesSent = 0;
- m_nBytesSentPerRange = 0;
+ m_ullBytesSentPerRange = 0;
int ret = 0;
m_major_version = (UINT32)pMsg->majorVersion();
@@ -437,7 +437,7 @@
UpdateServerInfoCounters();
char str[64];
- sprintf(str, "%ld.%ld", m_major_version, m_minor_version);
+ sprintf(str, "%lu.%lu", (unsigned long)m_major_version, (unsigned \
long)m_minor_version); IHXBuffer* pVersion = new ServerBuffer(TRUE);
pVersion->Set((UCHAR*)str, strlen(str) + 1);
pClient->GetClientStats()->SetVersion(pVersion);
@@ -673,7 +673,6 @@
IHXSessionStats* pSessionStats = pClient->GetClientStats()->GetSession(1);
IHXSessionStats3* pSessionStats3 = NULL;
IHXClipStats* pClipStats = NULL;
- IHXBuffer* pTemp = NULL;
if (pSessionStats && pSessionStats->QueryInterface(IID_IHXSessionStats3, \
(void**)&pSessionStats3) == HXR_OK) {
@@ -735,7 +734,7 @@
m_bPersistentConnection = FALSE;
}
- DPRINTF(D_INFO, ("HTTP: Socket Timeout: %ld\n", lSocketTimeout));
+ DPRINTF(D_INFO, ("HTTP: Socket Timeout: %ld\n", (long int)lSocketTimeout));
IHXSocket* pSock = m_pDemux->GetSock();
pSock->SetOption(HX_SOCKOPT_APP_IDLETIMEOUT, lSocketTimeout);
HX_RELEASE(pSock);
@@ -755,7 +754,7 @@
pClient->m_pProc->pc->registry->GetInt(MPEG2TS_TIMEOUT_CFG,
&lSocketTimeout, pClient->m_pProc);
- DPRINTF(D_INFO, ("HTTP: MPEG2/TS Socket Timeout: %ld\n", lSocketTimeout));
+ DPRINTF(D_INFO, ("HTTP: MPEG2/TS Socket Timeout: %ld\n", (long \
int)lSocketTimeout)); IHXSocket* pSock = m_pDemux->GetSock();
pSock->SetOption(HX_SOCKOPT_APP_IDLETIMEOUT, lSocketTimeout);
HX_RELEASE(pSock);
@@ -774,7 +773,8 @@
if (!m_pServerInfo)
{
Client* pClient = m_pDemux->GetClient(); //Not AddRef()'d in hxprot.h
- HX_RESULT hresult = \
pClient->m_pProc->pc->server_context->QueryInterface(IID_IHXServerInfo, \
(void**)&m_pServerInfo); + \
pClient->m_pProc->pc->server_context->QueryInterface(IID_IHXServerInfo, + \
(void**)&m_pServerInfo); }
if (m_pServerInfo)
@@ -1272,10 +1272,10 @@
return PrepareAndSendResponse(HXR_PE_INVALID_RANGE);
}
- if (m_pRange->IsByteRangeRequest() && \
m_pRange->ByteRangeStart(m_ulByteRangeIndex) > 0 && m_pHTTP) + if \
(m_pRange->IsByteRangeRequest() && m_pRange->ByteRangeStart(m_ullByteRangeIndex) > 0 \
&& m_pHTTP) {
//Initiate a seek on HTTP and return;
- m_pHTTP->http_start_seek(m_pRange->ByteRangeStart(m_ulByteRangeIndex));
+ m_pHTTP->http_start_seek(m_pRange->ByteRangeStart(m_ullByteRangeIndex));
return TRUE;
}
}
@@ -1467,7 +1467,7 @@
#ifdef HELIX_FEATURE_SERVER_HTTP11
if (m_pRange->IsByteRangeRequest() && nStatus == 206)
{
- UINT32 ulContentLen = 0;
+ UINT64 ulContentLen = 0;
ulContentLen = m_pRange->BytesToSendTotal();
m_pRange->AddByteRangeHeader(pMsg, m_nFileSize);
@@ -1475,11 +1475,11 @@
{
UINT32 ulMimeSize = m_pRange->MultipartMimeHeaderSize() +
strlen(m_pRange->ByteRangeBoundary()) + 6;
- sprintf(lenStr, "%ld", ulContentLen + ulMimeSize);
+ sprintf(lenStr, "%llu", (long long unsigned)(ulContentLen + \
ulMimeSize)); }
else
{
- sprintf(lenStr, "%ld", ulContentLen);
+ sprintf(lenStr, "%llu", (long long unsigned)ulContentLen);
}
pMsg->addHeader("Content-Length", lenStr);
}
@@ -1490,7 +1490,7 @@
else
#endif
{
- sprintf(lenStr, "%ld", nSize);
+ sprintf(lenStr, "%llu", (long long unsigned)nSize);
pMsg->addHeader("Content-Length", lenStr);
}
@@ -1498,7 +1498,8 @@
{
char str[512];
sprintf(str, "%s.FileSize", m_pRegistryKey);
- pClient->m_pProc->pc->registry->AddInt(str, nSize, pClient->m_pProc);
+ INT32 ulSize32 = (UINT32)nSize; //XXX not 64-bit safe
+ pClient->m_pProc->pc->registry->AddInt(str, ulSize32, pClient->m_pProc);
}
pSessionStats->SetFileSize(nSize);
@@ -1638,17 +1639,17 @@
bIsByteRange = m_pRange->IsByteRangeRequest();
if (bIsByteRange)
{
- if (m_pRange->BytesToSend(m_ulByteRangeIndex) > m_nBytesSentPerRange + \
uDataLen) + if (m_pRange->BytesToSend(m_ullByteRangeIndex) > \
m_ullBytesSentPerRange + uDataLen) {
m_nBytesSent += uDataLen;
- m_nBytesSentPerRange += uDataLen;
+ m_ullBytesSentPerRange += uDataLen;
}
- else if (m_pRange->BytesToSend(m_ulByteRangeIndex) > \
m_nBytesSentPerRange &&
- m_ulByteRangeIndex < m_pRange->ByteRangeCount())
+ else if (m_pRange->BytesToSend(m_ullByteRangeIndex) > \
m_ullBytesSentPerRange && + m_ullByteRangeIndex < \
m_pRange->ByteRangeCount()) {
- nSendLength = m_pRange->BytesToSend(m_ulByteRangeIndex) - \
m_nBytesSentPerRange; + nSendLength = \
m_pRange->BytesToSend(m_ullByteRangeIndex) - m_ullBytesSentPerRange; m_nBytesSent += \
nSendLength;
- m_nBytesSentPerRange += nSendLength;
+ m_ullBytesSentPerRange += nSendLength;
bIsRangeDone = 1;
}
else
@@ -1676,7 +1677,7 @@
if (m_pRange->IsByteRangeRequest() &&
m_pRange->ByteRangeCount() > 1)
{
- m_pRange->WrapData(m_ulByteRangeIndex, pData, uDataLen,
+ m_pRange->WrapData(m_ullByteRangeIndex, pData, uDataLen,
nSendLength, pBuf);
}
else if (m_bUseChunkedEncoding)
@@ -1717,17 +1718,17 @@
#ifdef HELIX_FEATURE_SERVER_HTTP11
if (bIsByteRange && bIsRangeDone)
{
- ++m_ulByteRangeIndex;
- if (m_ulByteRangeIndex == m_pRange->ByteRangeCount())
+ ++m_ullByteRangeIndex;
+ if (m_ullByteRangeIndex == m_pRange->ByteRangeCount())
{
bAllDone = TRUE;
}
else
{
- m_nBytesSentPerRange = 0;
- if (m_ulByteRangeIndex < m_pRange->ByteRangeCount())
+ m_ullBytesSentPerRange = 0;
+ if (m_ullByteRangeIndex < m_pRange->ByteRangeCount())
{
- m_pHTTP->http_start_seek(m_pRange->ByteRangeStart(m_ulByteRangeIndex));
+ m_pHTTP->http_start_seek(m_pRange->ByteRangeStart(m_ullByteRangeIndex));
}
}
}
@@ -1934,7 +1935,7 @@
"</HTML>\n";
sprintf(errMsgBody, szBody, errTxt, errTxt);
- sprintf(szContentLength, "%d", strlen(errMsgBody));
+ sprintf(szContentLength, "%d", (int)strlen(errMsgBody));
pMsg->addHeader("Content-Type", "text/html");
pMsg->addHeader("Content-Length", szContentLength);
@@ -1956,7 +1957,7 @@
if (m_pDemux)
{
DPRINTF(D_INFO, ("%lu: failed to send HTTP response message\n",
- m_pDemux->GetClient()->m_ulConnId));
+ (unsigned long)m_pDemux->GetClient()->m_ulConnId));
}
}
else
@@ -2100,7 +2101,7 @@
if (pClient)
{
DPRINTF(D_INFO, ("%lu: failed to send HTTP message\n",
- pClient->m_ulConnId));
+ (unsigned long)pClient->m_ulConnId));
}
}
#endif
@@ -2130,7 +2131,6 @@
m_pRequest->SetURL(szFullURL);
}
- IHXValues* pRequestHeaders = NULL;
if (!m_pRequestHeaders)
{
IHXValues* pRequestHeaders = new CHXHeader();
@@ -2306,7 +2306,7 @@
else if (m_posting)
{
m_pRequestHeaders->SetPropertyULONG32("PostDataLength",
- (ULONG32)m_content_len);
+ (ULONG32)m_content_len); //XXX not 64-bit safe
if (m_content_len > m_max_post_data)
{
m_need_posthandler = TRUE;
@@ -2537,7 +2537,7 @@
pMsg->asString(pBuf+2, nMsgLen, 4093);
char lenStr[30];
- sprintf(lenStr, "%ld", nMsgLen);
+ sprintf(lenStr, "%ld", (long int)nMsgLen);
pResp->addHeader("Content-Length", lenStr);
pResp->addHeader("Content-Type", "message/http");
pResp->setContent((BYTE*)pBuf, nMsgLen);
@@ -2762,7 +2762,7 @@
status = HXR_OK;
}
- if (m_ulByteRangeIndex == 0) //first byte-range
+ if (m_ullByteRangeIndex == 0) //first byte-range
{
PrepareAndSendResponse(status);
}
@@ -2953,7 +2953,7 @@
if (pQP)
{
INT32 pos = pQP - m_pURL;
- if (pos + 1 != strlen(m_pURL)) //only if there are QP after ?
+ if (pos + 1 != (INT32)strlen(m_pURL)) //only if there are QP after ?
{
m_pM3U8Hdlr = new M3U8Handler(m_pDemux->GetClient()->m_pProc,
m_pURL, pos, m_pRequestHeaders);
@@ -2976,7 +2976,7 @@
HTTPProtocol::ByteRangeReset(void)
{
#ifdef HELIX_FEATURE_SERVER_HTTP11
- m_ulByteRangeIndex = 0;
+ m_ullByteRangeIndex = 0;
m_pRange->ResetByteRangeSettings();
#endif
}
Index: httpbaseproto.cpp
===================================================================
RCS file: /cvsroot/server/protocol/http/httpbaseproto.cpp,v
retrieving revision 1.14
retrieving revision 1.15
diff -u -d -r1.14 -r1.15
--- httpbaseproto.cpp 17 Mar 2010 01:15:41 -0000 1.14
+++ httpbaseproto.cpp 21 Oct 2010 21:46:00 -0000 1.15
@@ -288,7 +288,7 @@
HX_ASSERT(pClipStats3);
IHXBuffer* pBuffer = new ServerBuffer(TRUE);
- pBuffer->Set((UCHAR*)szURL, strlen(szURL)+1);
+ pBuffer->Set((UCHAR*)szURL, (UINT32)strlen(szURL)+1);
char str[512];
@@ -431,7 +431,7 @@
// If no/unknown method, don't put in anything.
if (pValue)
{
- pValue->Set((UCHAR*)method, strlen(method) + 1);
+ pValue->Set((UCHAR*)method, (UINT32)strlen(method) + 1);
if (pClient->UseRegistryForStats() && pName)
{
@@ -513,7 +513,7 @@
pBuf[pEnd-pStart] = '\0';
pHost = new ServerBuffer(TRUE);
- pHost->Set(pBuf, pEnd - pStart + 1);
+ pHost->Set(pBuf, (UINT32)(pEnd - pStart + 1));
if (pClient->UseRegistryForStats())
{
@@ -557,15 +557,15 @@
if (pClient->UseRegistryForStats())
{
pBuffer = new ServerBuffer(TRUE);
- sprintf(pBuf, "%lu", pClient->GetRegistryConnId());
- pBuffer->Set((UCHAR*)pBuf, strlen(pBuf) + 1);
+ sprintf(pBuf, "%lu", (unsigned long)pClient->GetRegistryConnId());
+ pBuffer->Set((UCHAR*)pBuf, (UINT32)strlen(pBuf) + 1);
pRequestHeaders->SetPropertyCString("ConnID", pBuffer);
HX_RELEASE(pBuffer);
}
pBuffer = new ServerBuffer(TRUE);
- sprintf(pBuf, "%lu", pClient->GetClientStatsObjId());
- pBuffer->Set((UCHAR*)pBuf, strlen(pBuf) + 1);
+ sprintf(pBuf, "%lu", (unsigned long)pClient->GetClientStatsObjId());
+ pBuffer->Set((UCHAR*)pBuf, (UINT32)strlen(pBuf) + 1);
pRequestHeaders->SetPropertyCString("ClientStatsObjId", pBuffer);
HX_RELEASE(pBuffer);
_______________________________________________
Server-cvs mailing list
Server-cvs@helixcommunity.org
http://lists.helixcommunity.org/mailman/listinfo/server-cvs
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic