[prev in list] [next in list] [prev in thread] [next in thread]
List: helix-server-cvs
Subject: [Server-cvs] engine/inputsource base_shim.cpp, 1.27,
From: jzeng () helixcommunity ! org
Date: 2009-03-31 20:29:36
Message-ID: 200903312131.n2VLVWZM030588 () mailer ! progressive-comp ! com
[Download RAW message or body]
Update of /cvsroot/server/engine/inputsource
In directory cvs01.internal.helixcommunity.org:/tmp/cvs-serv4535/server/engine/inputsource
Modified Files:
base_shim.cpp live_source_wrapper.cpp mdpshim.cpp
ppmstaticshim.cpp
Log Message:
Synopsis
========
fix 231971 Client goes into rebuffering with Repeated switching, and other live fcs issues
Branches: head, server_12_1
Suggested Reviewer: Chytanya, Jamie
Description
===========
This should fix most of the fcs/sspl live issues. Here is the
changes:
To make the gap between audio and video, we may new a few audio packets in front of the video
keyframe. The way to do it is to seek through the rsd queue until you see the video keyframe, and
save all audio packets along the way. In the end we can decide how many audio packets we want to
keep.
1. In bcastmgr:
Current rsd queues don't guarantee you will see another video keyframe if you skip the first
one. The fix is to introduce a third queue: the long queue, in addition to current and future
queue.
When future queue becomes current, the current queue will become the long queue.
2. In live source wrapper:
Before it starts sending packets to base shim, it will get the long rsd queue, and seek through it
to get to the video keyframe. It will save all the audio packets. When it sees a video keyframe, it
will check if the audio packets' duration satifies the gap. If yes, it will stop seeking, and
shrink the audio packets' size to just over the gap.
Even the gap is 0, we still save audio packets, to the point that the first audio packet is earlier
than the video key frame.
3. In base shim:
Now we can get the duration of last packet, please see the comment for more details:
//We switch off this packet, and we can get accurate duration of last packet.
//This solves the problem. We switch off video stream, so the timeline of video stream //is
//accurate now. For audio stream packets, since they always has constant size, so the //old way of
//estimate is sufficient.
It also use the IHXLiveSourceWrapper::GetFirstAudioPacket to get the audio packet, so it can sync on
the audio(As most players do).
4. In clientsession:
I add some code to parse additional headers, and asm rulebook for information needed in live source
wrapper.
5. In ppm:
make fds/sspl always timestamp delivery, since we may have bitrate fluctuation.
Files Affected
==============
server/common/util/header_helper.cpp,v
server/engine/core/bcastmgr.cpp,v
server/engine/core/pub/bcastmgr.h,v
server/engine/dataflow/ppm.cpp,v
server/engine/inputsource/base_shim.cpp,v
server/engine/inputsource/live_source_wrapper.cpp,v
server/engine/inputsource/mdpshim.cpp,v
server/engine/inputsource/ppmstaticshim.cpp,v
server/engine/inputsource/pub/base_shim.h,v
server/engine/inputsource/pub/live_source_wrapper.h,v
server/engine/session/clientsession.cpp,v
server/engine/session/pub/clientsession.h,v
server/include/sink.h,v
server-restricted/broadcast/transport/bcng/recv/strmsess.cpp,v
Testing Performed
=================
Unit Tests:
None.
Integration Tests:
Verify 3gp sr/mr live switching work perfectly.
Verify realmedia sr/mr switching work okay, sometimes a little bit longer of frozen frames, but the
a/v are never out of sync.
Verify on-demand and regular live playback still work.
Leak Tests:
--lct did find a few leaks. The last run looks clean to me.
Performance Tests:
- None
Platforms Tested: linux-rhel5-i686
Build verified: linux-rhel5-i686
Index: ppmstaticshim.cpp
===================================================================
RCS file: /cvsroot/server/engine/inputsource/ppmstaticshim.cpp,v
retrieving revision 1.38
retrieving revision 1.39
diff -u -d -r1.38 -r1.39
--- ppmstaticshim.cpp 2 Mar 2009 18:15:41 -0000 1.38
+++ ppmstaticshim.cpp 31 Mar 2009 20:29:34 -0000 1.39
@@ -176,6 +176,8 @@
(void**)&m_pLiveSourceWrapper)))
{
m_bWirePayload = m_pLiveSourceWrapper->IsPayloadWirePacket();
+
+ m_pLiveSourceWrapper->SetAudioAheadDuration(m_ulAudioAhead);
}
m_pPacketSource->Init(this);
@@ -207,6 +209,7 @@
(pPacket->GetASMFlags() & HX_ASM_SWITCH_OFF);
if (m_bSwitchSource)
{
+ GetLastPacketDiff(pPacket);
m_bSwitchSource = FALSE;
SwitchSourceNow();
return HXR_OK;
Index: live_source_wrapper.cpp
===================================================================
RCS file: /cvsroot/server/engine/inputsource/live_source_wrapper.cpp,v
retrieving revision 1.15
retrieving revision 1.16
diff -u -d -r1.15 -r1.16
--- live_source_wrapper.cpp 11 Mar 2009 16:09:59 -0000 1.15
+++ live_source_wrapper.cpp 31 Mar 2009 20:29:34 -0000 1.16
@@ -46,6 +46,7 @@
#include "netbyte.h"
#include "hxmarsh.h"
#include "header_helper.h"
+#include "hxtypes.h"
CLiveSourceWrapper::CLiveSourceWrapper(IHXPSourceLivePackets* pLiveSource,
@@ -65,6 +66,10 @@
, m_bIsWirePayload(bIsWirePayload)
, m_bGotRTPOffset(FALSE)
, m_packetType(HX_PACKET_UNKNOWN)
+ , m_ulAudioAhead(0)
+ , m_usAudioStream(MAX_UINT16)
+ , m_usVideoStream(MAX_UINT16)
+ , m_bVideoKeyFrameFound(FALSE)
{
pLiveSource->AddRef();
m_pCCF->AddRef();
@@ -95,6 +100,15 @@
{
m_pbSwtichGroupRegistered[i] = TRUE;
m_pulRegisteredStream[i] = pStreamInfoArray[i]->m_stream_number;
+
+ if(pStreamInfoArray[i]->m_bIsAudioStream)
+ {
+ m_usAudioStream = pStreamInfoArray[i]->m_stream_number;
+ }
+ else if(pStreamInfoArray[i]->m_bIsVideoKeyframeStream)
+ {
+ m_usVideoStream = pStreamInfoArray[i]->m_stream_number;
+ }
}
if (m_bIsWirePayload && pStreamInfoArray[i])
@@ -182,7 +196,7 @@
(void**)&pPacketBufferProvider);
if (pPacketBufferProvider)
{
- pPacketBufferProvider->GetPacketBufferQueue(pQueue);
+ pPacketBufferProvider->GetLongPacketBufferQueue(pQueue);
}
HX_RELEASE(pPacketBufferProvider);
@@ -193,6 +207,16 @@
m_bNeedToHandleSyncPackets = TRUE;
m_pRSDQueue->AddPacketBufferQueue(pQueue);
HX_RELEASE(pQueue);
+
+ //only when we have both audio and video stream, we need to sync.
+ if(m_usAudioStream != MAX_UINT16 && m_usVideoStream != MAX_UINT16)
+ {
+ GetAudioAheadPacket();
+ }
+ else
+ {
+ m_bVideoKeyFrameFound = TRUE;
+ }
}
for (UINT16 i = 0; i < m_usNumStreams; i++)
@@ -375,6 +399,7 @@
if (!IsPacketSubscribed(usPktStrmNum, usRuleNum, usRegisterStreamNumber))
{
+
//not subscribed
HX_RELEASE(pPacket);
continue;
@@ -387,10 +412,10 @@
pPacket = pNewPacket;
}
- if (usPktStrmNum != unStreamNumber)
+ if (usRegisterStreamNumber != unStreamNumber)
{
//not the packet asked for, save it and serve it later
- m_ppSavedPacketList[usPktStrmNum]->AddTail(pPacket);
+ m_ppSavedPacketList[usRegisterStreamNumber]->AddTail(pPacket);
//We don't release here, the refcount is taken by the list.
pPacket = NULL;
continue;
@@ -398,7 +423,6 @@
//we get the packet if reach here
}
-
m_pPacketSink->PacketReady(HXR_OK, pPacket);
HX_RELEASE(pPacket);
return HXR_OK;
@@ -465,9 +489,21 @@
}
UINT16 usRegisterStreamNumber = 0;
- UINT16 usRuleNum = pPacket->GetASMRuleNumber();
+ UINT16 usRuleNum = pPacket->GetASMRuleNumber();
if (IsPacketSubscribed(usPktStrmNum, usRuleNum, usRegisterStreamNumber))
+ {
+ if(!m_bVideoKeyFrameFound && usRegisterStreamNumber == m_usVideoStream)
{
+ if(!IsVideoKeyframe(usRegisterStreamNumber, usRuleNum, pPacket->GetASMFlags()))
+ {
+ return;
+ }
+ else
+ {
+ m_bVideoKeyFrameFound = TRUE;
+ }
+ }
+
if (m_pOwedPacket[usRegisterStreamNumber] > 0)
{
m_pOwedPacket[usRegisterStreamNumber]--;
@@ -523,6 +559,159 @@
{
return m_bIsWirePayload;
}
+STDMETHODIMP
+CLiveSourceWrapper::SetAudioAheadDuration(UINT32 ulAudioAhead)
+{
+ m_ulAudioAhead = ulAudioAhead;
+}
+
+STDMETHODIMP_(IHXPacket*)
+CLiveSourceWrapper::GetFirstAudioPacket()
+{
+ if(m_usAudioStream == MAX_UINT16 || m_ppSavedPacketList[m_usAudioStream] == NULL)
+ {
+ return NULL;
+ }
+
+ if(m_ppSavedPacketList[m_usAudioStream]->IsEmpty())
+ {
+ return NULL;
+ }
+ IHXPacket* pPacket = (IHXPacket*)m_ppSavedPacketList[m_usAudioStream]->GetHead();
+ pPacket->AddRef();
+ return pPacket;
+}
+
+void
+CLiveSourceWrapper::GetAudioAheadPacket()
+{
+ IHXPacket* pPacket = NULL;
+ UINT32 ulVKMediaTime = 0;
+ HX_RESULT rc = HXR_OK;
+ while (1)
+ {
+ rc = m_pRSDQueue->GetPacket(pPacket);
+ if (FAILED(rc))
+ {
+ //no packet available at the current time
+ break;
+ }
+
+ UINT16 usRuleNum = pPacket->GetASMRuleNumber();
+ UINT16 usPktStrmNum = pPacket->GetStreamNumber();
+ UINT8 ubASMFlags = pPacket->GetASMFlags();
+ UINT16 usRegisterStreamNumber = 0;
+
+ if (!IsPacketSubscribed(usPktStrmNum, usRuleNum, usRegisterStreamNumber))
+ {
+
+ //not subscribed
+ HX_RELEASE(pPacket);
+ continue;
+ }
+
+ if (usPktStrmNum != usRegisterStreamNumber)
+ {
+ IHXPacket* pNewPacket = CreateNewPacket(pPacket, usRegisterStreamNumber);
+ pPacket->Release();
+ pPacket = pNewPacket;
+ }
+
+ if(m_usAudioStream == usRegisterStreamNumber)
+ {
+ m_ppSavedPacketList[usRegisterStreamNumber]->AddTail(pPacket);
+ }
+ else
+ {
+ if(IsVideoKeyframe(usRegisterStreamNumber, usRuleNum, ubASMFlags))
+ {
+ IHXServerPacketExt* pExt = NULL;
+ pPacket->QueryInterface(IID_IHXServerPacketExt, (void**)&pExt);
+ ulVKMediaTime = pExt->GetMilliSecondTS();
+ pExt->Release();
+
+ if(HasEnoughAudioPacket(ulVKMediaTime))
+ {
+ m_ppSavedPacketList[usRegisterStreamNumber]->AddTail(pPacket);
+ m_bVideoKeyFrameFound = TRUE;
+ break;
+ }
+ }
+ HX_RELEASE(pPacket);
+ }
+ }
+
+ if(m_bVideoKeyFrameFound)
+ {
+ ShrinkAudioPacketCount(ulVKMediaTime);
+ }
+ else
+ {
+ IHXPacket* pTempPacket = NULL;
+ while (m_ppSavedPacketList[m_usAudioStream]->IsEmpty() == FALSE)
+ {
+ pTempPacket = (IHXPacket*)m_ppSavedPacketList[m_usAudioStream]->RemoveHead();
+ pTempPacket->Release();
+ }
+ }
+}
+
+BOOL CLiveSourceWrapper::IsVideoKeyframe(UINT16 usStrm, UINT16 usRule, UINT16 ubFlags)
+{
+ if((ubFlags & HX_ASM_SWITCH_ON) && !(ubFlags & HX_ASM_SIDE_EFFECT))
+ {
+ if(m_usVideoStream == usStrm)
+ {
+ if(m_pStreamInfoArray[usStrm]->m_pbKeyframeRule[usRule])
+ {
+ return TRUE;
+ }
+ }
+ }
+ return FALSE;
+}
+
+void CLiveSourceWrapper::ShrinkAudioPacketCount(UINT32 ulVideoKFTS)
+{
+ IHXPacket* pPacket = NULL;
+ while(HasEnoughAudioPacket(ulVideoKFTS))
+ {
+ HX_RELEASE(pPacket);
+
+ if(m_ppSavedPacketList[m_usAudioStream]->IsEmpty())
+ {
+ HX_ASSERT(FALSE);
+ return;
+ }
+
+ pPacket = (IHXPacket*)m_ppSavedPacketList[m_usAudioStream]->RemoveHead();
+ }
+ m_ppSavedPacketList[m_usAudioStream]->AddHead(pPacket);
+}
+
+BOOL CLiveSourceWrapper::HasEnoughAudioPacket(UINT32 ulVideoKFTS)
+{
+ if(m_ppSavedPacketList[m_usAudioStream]->IsEmpty())
+ {
+ return FALSE;
+ }
+
+ IHXPacket* pPacket = NULL;
+ pPacket = (IHXPacket*)m_ppSavedPacketList[m_usAudioStream]->GetHead();
+
+ IHXServerPacketExt* pExt = NULL;
+ UINT32 ulMediaTime = 0;
+ pPacket->QueryInterface(IID_IHXServerPacketExt, (void**)&pExt);
+ ulMediaTime = pExt->GetMilliSecondTS();
+ HX_RELEASE(pExt);
+
+ if(ulMediaTime > ulVideoKFTS)
+ {
+ return FALSE;
+ }
+
+ return (ulVideoKFTS - ulMediaTime) > m_ulAudioAhead;
+}
STDMETHODIMP_(ULONG32)
CLiveSourceWrapper::AddRef()
Index: mdpshim.cpp
===================================================================
RCS file: /cvsroot/server/engine/inputsource/mdpshim.cpp,v
retrieving revision 1.15
retrieving revision 1.16
diff -u -d -r1.15 -r1.16
--- mdpshim.cpp 2 Mar 2009 18:15:41 -0000 1.15
+++ mdpshim.cpp 31 Mar 2009 20:29:34 -0000 1.16
@@ -222,6 +222,7 @@
(pPacket->GetASMFlags() & HX_ASM_SWITCH_OFF);
if (m_bSwitchSource)
{
+ GetLastPacketDiff(pPacket);
m_bSwitchSource = FALSE;
SwitchSourceNow();
return HXR_OK;
Index: base_shim.cpp
===================================================================
RCS file: /cvsroot/server/engine/inputsource/base_shim.cpp,v
retrieving revision 1.27
retrieving revision 1.28
diff -u -d -r1.27 -r1.28
--- base_shim.cpp 2 Mar 2009 18:15:41 -0000 1.27
+++ base_shim.cpp 31 Mar 2009 20:29:34 -0000 1.28
@@ -66,6 +66,7 @@
#include "streamselector.h"
#include "ratedescmgr.h"
#include "livekeyframe.h"
+#include "header_helper.h"
BasePacketShim* BasePacketShim::CreatePacketShim(BOOL bMDP,
IUnknown* pSource,
@@ -134,6 +135,8 @@
,m_unNumStreamsStarted(0)
,m_unNumStreamsMovedPastStartTS(0)
,m_bDeliveryOffsetSet(FALSE)
+ ,m_usAudioStream(MAX_UINT16)
+ ,m_ulAudioAhead(0)
{
DPRINTF(D_PLMINFO, ("BasePacketShim::BasePacketShim\n"));
if (pCCF)
@@ -180,6 +183,11 @@
continue;
}
+ if(ppStrmInfo[i]->m_bIsAudioStream)
+ {
+ m_usAudioStream = ppStrmInfo[i]->m_stream_number;
+ }
+
m_pbFirstPacketFromSwitchedSource[i] = TRUE;
m_pbStreamHasMovedPastStartTS[i] = FALSE;
m_pulStartingTSPerStream[i] = 0;
@@ -445,10 +453,33 @@
ParsePacket(pPacket, &packetData);
UINT16 unStreamNum = packetData.unStreamNumber;
+ BOOL bFirstPacketRecorded = FALSE;
if(m_bFirstPacketFromSource)
{
- // only the very first packet's data after the switch is recorded
- RecordFirstPacket(&packetData);
+ // only the very first packet's data after the switch is recorded
+ // XXXJJ usually the first packet arriving is a video packet. But for live we
+ // want to sync to the audio packet which is earlier. So here we use GetFirstAudioPacket
+ // and sync to it.
+
+ if(m_pLiveSourceWrapper && m_usAudioStream != MAX_UINT16 && m_usAudioStream != m_ulSwitchStream)
+ {
+ IHXPacket* pFirstAudioPacket = m_pLiveSourceWrapper->GetFirstAudioPacket();
+ if(pFirstAudioPacket)
+ {
+ FCSPacketData firstAudioPacketData;
+ ParsePacket(pFirstAudioPacket, &firstAudioPacketData);
+ RecordFirstPacket(&firstAudioPacketData);
+ HX_RELEASE(pFirstAudioPacket);
+
+ bFirstPacketRecorded = TRUE;
+ }
+ }
+
+ if(!bFirstPacketRecorded)
+ {
+ RecordFirstPacket(&packetData);
+ }
+
m_bFirstPacketFromSource = FALSE;
m_pbFirstPacketFromSwitchedSource[unStreamNum] = FALSE;
m_pulStartingTSPerStream[unStreamNum] = packetData.ulTS;
@@ -599,7 +630,9 @@
//XXXJJ 8/15/08 For server side playlist we always want to start from 0.
- if(m_bIsPlaylist || m_ulRVStream == 0xFFFFFFFF || m_ulDeliveryTimeOffset != 0)
+ //XXXJJ 3/31/09 I finally figure out the cause: real audio superblocking. Since we don't
+ // recommand superblock for fcs, I removed the if here.
+ //if(m_bIsPlaylist || m_ulRVStream == 0xFFFFFFFF || m_ulDeliveryTimeOffset != 0)
{
m_ulStartingTS = pPacketData->ulTS;
}
@@ -707,3 +740,58 @@
m_bDeliveryOffsetSet = TRUE;
m_ulDeliveryTimeOffset = ulOffset;
}
+
+//We switch off this packet, and we can get accurate duration of last packet.
+//This solves the problem. We switch off video stream, so the timeline of video stream
+//is accurate now. For audio stream packets, since they always has constant size, so the
+//old way of estimate is sufficient.
+
+ void
+BasePacketShim::GetLastPacketDiff(IHXPacket* pPacket)
+{
+ FCSStreamData* pStreamInfo = &m_pStreamInfo[pPacket->GetStreamNumber()];
+
+ UINT32 ulAdjustedTS = pPacket->GetTime() - m_ulStartingTS + m_ulDeliveryTimeOffset;
+ pStreamInfo->ulPacketDiff = ulAdjustedTS - pStreamInfo->ulLastPacketTS;
+
+ if(m_bIsRTPPacket)
+ {
+ IHXRTPPacket* pRTPPacket = NULL;
+ pPacket->QueryInterface(IID_IHXRTPPacket, (void**)&pRTPPacket);
+ UINT32 ulAdjustedRTPTS = pRTPPacket->GetRTPTime() - pStreamInfo->ulStartingRTPTS
+ + pStreamInfo->ulRTPTimeOffset;
+ pStreamInfo->ulPacketRTPDiff = ulAdjustedRTPTS - pStreamInfo->ulLastPacketRTPTS;
+ pRTPPacket->Release();
+ }
+ else
+ {
+ pStreamInfo->ulPacketRTPDiff = pStreamInfo->ulPacketDiff;
+ }
+
+ m_ulAudioAhead = 0;
+ if(m_usAudioStream != MAX_UINT16 && m_usAudioStream != m_ulSwitchStream)
+ {
+ FCSStreamData* pVideoStreamInfo = &m_pStreamInfo[m_ulSwitchStream];
+ FCSStreamData* pAudioStreamInfo = &m_pStreamInfo[m_usAudioStream];
+ UINT32 ulVideoEndTS = pVideoStreamInfo->ulLastPacketRTPTS + pVideoStreamInfo->ulPacketRTPDiff;
+ UINT32 ulAudioEndTs = pAudioStreamInfo->ulLastPacketRTPTS + pAudioStreamInfo->ulPacketRTPDiff;
+
+ if(m_pStreamInfo[m_usAudioStream].pTSConverter)
+ {
+ ulAudioEndTs = m_pStreamInfo[m_usAudioStream].pTSConverter->
+ rtp2hxa(ulAudioEndTs);
+ }
+
+ if(m_pStreamInfo[m_ulSwitchStream].pTSConverter)
+ {
+ ulVideoEndTS = m_pStreamInfo[m_ulSwitchStream].pTSConverter->
+ rtp2hxa(ulVideoEndTS);
+ }
+
+ if(ulVideoEndTS > ulAudioEndTs)
+ {
+ m_ulAudioAhead = ulVideoEndTS - ulAudioEndTs;
+ }
+ }
+}
+
_______________________________________________
Server-cvs mailing list
Server-cvs@helixcommunity.org
http://lists.helixcommunity.org/mailman/listinfo/server-cvs
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic