[prev in list] [next in list] [prev in thread] [next in thread]
List: helix-datatype-cvs
Subject: [Datatype-cvs] mpg/payload mpvespyld.cpp,1.1,1.1.2.1
From: yuxinliu () helixcommunity ! org
Date: 2012-02-24 7:35:03
[Download RAW message or body]
Update of /cvsroot/datatype/mpg/payload
In directory cvs01.internal.helixcommunity.org:/tmp/cvs-serv28356
Modified Files:
Tag: PRODUCER_14_0_RN
mpvespyld.cpp
Log Message:
Synopsis
========
This CR fix one error that producer can’t deal with the TS file which one PES packet \
contains multi frames , and porting some fix from SERVER_14_2_GLOBECOMM_LR. Producer \
still prompt SA757 but it can encode correctly now.
Branch : PRODUCER_14_0_RN
Reviewed by : Chytanya
Description:
==========
When use VLC to deliver MPEG-TS to producer, the producer would prompt "Error: Video \
Frame rate is low (SA757)" after a while and the stream which is broadcasted to HMS \
can't be played at all.
The root cause is:
Producer never deal with the following condition:
1)One PES packet contains one PTS but contains multi frames.
2)One PES packet without PTS and maybe contains multi frames too.
My fix:
1 when mpeg2 video decoder receives a PES packet. It will check number of B frame \
between reference(I or P) frames, and record the number of B frames. All PES packet \
that used by checking number of B frams will be removed from m_InputQueue and then \
inserted into a cache queue m_CacheQueue. For example, the input is “I B B P B B P \
BB”, the number of B frames should be two. And the packet “I B B P B B P” will be \
inserted into m_CacheQueue. 2 After mpeg2 video decoder got the number of B frames, \
it will get packet from cache queue first, if there is no packet in cache, it will \
get packet from m_InputQueue. 3 For every packet:
1) If the packet contains multi frames, we will separate these frame according to \
PICTURE_START_CODE, and calculate PTS for each frame according to frame type, PTS of \
previous I/P frame and number of B frames. 2) If the packet contains one frame, we \
will set PTS for the frame according to its PTS of PES packet.
Porting fix of SERVER_14_2_GLOBECOMM_LR:
1. Adaptation flags will only be present in the TS header when adaptation field \
length is non-zero. There are a couple of places in CTSDemuxer where we parse the \
byte next to adaptation field length byte without checking if flags are present or \
not. As a result, discontinuity flag is getting tuned on incorrectly and we are \
skipping the current frame's data and creating a lost packet.
Fixed this by first checking the adaptation field length and only parsing the flags \
if the field length calculated is greater than 1 (1 byte is for the adaptation field \
length byte itself).
Files affected:
=========
datatype/mpg/common/mpgvidcommon.cpp
datatype/mpg/common/pub/mpgvidcommon.h
datatype/mpeg2ts/demuxer/tsdemuxer.cpp
datatype/mpg/payload/Umakefil
datatype/mpg/payload/mpvespyld.cpp
datatype/mpg/payload/pub/mpvespyld.h
datatype/mp4/video/renderer/dllumakefil
Testing Performed:
================
Unit Tests:
1 For error: PES packet contains multi frames, every PES packet all contains PTS. \
Using my test tool to send bad_outputDump_27.ps, and run the following command: \
producer.exe –j producer_test.rpjf
Expect: get a mp4 file that can play correctly by VLC.
Result: yes.
Taking use of producermemoryutilization.sh to test memory leak: After producer \
encoded all of bad_outputDump_27.ps(about 5 minutes), I never find memory leak.
2 For error: PES packet never contains PTS and maybe contains multi frames too. Using \
VLC to send bad_outputDump_27.ps, and run the following command: producer.exe –j \
producer_test.rpjf
Expect: get a mp4 file that can played correctly by VLC.
Result: yes.
Taking use of producermemoryutilization.sh to test memory leak: After producer \
encoded all of bad_outputDump_27.ps, I never find memory leak
3 Using VLC to send good_outputDump_37.ps, and run the following command:
producer.exe –j producer_test.rpjf
Expect: get a mp4 file that can played correctly by VLC.
Result: yes.
Taking use of producermemoryutilization.sh to test memory leak: After producer \
encoded all of bad_outputDump_27.ps(about 5 minutes), I never find memory leak.
4 Using my test tool to send good_outputDump_37.ps, and run the following command:
producer.exe –j producer_test.rpjf
Expect: get a mp4 file that can played correctly by VLC.
Result: yes
5 Testing broadcast. Using VLC to send bad_outputDump_27.ps to producer, producer \
convert the TS input into MP4 live stream and send the live stream to helix server, \
and then use VLC to play the live feed. 1) producer.exe –j producer_broadcast.rpjf
2) run command: rtsp://IP:port/broadcast/live.mp4 in VLC
Expect: VLC play live.mp4 correctly.
Result: yes.
6 Writing hard code to set frame rate to 0. Using VLC to send bad_outputDump_27.ps to \
producer to check whether producer will stop encode.
Expect: producer will print “Because we can't get frame rate, so we can't calculate \
PTS. Please exit Producer!” in cmd windows. And waiting user to stop \
it.
Result: yes
Unit Tests: None
Leak Tests: None
Performance Tests: N/A
Platforms Tested: win32-i386-vc9
Builds Verified: win32-i386-vc9
QA Hints
========
None
Index: mpvespyld.cpp
===================================================================
RCS file: /cvsroot/datatype/mpg/payload/mpvespyld.cpp,v
retrieving revision 1.1
retrieving revision 1.1.2.1
diff -u -d -r1.1 -r1.1.2.1
--- mpvespyld.cpp 4 Apr 2011 06:29:58 -0000 1.1
+++ mpvespyld.cpp 24 Feb 2012 07:35:00 -0000 1.1.2.1
@@ -70,6 +70,9 @@
#include "hxalloc.h"
#include "mpvespyld.h"
+#include "hxtlogutil.h"
+#include "mpgvidcommon.h"
+#include "hxbuffer.h"
/* Defines
*/
@@ -100,7 +103,12 @@
, m_ulSamplesPerSecond(1000)
, m_ulRTPSamplesPerSecond(0)
{
- ;
+ m_ulPreTimeStamp = 0;
+ m_bFindFirstPTS = FALSE;
+ m_lGapSize = -1;
+ m_ulBFrameNumber = 0;
+ m_bFindFirstRefFrame = FALSE;
+ m_ulFrameRate = 0;
}
MPEGESPayloadFormat::~MPEGESPayloadFormat()
@@ -228,10 +236,10 @@
}
void MPEGESPayloadFormat::SetAllocator(CHXBufferMemoryAllocator* pAllocator)
-{
+{
HX_RELEASE(m_pAllocator);
m_pAllocator = pAllocator;
- HX_ADDREF(m_pAllocator);
+ HX_ADDREF(m_pAllocator);
}
HX_RESULT MPEGESPayloadFormat::SetAssemblerHeader(IHXValues* pHeader)
@@ -275,8 +283,9 @@
if (SUCCEEDED(retVal))
{
- m_pStreamHeader->GetPropertyULONG32("SamplesPerSecond",
+ m_pStreamHeader->GetPropertyULONG32("SamplesPerSecond",
m_ulRTPSamplesPerSecond);
+ m_pStreamHeader->GetPropertyULONG32("FramesPerMSecond", m_ulFrameRate);
}
HX_RELEASE(pMimeType);
@@ -299,6 +308,7 @@
return retVal;
}
+
STDMETHODIMP
MPEGESPayloadFormat::SetPacket(IHXPacket* pPacket)
{
@@ -343,6 +353,7 @@
return retVal;
}
+
STDMETHODIMP
MPEGESPayloadFormat::GetPacket(REF(IHXPacket*) pOutPacket)
{
@@ -353,7 +364,7 @@
HX_RESULT MPEGESPayloadFormat::CreateHXCodecPacket(UINT32* &pHXCodecDataOut)
{
HX_RESULT retVal = HXR_OK;
-
+
void* pHXCodecData = NULL;
if (!m_OutputQueue.IsEmpty())
{
@@ -431,7 +442,6 @@
// Add this packet to our list of input packets
if (SUCCEEDED(retVal))
{
- HX_ASSERT(m_InputQueue.IsEmpty());
pPacket->AddRef();
m_InputQueue.AddTail(pPacket);
m_ulInputQueueTotalSize += ulBuffSize;
@@ -455,75 +465,383 @@
return retVal;
}
-HX_RESULT MPEGESPayloadFormat::ReapMediaPacket(void)
+HX_RESULT MPEGESPayloadFormat::ParseMediaPacket(IHXPacket* pPacket)
{
- HX_RESULT retVal = HXR_OK;
-
- IHXPacket* pPacket = NULL;
+ HX_RESULT retVal = HXR_OK;
IHXBuffer* pBuffer = NULL;
UINT32 ulSize = 0;
-
- HXCODEC_DATA* pHXCodecData = NULL;
- HXCODEC_SEGMENTINFO* pHXCodecSegmentInfo = NULL;
- UINT32* pCodecDataBuf = NULL;
+ UINT8* pData = NULL;
+ UINT8 ulFrameType = 0;
- pPacket = (IHXPacket*)(m_InputQueue.GetHead());
- if (!pPacket)
+ //in some cases, one pPacket may contains many frames,
+ //so we need to get frame number from pPacket first.
+ UINT32 ulNumFrames = 0;
+ pBuffer = pPacket->GetBuffer();
+ if(pBuffer)
+ {
+ pData = pBuffer->GetBuffer();
+ ulSize = pBuffer->GetSize();
+
+ }
+ if(!pData)
{
+ HX_RELEASE(pBuffer);
return HXR_NO_DATA;
}
+ ulNumFrames = GetNumberOfFrames(pData, ulSize);
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,
+ "ReapMediaPacket frame number:%u, bufsize:%u", ulNumFrames, ulSize);
- // Allocate codec data header
- pCodecDataBuf = new UINT32[(sizeof(HXCODEC_DATA) + 3) / 4];
- if(!pCodecDataBuf)
+ if((ulNumFrames > 0) && !m_ulFrameRate)
{
- return HXR_OUTOFMEMORY;
+ static BOOL bStopFlag = FALSE;
+ if(!bStopFlag)
+ {
+ HXTLOG_APPROVED(LC_DEV_INFO, NETINPUT,
+ "Unable to determine the input video FrameRate which is required for \
calculating timestamps for this presentation. Stopping the encode!"); + \
HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT, + "Unable to determine the \
input video FrameRate which is required for calculating timestamps for this \
presentation. Stopping the encode!"); + }
+ bStopFlag = TRUE;
+ return HXR_NO_DATA;
}
- pBuffer = pPacket->GetBuffer();
- ulSize = pBuffer->GetSize();
+ //save cirtial variable
+ ULONG32 ulTimeStamp;
+ UINT16 uFlags;
+ ulTimeStamp = GetPacketTime(pPacket);
+ uFlags = pPacket->GetASMFlags();
+
+ //we just record PTS of I or P frame as previous PTS
+ if(!m_bFindFirstPTS)
+ {
+ ulFrameType = GetFrameType(pData, ulSize);
+ if((ulFrameType!=FRAME_TYPE_I) && (ulFrameType!=FRAME_TYPE_P))
+ {
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,
+ "if the first frame is not I or P, discard it %u", ulFrameType);
+ HX_RELEASE(pBuffer);
+ return HXR_NO_DATA;
+ }
+ else
+ {
+ m_bFindFirstPTS = TRUE;
+ m_ulPreTimeStamp = ulTimeStamp;
+ m_ulBFrameNumber = 0;
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,"got first ulTimeStamp:%u", \
ulTimeStamp); + }
+ }
+
+ HXCODEC_DATA* pHXCodecData = NULL;
+ UINT32* pCodecDataBuf = NULL;
+ UINT i = 0;
+ UINT32 ulBufSize = 0;
+ BYTE* pPtr = NULL;
+ UINT32 ulFrameDuration = 0;
+ ULONG32 ulTempTimeStamp = 0;
+ pPtr = pData;
+ ulBufSize = ulSize;
+ ulFrameDuration = (UINT32)((1.0*1000.0*1000000.0)/(double) m_ulFrameRate);
+ for(i=0; i<ulNumFrames; i++)
+ {
+ UINT32 ulAulength = 0;
- // Init. codec data header
- pHXCodecData = (HXCODEC_DATA*) pCodecDataBuf;
- pHXCodecData->timestamp = GetPacketTime(pPacket);
- pHXCodecData->sequenceNum = (UINT16)m_ulCodecDataSeqNumber;
- pHXCodecData->flags = pPacket->GetASMFlags();
- pHXCodecData->lastPacket = FALSE;
- pHXCodecData->dataLength = ulSize;
+ //get frame length
+ ulFrameType = GetFrameType(pPtr, ulBufSize);
+ switch(ulFrameType)
+ {
+ case FRAME_TYPE_I://I frame
+ case FRAME_TYPE_P://P frame
+ if(i==0)
+ {
+ //TS-demux will send a PES packet until it encounter a PES packet \
with PTS. + //if one PES packet without PTS, the PES packet will be \
appended to + //the previous PES packet that contains PTS.
+ //so, the first frame can get PTS from PES packet.
+ ulTempTimeStamp = ulTimeStamp;
+ m_ulPreTimeStamp = ulTempTimeStamp;
+ m_ulBFrameNumber = 0;
+ }
+ else
+ {
+ ulTempTimeStamp = m_ulPreTimeStamp + (m_lGapSize+1)*ulFrameDuration;
+ m_ulPreTimeStamp = ulTempTimeStamp;
+ m_ulBFrameNumber = 0;
+ }
+ break;
- pHXCodecData->numSegments = 1;
- pHXCodecData->Segments[0].bIsValid = TRUE;
- pHXCodecData->Segments[0].ulSegmentOffset = 0;
+ case FRAME_TYPE_B://B frame
+ if(i==0)
+ {
+ ulTempTimeStamp = ulTimeStamp;
+ }
+ else
+ {
+ ulTempTimeStamp = m_ulPreTimeStamp - \
(m_lGapSize-m_ulBFrameNumber)*ulFrameDuration; + }
+ m_ulBFrameNumber++;
+ break;
- if (m_pAllocator)
+ default:
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,"frame type: illegal, %lu", \
ulFrameType); + break;
+ }
+
+ if(i+1 == ulNumFrames)
+ {
+ ulAulength = ulBufSize;
+ }
+ else
+ {
+ ulAulength = GetFrameLength(pPtr, ulBufSize);
+ }
+
+ // Allocate codec data header
+ pCodecDataBuf = new UINT32[(sizeof(HXCODEC_DATA) + 3) / 4];
+ if(!pCodecDataBuf)
+ {
+ HX_RELEASE(pBuffer);
+ return HXR_OUTOFMEMORY;
+ }
+
+ // Init. codec data header
+ pHXCodecData = (HXCODEC_DATA*) pCodecDataBuf;
+ pHXCodecData->timestamp = ulTempTimeStamp;
+
+ pHXCodecData->sequenceNum = (UINT16)m_ulCodecDataSeqNumber;
+ if(i == 0)
+ {
+ pHXCodecData->flags = uFlags;
+ }
+ else
+ {
+ pHXCodecData->flags = HX_ASM_SWITCH_OFF;
+ }
+ pHXCodecData->lastPacket = FALSE;
+ pHXCodecData->dataLength = ulAulength; //for each AU length
+
+ pHXCodecData->numSegments = 1;
+ pHXCodecData->Segments[0].bIsValid = TRUE;
+ pHXCodecData->Segments[0].ulSegmentOffset = 0;
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,"index %u, ulAulength: %u, bufszie: \
%u, timestamp: %u, duration:%u", + i, ulAulength, ulBufSize, \
pHXCodecData->timestamp, ulFrameDuration); +
+ if((ulNumFrames > 1) && m_pAllocator)
+ {
+ //we need to creat new CHXBuffer for each frame
+ UINT8* pTempData = NULL;
+ IHXBuffer* pTempBuffer = NULL;
+ m_pClassFactory->CreateInstance(CLSID_IHXBuffer,(void**)&pTempBuffer);
+ if(pTempBuffer)
+ {
+ pTempBuffer->SetSize(ulAulength);
+ pTempData = pTempBuffer->GetBuffer();
+ if(!pTempData)
+ {
+ HX_DELETE(pHXCodecData);
+ HX_RELEASE(pBuffer);
+ HX_RELEASE(pTempBuffer);
+ return HXR_OUTOFMEMORY;
+ }
+ memcpy(pTempData, pPtr, ulAulength);
+
+ pHXCodecData->data = m_pAllocator->AddBuffer(pTempBuffer);
+ HX_RELEASE(pTempBuffer);
+ }
+ else
+ {
+ HX_DELETE(pHXCodecData);
+ HX_RELEASE(pBuffer);
+ return HXR_OUTOFMEMORY;
+ }
+ }
+ else if(m_pAllocator)
+ {
+ pHXCodecData->data = m_pAllocator->AddBuffer(pBuffer);
+ }
+ else
+ {
+ pHXCodecData->data = (UINT8*) new UINT32[(ulAulength + 3)/ 4];
+ if (pHXCodecData->data)
+ {
+ memcpy(pHXCodecData->data, pPtr, ulAulength);
+ }
+ else
+ {
+ HX_DELETE(pHXCodecData);
+ HX_RELEASE(pBuffer);
+ return HXR_OUTOFMEMORY;
+ }
+ }
+
+ m_OutputQueue.AddTail(pHXCodecData);
+ m_ulCodecDataSeqNumber++;
+
+ ulBufSize -= ulAulength;
+ pPtr += ulAulength;
+ }
+
+ HX_RELEASE(pBuffer);
+ return retVal;
+}
+
+
+HX_RESULT MPEGESPayloadFormat::GetNumberOfBFrames(IHXPacket* pPacket)
+{
+ HX_RESULT retVal = HXR_OK;
+ IHXBuffer* pBuffer = NULL;
+ UINT32 ulSize = 0;
+ UINT8* pData = NULL;
+ UINT32 ulNumFrames = 0;
+
+ pBuffer = pPacket->GetBuffer();
+ if(pBuffer)
{
- pHXCodecData->data = m_pAllocator->AddBuffer(pBuffer);
+ pData = pBuffer->GetBuffer();
+ ulSize = pBuffer->GetSize();
}
- else
+ if(!pData)
{
- pHXCodecData->data = (UINT8*) new UINT32[(ulSize + 3)/ 4];
- if (pHXCodecData->data)
+ HX_RELEASE(pBuffer);
+ return HXR_NO_DATA;
+ }
+ ulNumFrames = GetNumberOfFrames(pData, ulSize);
+
+ //seperate frames from buf if frame number>1
+ UINT i = 0;
+ UINT32 ulBufSize = 0;
+ BYTE* pPtr = NULL;
+ static UINT8 ulNumRefFrames = 0;
+
+ pPtr = pData;
+ ulBufSize = ulSize;
+ for(i=0; i<ulNumFrames; i++)
+ {
+ UINT32 ulAulength = 0;
+ UINT8 ulFrameType = 0;
+
+ //get frame type
+ ulFrameType = GetFrameType(pPtr, ulBufSize);
+ switch(ulFrameType)
{
- memcpy(pHXCodecData->data, pBuffer->GetBuffer(), ulSize);
+ case FRAME_TYPE_I://I frame
+ case FRAME_TYPE_P://P frame
+ if(!m_bFindFirstRefFrame)
+ {
+ //find first key frame(I or P), We need to record it.
+ m_ulBFrameNumber = 0;
+ m_bFindFirstRefFrame = TRUE;
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,"find first P frame");
+ }
+ else
+ {
+ if(!ulNumRefFrames)
+ {
+ //we need to record the number of B frames between secnod and \
third refrencr frames, + //because we maybe receive the following \
frames: I P B B P B B etc. + ulNumRefFrames++;
+ m_bFindFirstRefFrame = 0;
+ m_ulBFrameNumber = 0;
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,"find the first and second \
ref frame"); + }
+ else
+ {
+ //find second key frame(I or P), We can get number of B frames.
+ m_lGapSize = m_ulBFrameNumber;
+ m_ulBFrameNumber = 0;
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,"gap size:%u", \
m_lGapSize); + HX_RELEASE(pBuffer);
+ return retVal;
+ }
+ }
+ break;
+ case FRAME_TYPE_B://B frame
+ if(m_bFindFirstRefFrame)
+ {
+ m_ulBFrameNumber++;
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT," number of B frame: %u", \
m_ulBFrameNumber); + }
+ break;
+ default:
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,"frame type: illegal, %lu", \
ulFrameType); + break;
+ }
+
+ if(i+1 == ulNumFrames)
+ {
+ ulAulength = ulBufSize;
}
else
{
- HX_DELETE(pHXCodecData);
- retVal = HXR_OUTOFMEMORY;
+ ulAulength = GetFrameLength(pPtr, ulBufSize);
}
- }
- HX_RELEASE(pBuffer);
+ // Allocate codec data header
+ ulBufSize -= ulAulength;
+ pPtr += ulAulength;
+ }
- if (SUCCEEDED(retVal))
+ HX_RELEASE(pBuffer);
+ return retVal;
+}
+
+
+HX_RESULT MPEGESPayloadFormat::ReapMediaPacket(void)
+{
+ HX_RESULT retVal = HXR_OK;
+ IHXPacket* pPacket = NULL;
+
+ if(m_lGapSize == -1)
{
+ pPacket = (IHXPacket*)(m_InputQueue.GetHead());
+ if (!pPacket)
+ {
+ return HXR_NO_DATA;
+ }
+
+ retVal = GetNumberOfBFrames(pPacket);
+
+ //we will move the packet into a cache quuee until we get gap size of B \
frame. + pPacket->AddRef();
+ m_CacheQueue.AddTail(pPacket);
m_InputQueue.RemoveHead();
- HX_RELEASE(pPacket);
-
- m_OutputQueue.AddTail(pHXCodecData);
- m_ulCodecDataSeqNumber++;
+ HX_RELEASE(pPacket);
}
+ //if we get gap size of B fram, we can start encode.
+ if(m_lGapSize != -1)
+ {
+ if(!m_CacheQueue.IsEmpty())
+ {
+ while (!m_CacheQueue.IsEmpty())
+ {
+ HXTLOG_APPROVED(LC_DEV_DIAG, NETINPUT,"num of packet in cache \
queue:%lu", m_CacheQueue.GetCount()); + //get packet from cache queue \
+ pPacket = (IHXPacket*)(m_CacheQueue.GetHead());
+ if (!pPacket)
+ {
+ return HXR_NO_DATA;
+ }
+ retVal = ParseMediaPacket(pPacket);
+
+ m_CacheQueue.RemoveHead();
+ HX_RELEASE(pPacket);
+ }
+ }
+ else
+ {
+ pPacket = (IHXPacket*)(m_InputQueue.GetHead());
+ if (!pPacket)
+ {
+ return HXR_NO_DATA;
+ }
+ retVal = ParseMediaPacket(pPacket);
+
+ m_InputQueue.RemoveHead();
+ HX_RELEASE(pPacket);
+ }
+ }
+
return retVal;
}
@@ -545,6 +863,17 @@
{
IHXPacket* pDeadPacket;
+ //release cache queue first
+ while ((ulCount > 0) && (!m_CacheQueue.IsEmpty()))
+ {
+ pDeadPacket = (IHXPacket*) m_CacheQueue.RemoveHead();
+ HX_RELEASE(pDeadPacket);
+ if (ulCount != FLUSH_ALL_PACKETS)
+ {
+ ulCount--;
+ }
+ }
+
while ((ulCount > 0) && (!m_InputQueue.IsEmpty()))
{
pDeadPacket = (IHXPacket*) m_InputQueue.RemoveHead();
_______________________________________________
Datatype-cvs mailing list
Datatype-cvs@helixcommunity.org
http://lists.helixcommunity.org/mailman/listinfo/datatype-cvs
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic