[prev in list] [next in list] [prev in thread] [next in thread]
List: helix-producer-dev
Subject: [Helix-producer-dev] Re: decreasing audio latency
From: Steve McMillen <stevemc () real ! com>
Date: 2004-05-22 22:06:52
Message-ID: 40AFCEFC.5090808 () real ! com
[Download RAW message or body]
Yes, the reason audio latencies are higher is due to superblocking.
That's fixed per codec and not configurable just yet.
Steve
Hasan Bulut wrote:
>In addition to my previous email below -
>Why audio latency is higher than video latency? Video encoding uses more CPU
>and memory than audio encoding. One can expect video latency be higher than
>audio latency because of this reason.
>Steve mentioned me a technique called superblocking that is used in audio
>encoding. Is this the only reason for this high latency? Can I disable it
>from mediasinkencoder?
>
>Thanks,
>Hasan
>
>
>
>
>
>>Video latency is 7 sec and audio latency is 14sec. Is there a way to delay
>>video i.e. 7 or 8 sec. so the first audio packet will be received before
>>
>>
>the
>
>
>>first video packet by the server. for live streaming case.
>>Thanks
>>Hasan
>>
>>----- Original Message -----
>>From: "Jechiam Gural" <gural@noterik.nl>
>>To: "Wolfgang Schildbach" <wschildbach@helixcommunity.org>; "Steve
>>
>>
>McMillen"
>
>
>><stevemc@real.com>; "Hasan Bulut" <hbulut@indiana.edu>
>>Cc: <helix-producer-dev@lists.helixcommunity.org>; <sannie@noterik.nl>
>>Sent: Friday, May 21, 2004 4:27 AM
>>Subject: Re: [Helix-producer-dev] SMIL and live broadcast
>>
>>
>>
>>
>>>Wolfgang, Steve
>>>
>>>We have setups in which we run a live video only stream on one PC and a
>>>number of translated audio streams (1 up to X) on a separate PC that
>>>
>>>
>need
>
>
>>>to be delivered in sync to the clients (live mode). The systems clocks
>>>
>>>
>can
>
>
>>>be synced using NTP on a network. I'm not sure about the precise time
>>>resolution but more or less sufficient for lip sync applications as far
>>>
>>>
>as
>
>
>>>I remember (40 Millie Seconds). BTW there are special cards one can use
>>>
>>>
>to
>
>
>>>synchronize different computers to obtain an even higher sync
>>>
>>>
>resolution.
>
>
>>>Regards,
>>>
>>>Jechiam
>>>
>>>
>>>
>>>At 09:41 AM 5/21/2004, Wolfgang Schildbach wrote:
>>>
>>>
>>>>At 09:26 PM 5/20/2004 -0700, Steve McMillen wrote:
>>>>
>>>>
>>>>>I think I'll let Wolfgang or someone else speak to the ability for the
>>>>>audioDelayCompPrefilter to correct the latency. I get all cross-eyed
>>>>>when I think about advance or delay of audio relative to the video
>>>>>stream. I'm really not sure if its even possible to apply an
>>>>>
>>>>>
>"advance"
>
>
>>>>>(or should I say delay?) on audio for a live stream but my
>>>>>
>>>>>
>understanding
>
>
>>>>>is you are just tweaking timestamps, not really moving the packets
>>>>>forward in time. If the later were possible, I think we'd be awarding
>>>>>Wolfgang a Nobel prize:)
>>>>>
>>>>>
>>>>Thanks for pointing out that obvious omission :-)
>>>>
>>>>The delayCompPrefilter won't be able to shorten the delay on live
>>>>
>>>>
>feeds;
>
>
>>>>indeed it primarily just tweaks timestamps (plus some fixing up of the
>>>>beginning of the stream to make sure we don't get negative timestamps
>>>>
>>>>
>>etc.).
>>
>>
>>>>You may have mentioned this earlier, but why are you running two
>>>>
>>>>
>separate
>
>
>>>>encoders again? I tend to believe that if you were encoding this into
>>>>
>>>>
>one
>
>
>>>>stream, things would work. I believe that if you send them as two
>>>>different streams, the player will not reference them to the same
>>>>
>>>>
>>timeline
>>
>>
>>>>(but be sure to ask over on the client list for details).
>>>>
>>>>If you are seeing the delay in a live stream situation, you need to
>>>>
>>>>
>take
>
>
>>a
>>
>>
>>>>look at the encoder/server side of things. In my understanding, SMIL is
>>>>
>>>>
>a
>
>
>>>>client-side technology and the server does not care about it at all.
>>>>
>>>>
>And
>
>
>>>>since the streams are live, beginning or ending them at different times
>>>>should not change the delay between them.
>>>>
>>>>So, could you detail again your encoding setup, and explain why you
>>>>
>>>>
>need
>
>
>>>>to separate streams?
>>>>
>>>>- Wolfgang
>>>>
>>>>
>>>>
>>>>
>>>>>The latency changes in the player for each run would suggest we need
>>>>>
>>>>>
>to
>
>
>>>>>get some of the client folks involved.
>>>>>
>>>>>Thx, Steve
>>>>>
>>>>>Hasan Bulut wrote:
>>>>>
>>>>>
>>>>>
>>>>>>In my case video latency is less than audio latency. That is, I see
>>>>>>
>>>>>>
>>video
>>
>>
>>>>>>almost 7 seconds before the audio speaken at that time. I need to
>>>>>>
>>>>>>
>delay
>
>
>>>>>>video ( let's say 7 sec in this case). Will I be able to shorten the
>>>>>>
>>>>>>
>>delay
>>
>>
>>>>>>of audio so that both can be synchronized?
>>>>>>
>>>>>>I also noticed that the latency difference between audio and video
>>>>>>
>>>>>>
>>changes
>>
>>
>>>>>>every time I restart the player for the live streams. In this case I
>>>>>>
>>>>>>
>>don't
>>
>>
>>>>>>stop the encoding, I just stop the player, and then I start it again
>>>>>>
>>>>>>
>>while
>>
>>
>>>>>>the same encoding continues.
>>>>>>Meanwhile I'll look into audioDelayCompPrefilter.
>>>>>>Thanks,
>>>>>>Hasan
>>>>>>
>>>>>>
>>>>>>----- Original Message ----- From: "Steve McMillen"
>>>>>>
>>>>>>
><stevemc@real.com>
>
>
>>>>>>To: "Hasan Bulut" <hbulut@indiana.edu>
>>>>>>Cc: <helix-producer-dev@lists.helixcommunity.org>
>>>>>>Sent: Thursday, May 20, 2004 7:29 PM
>>>>>>Subject: Re: [Helix-producer-dev] SMIL and live broadcast
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Ah! I had not realized that clipBegin could not be applied to live
>>>>>>>
>>>>>>>
>>feeds.
>>
>>
>>>>>>>One other option is to delay or advance the audio stream in producer
>>>>>>>using the audioDelayCompPrefilter (you can thank Wolfgang for this
>>>>>>>contribution). See the sdk on how to use this prefilter.
>>>>>>>
>>>>>>>If that does not work out, we should get the client dev mailing list
>>>>>>>onto this thread to figure out if there are other ways that streams
>>>>>>>
>>>>>>>
>>can
>>
>>
>>>>>>>be shifted relative to one another in the client.
>>>>>>>
>>>>>>>Thx, Steve
>>>>>>>
>>>>>>>Hasan Bulut wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>Unfortunately I could not make it work using clipBegin. And I also
>>>>>>>>
>>>>>>>>
>>>>>>noticed
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>this line in the following link:
>>>>>>>>"The clip-begin and clip-end attributes do not work with
>>>>>>>>
>>>>>>>>
>broadcasts.
>
>
>>>>>>>>However, you can terminate a broadcast before the stream stops by
>>>>>>>>
>>>>>>>>
>>using
>>
>>
>>>>>>the
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>SMIL end attribute. "
>>>>>>>>
>>>>>>>>
>>>>>>http://service.real.com/help/library/guides/production8/htmfiles/live.h
>>>>>>
>>>>>>
>>tm
>>
>>
>>>>>>>>I also add "?wallclock=group1" to the end of the audio and video
>>>>>>>>
>>>>>>>>
>>stream
>>
>>
>>>>>>>>links. Both encoding sessions are running on the same machine. I
>>>>>>>>
>>>>>>>>
>pass
>
>
>>the
>>
>>
>>>>>>>>packets from an application and the start time of the first packet
>>>>>>>>
>>>>>>>>
>of
>
>
>>>>>>both
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>audio and video is 0 (zero). Video latency is 7 seconds and audio
>>>>>>>>
>>>>>>>>
>>latency
>>
>>
>>>>>>is
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>14 seconds. The important thing for me is to start them
>>>>>>>>
>>>>>>>>
>>synchronized.
>>
>>
>>>>>>>>Can I delay the video stream at the producer before sending the
>>>>>>>>
>>>>>>>>
>>stream to
>>
>>
>>>>>>>>the helix server?
>>>>>>>>
>>>>>>>>Thanks,
>>>>>>>>Hasan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>----- Original Message ----- From: "Steve McMillen"
>>>>>>>>
>>>>>>>>
>><stevemc@real.com>
>>
>>
>>>>>>>>To: "Hasan Bulut" <hbulut@indiana.edu>
>>>>>>>>Cc: <helix-producer-dev@lists.helixcommunity.org>
>>>>>>>>Sent: Thursday, May 20, 2004 3:15 PM
>>>>>>>>Subject: Re: [Helix-producer-dev] SMIL and live broadcast
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>First, I think you should use clipBegin. clipBegin indicates the
>>>>>>>>>
>>>>>>>>>
>>clip's
>>
>>
>>>>>>>>>internal timing where being just means delay start of playback for
>>>>>>>>>
>>>>>>>>>
>a
>
>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>while.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>Also, I you may have mentioned this already but its been a long
>>>>>>>>>
>>>>>>>>>
>>thread.
>>
>>
>>>>>>>>>Make sure you do have both encodes on the same box or both boxes
>>>>>>>>>
>>>>>>>>>
>>have
>>
>>
>>>>>>>>>their clocks synchronized.
>>>>>>>>>
>>>>>>>>>Finally, are you sure you are using wallclock timestamps in the
>>>>>>>>>
>>>>>>>>>
>>Producer
>>
>>
>>>>>>>>>SDK. I think that's default but I'm not positive.
>>>>>>>>>
>>>>>>>>>Given you have done the above, then the player should hold one
>>>>>>>>>
>>>>>>>>>
>>stream
>>
>>
>>>>>>>>>until wallclock timestamps align if you include the wallclock=
>>>>>>>>>
>>>>>>>>>
>>property
>>
>>
>>>>>>>>>you have below.
>>>>>>>>>
>>>>>>>>>Steve
>>>>>>>>>
>>>>>>>>>Hasan Bulut wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>Hi,
>>>>>>>>>>After fixing latency problem for audio, I tried to run audio and
>>>>>>>>>>
>>>>>>>>>>
>>video
>>
>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>streams using a SMIL file residing on the helix server. When I run
>>>>>>>>audio-only or video-only, their latency stays constant. Video has 7
>>>>>>>>
>>>>>>>>
>>sec
>>
>>
>>>>>>>>latency and audio has 14 second latency. I tried to run them
>>>>>>>>
>>>>>>>>
>together
>
>
>>>>>>using
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>a SMIL file. But it seems whenever I start player linking to the
>>>>>>>>
>>>>>>>>
>SMIL
>
>
>>>>>>file,
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>audio and video are not synchronized. I modify the SMIL file to
>>>>>>>>
>>>>>>>>
>>adjust
>>
>>
>>>>>>the
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>latency difference, but they seem again unsynchronized. I'm not
>>>>>>>>
>>>>>>>>
>sure
>
>
>>if
>>
>>
>>>>>>I'm
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>doing something wrong with the SMIL. Both streams are live streams.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>>Here is my sample SMIL file content. Everything is same except
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>myhelixserveraddress indicates helix server address.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>><smil xmlns="http://www.w3.org/2001/SMIL20/Language">
>>>>>>>>>><head>
>>>>>>>>>><layout>
>>>>>>>>>><root-layout width="354" height="290" backgroundColor="blue" />
>>>>>>>>>><region id="video_region" top="1" left="1" width="352"
>>>>>>>>>>
>>>>>>>>>>
>height="288"
>
>
>>>>>>/>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>></layout>
>>>>>>>>>></head>
>>>>>>>>>><body>
>>>>>>>>>><par endsync="video">
>>>>>>>>>> <video
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>src="rtsp://myhelixserveraddress/broadcast/video.rm?wallclock=group1""
>>>>>>
>>>>>>
>>>>>>>>id="video" region="video_region" begin="7s"/>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> <audio
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>src="rtsp://myhelixserveraddress/broadcast/audio.rm?wallclock=group1""/
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>></par>
>>>>>>>>>></body>
>>>>>>>>>></smil>
>>>>>>>>>>
>>>>>>>>>>Audio streaming causes to reconnect to helix server in the first
>>>>>>>>>>
>>>>>>>>>>
>>1-2
>>
>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>second whenever I restart player. The distributed producer version
>>>>>>>>
>>>>>>>>
>10
>
>
>>>>>>that
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>runs from command line does the same thing. Do you think this
>>>>>>>>
>>>>>>>>
>causes
>
>
>>this
>>
>>
>>>>>>>>synchronization problem? or is there anything with my SMIL file?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>>Thanks,
>>>>>>>>>>Hasan
>>>>>>>>>>_______________________________________________
>>>>>>>>>>Helix-producer-dev mailing list
>>>>>>>>>>Helix-producer-dev@lists.helixcommunity.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>http://lists.helixcommunity.org/mailman/listinfo/helix-producer-dev
>>>>>>>>
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>_______________________________________________
>>>>>Helix-producer-dev mailing list
>>>>>Helix-producer-dev@lists.helixcommunity.org
>>>>>http://lists.helixcommunity.org/mailman/listinfo/helix-producer-dev
>>>>>
>>>>>
>>>>-
>>>>- Wolfgang Schildbach, Principal Codec Engineer, RealNetworks Codec
>>>>
>>>>
>Group
>
>
>>>>- email wschildbach@helixcommunity.org
>>>>
>>>>
>>>>
>>>>_______________________________________________
>>>>Helix-producer-dev mailing list
>>>>Helix-producer-dev@lists.helixcommunity.org
>>>>http://lists.helixcommunity.org/mailman/listinfo/helix-producer-dev
>>>>
>>>>
>>>>
>>>>
>>--------------------------------------------------------------------------
>>-------------------------------------------
>>
>>
>>>Noterik B.V.
>>>Tel. +31-(0)20-663 62 62
>>>Fax.+31-(0)20-468 84 05
>>>http://www.noterik.nl
>>>
>>>Take a look at our streaming solutions:
>>>
>>>
>>http://www.streamedit.com/demo.html
>>
>>
>>>Recent project: http://managenergy.noterik.com/flashintroduction.html
>>>
>>>
>>--------------------------------------------------------------------------
>>-------------------------------------------
>>
>>
>>>
>>>
>>>
>>>
>>_______________________________________________
>>Helix-producer-dev mailing list
>>Helix-producer-dev@lists.helixcommunity.org
>>http://lists.helixcommunity.org/mailman/listinfo/helix-producer-dev
>>
>>
>>
>>
>>
>
>
>
>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic