[prev in list] [next in list] [prev in thread] [next in thread]
List: helix-producer-dev
Subject: Re: [Helix-producer-dev] SMIL and live broadcast - some progress
From: "Hasan Bulut" <hbulut () indiana ! edu>
Date: 2004-05-22 22:26:44
Message-ID: 001301c4404b$dfbc21f0$4468389c () ucs ! indiana ! edu
[Download RAW message or body]
I tried another thing and did the test. Currently it is better than before,
but still sometimes it gets out of sync. Besides this the video sometimes
freezes.
As I mentioend before I was timestamping packets starting from 0 (zero). I
changed this and I get the time of the first packet from system clock. The
timestamp of other packets are shifted by the amount of the timestamp of the
first packet. In addition to this I modified the SMIL file like this:
<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
<head>
<layout>
<root-layout width="354" height="290" backgroundColor="blue" />
<region id="video_region" top="1" left="1"
width="352" height="288" />
</layout>
</head>
<body>
<par endsync="audio">
<video src="rtsp://myhelixserver/broadcast/video.rm?wallclock=sync"
id="video" region="video_region" begin="audio.beginEvent
+4s"/>
<audio src="rtsp://myhelixserver/broadcast/audio.rm?wallclock=sync"
id="audio"/>
</par>
</body>
</smil>
With this SMIL file, most of the time it starts synchronized. But as you
notice I delay video also (audio.beginEvent + 4s) If I don't add this 4s, it
is not syncronized. Also there are times it is not synchronized at the start
(1 sec). When it is not synchronized at the start, I simply shut down
encoder and start again. But I beleive I still have the problem due to this
random behaviour.
When the video freezes, I also noticed that helix server also prints
messages like U: 0:0 or U: 0:1. I posted this to helix-server list, but
nobody replied yet.
> Can't you just hold a fixed buffer of audio or video in your app before
> you pass on the samples to the producer SDK?
I can try this. But if the audio or video latency changes when I run them on
seperate machine, then I'm not sure if it works.
Thanks,
Hasan
----- Original Message -----
From: "Steve McMillen" <stevemc@real.com>
To: "Hasan Bulut" <hbulut@indiana.edu>
Cc: "Wolfgang Schildbach" <wschildbach@helixcommunity.org>; "Jechiam Gural"
<gural@noterik.nl>; <helix-producer-dev@lists.helixcommunity.org>;
<sannie@noterik.nl>
Sent: Saturday, May 22, 2004 5:07 PM
Subject: Re: [Helix-producer-dev] SMIL and live broadcast
> Can't you just hold a fixed buffer of audio or video in your app before
> you pass on the samples to the producer SDK?
>
> Steve
>
> Hasan Bulut wrote:
>
> >Video latency is 7 sec and audio latency is 14sec. Is there a way to
delay
> >video i.e. 7 or 8 sec. so the first audio packet will be received before
the
> >first video packet by the server. for live streaming case.
> >Thanks
> >Hasan
> >
> >----- Original Message -----
> >From: "Jechiam Gural" <gural@noterik.nl>
> >To: "Wolfgang Schildbach" <wschildbach@helixcommunity.org>; "Steve
McMillen"
> ><stevemc@real.com>; "Hasan Bulut" <hbulut@indiana.edu>
> >Cc: <helix-producer-dev@lists.helixcommunity.org>; <sannie@noterik.nl>
> >Sent: Friday, May 21, 2004 4:27 AM
> >Subject: Re: [Helix-producer-dev] SMIL and live broadcast
> >
> >
> >
> >
> >>Wolfgang, Steve
> >>
> >>We have setups in which we run a live video only stream on one PC and a
> >>number of translated audio streams (1 up to X) on a separate PC that
need
> >>to be delivered in sync to the clients (live mode). The systems clocks
can
> >>be synced using NTP on a network. I'm not sure about the precise time
> >>resolution but more or less sufficient for lip sync applications as far
as
> >>I remember (40 Millie Seconds). BTW there are special cards one can use
to
> >>synchronize different computers to obtain an even higher sync
resolution.
> >>
> >>Regards,
> >>
> >>Jechiam
> >>
> >>
> >>
> >>At 09:41 AM 5/21/2004, Wolfgang Schildbach wrote:
> >>
> >>
> >>>At 09:26 PM 5/20/2004 -0700, Steve McMillen wrote:
> >>>
> >>>
> >>>>I think I'll let Wolfgang or someone else speak to the ability for the
> >>>>audioDelayCompPrefilter to correct the latency. I get all cross-eyed
> >>>>when I think about advance or delay of audio relative to the video
> >>>>stream. I'm really not sure if its even possible to apply an
"advance"
> >>>>(or should I say delay?) on audio for a live stream but my
understanding
> >>>>is you are just tweaking timestamps, not really moving the packets
> >>>>forward in time. If the later were possible, I think we'd be awarding
> >>>>Wolfgang a Nobel prize:)
> >>>>
> >>>>
> >>>Thanks for pointing out that obvious omission :-)
> >>>
> >>>The delayCompPrefilter won't be able to shorten the delay on live
feeds;
> >>>indeed it primarily just tweaks timestamps (plus some fixing up of the
> >>>beginning of the stream to make sure we don't get negative timestamps
> >>>
> >>>
> >etc.).
> >
> >
> >>>You may have mentioned this earlier, but why are you running two
separate
> >>>encoders again? I tend to believe that if you were encoding this into
one
> >>>stream, things would work. I believe that if you send them as two
> >>>different streams, the player will not reference them to the same
> >>>
> >>>
> >timeline
> >
> >
> >>>(but be sure to ask over on the client list for details).
> >>>
> >>>If you are seeing the delay in a live stream situation, you need to
take
> >>>
> >>>
> >a
> >
> >
> >>>look at the encoder/server side of things. In my understanding, SMIL is
a
> >>>client-side technology and the server does not care about it at all.
And
> >>>since the streams are live, beginning or ending them at different times
> >>>should not change the delay between them.
> >>>
> >>>So, could you detail again your encoding setup, and explain why you
need
> >>>to separate streams?
> >>>
> >>>- Wolfgang
> >>>
> >>>
> >>>
> >>>
> >>>>The latency changes in the player for each run would suggest we need
to
> >>>>get some of the client folks involved.
> >>>>
> >>>>Thx, Steve
> >>>>
> >>>>Hasan Bulut wrote:
> >>>>
> >>>>
> >>>>
> >>>>>In my case video latency is less than audio latency. That is, I see
> >>>>>
> >>>>>
> >video
> >
> >
> >>>>>almost 7 seconds before the audio speaken at that time. I need to
delay
> >>>>>video ( let's say 7 sec in this case). Will I be able to shorten the
> >>>>>
> >>>>>
> >delay
> >
> >
> >>>>>of audio so that both can be synchronized?
> >>>>>
> >>>>>I also noticed that the latency difference between audio and video
> >>>>>
> >>>>>
> >changes
> >
> >
> >>>>>every time I restart the player for the live streams. In this case I
> >>>>>
> >>>>>
> >don't
> >
> >
> >>>>>stop the encoding, I just stop the player, and then I start it again
> >>>>>
> >>>>>
> >while
> >
> >
> >>>>>the same encoding continues.
> >>>>>Meanwhile I'll look into audioDelayCompPrefilter.
> >>>>>Thanks,
> >>>>>Hasan
> >>>>>
> >>>>>
> >>>>>----- Original Message ----- From: "Steve McMillen"
<stevemc@real.com>
> >>>>>To: "Hasan Bulut" <hbulut@indiana.edu>
> >>>>>Cc: <helix-producer-dev@lists.helixcommunity.org>
> >>>>>Sent: Thursday, May 20, 2004 7:29 PM
> >>>>>Subject: Re: [Helix-producer-dev] SMIL and live broadcast
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>Ah! I had not realized that clipBegin could not be applied to live
> >>>>>>
> >>>>>>
> >feeds.
> >
> >
> >>>>>>One other option is to delay or advance the audio stream in producer
> >>>>>>using the audioDelayCompPrefilter (you can thank Wolfgang for this
> >>>>>>contribution). See the sdk on how to use this prefilter.
> >>>>>>
> >>>>>>If that does not work out, we should get the client dev mailing list
> >>>>>>onto this thread to figure out if there are other ways that streams
> >>>>>>
> >>>>>>
> >can
> >
> >
> >>>>>>be shifted relative to one another in the client.
> >>>>>>
> >>>>>>Thx, Steve
> >>>>>>
> >>>>>>Hasan Bulut wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>Unfortunately I could not make it work using clipBegin. And I also
> >>>>>>>
> >>>>>>>
> >>>>>noticed
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>this line in the following link:
> >>>>>>>"The clip-begin and clip-end attributes do not work with
broadcasts.
> >>>>>>>However, you can terminate a broadcast before the stream stops by
> >>>>>>>
> >>>>>>>
> >using
> >
> >
> >>>>>the
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>SMIL end attribute. "
> >>>>>>>
> >>>>>>>
>
>>>>>>http://service.real.com/help/library/guides/production8/htmfiles/live.
h
> >>>>>>
> >>>>>>
> >tm
> >
> >
> >>>>>>>I also add "?wallclock=group1" to the end of the audio and video
> >>>>>>>
> >>>>>>>
> >stream
> >
> >
> >>>>>>>links. Both encoding sessions are running on the same machine. I
pass
> >>>>>>>
> >>>>>>>
> >the
> >
> >
> >>>>>>>packets from an application and the start time of the first packet
of
> >>>>>>>
> >>>>>>>
> >>>>>both
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>audio and video is 0 (zero). Video latency is 7 seconds and audio
> >>>>>>>
> >>>>>>>
> >latency
> >
> >
> >>>>>is
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>14 seconds. The important thing for me is to start them
> >>>>>>>
> >>>>>>>
> >synchronized.
> >
> >
> >>>>>>>Can I delay the video stream at the producer before sending the
> >>>>>>>
> >>>>>>>
> >stream to
> >
> >
> >>>>>>>the helix server?
> >>>>>>>
> >>>>>>>Thanks,
> >>>>>>>Hasan
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>----- Original Message ----- From: "Steve McMillen"
> >>>>>>>
> >>>>>>>
> ><stevemc@real.com>
> >
> >
> >>>>>>>To: "Hasan Bulut" <hbulut@indiana.edu>
> >>>>>>>Cc: <helix-producer-dev@lists.helixcommunity.org>
> >>>>>>>Sent: Thursday, May 20, 2004 3:15 PM
> >>>>>>>Subject: Re: [Helix-producer-dev] SMIL and live broadcast
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>First, I think you should use clipBegin. clipBegin indicates the
> >>>>>>>>
> >>>>>>>>
> >clip's
> >
> >
> >>>>>>>>internal timing where being just means delay start of playback for
a
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>while.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>Also, I you may have mentioned this already but its been a long
> >>>>>>>>
> >>>>>>>>
> >thread.
> >
> >
> >>>>>>>>Make sure you do have both encodes on the same box or both boxes
> >>>>>>>>
> >>>>>>>>
> >have
> >
> >
> >>>>>>>>their clocks synchronized.
> >>>>>>>>
> >>>>>>>>Finally, are you sure you are using wallclock timestamps in the
> >>>>>>>>
> >>>>>>>>
> >Producer
> >
> >
> >>>>>>>>SDK. I think that's default but I'm not positive.
> >>>>>>>>
> >>>>>>>>Given you have done the above, then the player should hold one
> >>>>>>>>
> >>>>>>>>
> >stream
> >
> >
> >>>>>>>>until wallclock timestamps align if you include the wallclock=
> >>>>>>>>
> >>>>>>>>
> >property
> >
> >
> >>>>>>>>you have below.
> >>>>>>>>
> >>>>>>>>Steve
> >>>>>>>>
> >>>>>>>>Hasan Bulut wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>Hi,
> >>>>>>>>>After fixing latency problem for audio, I tried to run audio and
> >>>>>>>>>
> >>>>>>>>>
> >video
> >
> >
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>streams using a SMIL file residing on the helix server. When I run
> >>>>>>>audio-only or video-only, their latency stays constant. Video has 7
> >>>>>>>
> >>>>>>>
> >sec
> >
> >
> >>>>>>>latency and audio has 14 second latency. I tried to run them
together
> >>>>>>>
> >>>>>>>
> >>>>>using
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>a SMIL file. But it seems whenever I start player linking to the
SMIL
> >>>>>>>
> >>>>>>>
> >>>>>file,
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>audio and video are not synchronized. I modify the SMIL file to
> >>>>>>>
> >>>>>>>
> >adjust
> >
> >
> >>>>>the
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>latency difference, but they seem again unsynchronized. I'm not
sure
> >>>>>>>
> >>>>>>>
> >if
> >
> >
> >>>>>I'm
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>doing something wrong with the SMIL. Both streams are live streams.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>>Here is my sample SMIL file content. Everything is same except
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>myhelixserveraddress indicates helix server address.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>><smil xmlns="http://www.w3.org/2001/SMIL20/Language">
> >>>>>>>>><head>
> >>>>>>>>><layout>
> >>>>>>>>><root-layout width="354" height="290" backgroundColor="blue" />
> >>>>>>>>><region id="video_region" top="1" left="1" width="352"
height="288"
> >>>>>>>>>
> >>>>>>>>>
> >>>>>/>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>>></layout>
> >>>>>>>>></head>
> >>>>>>>>><body>
> >>>>>>>>><par endsync="video">
> >>>>>>>>> <video
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
>
>>>>>>src="rtsp://myhelixserveraddress/broadcast/video.rm?wallclock=group1""
> >>>>>>
> >>>>>>
> >>>>>>>id="video" region="video_region" begin="7s"/>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>> <audio
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
>
>>>>>>src="rtsp://myhelixserveraddress/broadcast/audio.rm?wallclock=group1""
/
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>></par>
> >>>>>>>>></body>
> >>>>>>>>></smil>
> >>>>>>>>>
> >>>>>>>>>Audio streaming causes to reconnect to helix server in the first
> >>>>>>>>>
> >>>>>>>>>
> >1-2
> >
> >
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>second whenever I restart player. The distributed producer version
10
> >>>>>>>
> >>>>>>>
> >>>>>that
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>runs from command line does the same thing. Do you think this
causes
> >>>>>>>
> >>>>>>>
> >this
> >
> >
> >>>>>>>synchronization problem? or is there anything with my SMIL file?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>>Thanks,
> >>>>>>>>>Hasan
> >>>>>>>>>_______________________________________________
> >>>>>>>>>Helix-producer-dev mailing list
> >>>>>>>>>Helix-producer-dev@lists.helixcommunity.org
>
>>>>>>>>>http://lists.helixcommunity.org/mailman/listinfo/helix-producer-dev
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>_______________________________________________
> >>>>Helix-producer-dev mailing list
> >>>>Helix-producer-dev@lists.helixcommunity.org
> >>>>http://lists.helixcommunity.org/mailman/listinfo/helix-producer-dev
> >>>>
> >>>>
> >>>-
> >>>- Wolfgang Schildbach, Principal Codec Engineer, RealNetworks Codec
Group
> >>>- email wschildbach@helixcommunity.org
> >>>
> >>>
> >>>
> >>>_______________________________________________
> >>>Helix-producer-dev mailing list
> >>>Helix-producer-dev@lists.helixcommunity.org
> >>>http://lists.helixcommunity.org/mailman/listinfo/helix-producer-dev
> >>>
> >>>
> >>>
> >>>
>
>>--------------------------------------------------------------------------
> >>
> >>
> >-------------------------------------------
> >
> >
> >>Noterik B.V.
> >>Tel. +31-(0)20-663 62 62
> >>Fax.+31-(0)20-468 84 05
> >>http://www.noterik.nl
> >>
> >>Take a look at our streaming solutions:
> >>
> >>
> >http://www.streamedit.com/demo.html
> >
> >
> >>Recent project: http://managenergy.noterik.com/flashintroduction.html
>
>>--------------------------------------------------------------------------
> >>
> >>
> >-------------------------------------------
> >
> >
> >>
> >>
> >>
> >>
> >
> >
> >
> >
>
>
>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic