WebRTC Native APIs

WebRTC Native APIs

Version 2.0 (libjingle r115) February 2012 The WebRTC native APIs are implemented based on the following WebRTC spec. The code that implements WebRTC native APIs (including the Stream and the PeerConnection APIs) are available in libjingle. A sample client application is also provided there. The target audience of this document are those who want to use WebRTC Native APIs to implement WebRTC javascript APIs or to develop native RTC applications. Contents 1 What is new 2 Block diagram 3 Calling sequences 3.1 Set up a call 3.2 Receive a call 3.3 Close down a call 4 Threading model 5 Stream APIs (mediastream.h) 5.1 Class MediaStreamTrackInterface 5.2 Class VideoTrackInterface 5.3 Class LocalVideoTrackInterface 5.4 Class AudioTrackInterface 5.5 Class LocalAudioTrackInterface 5.6 Class cricket::VideoRenderer, cricket::VideoCapturer 5.7 Class webrtc::AudioDeviceModule 5.8 Class MediaStreamInterface 5.9 Class LocalMediaStreamInterface 6 PeerConnection APIs (peerconnection.h) 6.1 Class StreamCollectionInterface 6.2 Class PeerConnectionObserver 6.3 Class PortAllocatorFactoryInterface 6.4 Class PeerConnectionFactoryInterface 6.5 Function CreatePeerConnectionFactory 6.6 Function CreatePeerConnectionFactory 6.7 Class PeerConnectionInterface 7 Reference [title2] What is new[/title2] Compared to the previous version of WebRTC native APIs (code is available here as part of libjingle r115), the major difference is the new version contains the implementation of the Stream APIs. With the Stream APIs, the audio/video I/O will be set to the MediaTrack instead of directly to the PeerConnection. Also PeerConnection now takes and returns the MediaStream object instead of just the media stream label as in the previous version. In addition to this, the signaling protocol has also changed to ROAP in this version, however, this should be transparent to the users of the APIs. For those who have already developed the application based on the previous version, please check this patch as an example on how to migrate to the new APIs. The sample client application mentioned above can also be used as a reference. [title2] Block diagram[/title2] [title2] Calling sequences[/title2] Set up a call Receive a call Close down a call Threading model WebRTC native APIs use two globally available threads: the signaling thread and the worker thread. Depending on how the PeerConnection factory is created, the application can either provide those 2 threads or just let them be created internally. The calls to the Stream APIs and the PeerConnection APIs will be proxied to the signaling thread which means that the application can call those APIs from whatever thread. All callbacks will be made on the signaling thread. The application should return the callback as quickly as possible to avoid blocking the signaling thread. Resource intensive processes should be posted to a different thread. The worker thread is used to handle more resource intensive processes such as data streaming. Stream APIs (mediastream.h) Class MediaStreamTrackInterface This class declares an abstract interface to the media stream track which represents a media source in the user agent. class MediaStreamTrackInterface : public talk_base::RefCountInterface, public NotifierInterface { [b]public: enum TrackState { kInitializing, kLive = 1, kEnded = 2, kFailed = 3, }; virtual std::string kind() const = 0; virtual std::string label() const = 0; virtual bool enabled() const = 0; virtual TrackState state() const = 0; virtual bool set_enabled(bool enable) = 0; virtual bool set_state(TrackState new_state) = 0; }; MediaStreamTrackInterface::TrackState This enumerator is used to specify the type of the track states. Syntax enum TrackState { kInitializing, kLive = 1, kEnded = 2, kFailed = 3, }; Remarks kInitializing - Track is beeing negotiated. kLive - Track is alive. kEnded - Track has ended. kFailed - Track negotiation failed. MediaStreamTrackInterface::kind Returns the string "audio" if the track is an audio track, "video" if the track is a video track, or a user-agent defined string otherwise. Syntax virtual std::string kind() const = 0; MediaStreamTrackInterface::label Returns the label of the track, if any. If the track has no label, it returns the empty string. Syntax virtual std::string label() const = 0; MediaStreamTrackInterface::enabled Returns “true” if the track is enabled, “false” otherwise. Syntax virtual bool enabled() const = 0; MediaStreamTrackInterface::state Returns the current state of the track. Syntax virtual TrackState state() const = 0; MediaStreamTrackInterface::set_enabled Enable (true) or disable (false) a track. Syntax virtual bool set_enabled(bool enable) = 0; Remarks Not implemented. MediaStreamTrackInterface::set_state Set the track to a new state. Syntax virtual bool set_state(TrackState new_state) = 0; Remarks This method is used by the PeerConnection internally, the application should not call it directly. Class VideoTrackInterface The VideoTrackInterface derives from MediaStreamTrackInterface with two extra interfaces to get and set the video renderer. class VideoTrackInterface : public MediaStreamTrackInterface { public: virtual void SetRenderer(VideoRendererWrapperInterface* renderer) = 0; virtual VideoRendererWrapperInterface* GetRenderer() = 0; protected: virtual ~VideoTrackInterface() {} }; Class LocalVideoTrackInterface The LocalVideoTrackInterface derives from VideoTrackInterface with one extra interface to get the video capture associated with the track. class LocalVideoTrackInterface : public VideoTrackInterface { public: virtual cricket::VideoCapturer* GetVideoCapture() = 0; protected: virtual ~LocalVideoTrackInterface() {} }; Class AudioTrackInterface The AudioTrackInterface derives from MediaStreamTrackInterface and currently without extra interface. class AudioTrackInterface : public MediaStreamTrackInterface { public: protected: virtual ~AudioTrackInterface() {} }; Class LocalAudioTrackInterface The LocalAudioTrackInterface derives from AudioTrackInterface with one extra interface to get audio device associated with this track. class LocalAudioTrackInterface : public AudioTrackInterface { public: virtual AudioDeviceModule* GetAudioDevice() = 0; protected: virtual ~LocalAudioTrackInterface() {} }; Class cricket::VideoRenderer, cricket::VideoCapturer These classes are defined by the open source project libjingle and will not be described in detail here.. Class webrtc::AudioDeviceModule The AudioDeviceModule is defined by the open source project webrtc. Please refer to this link for a detailed definition. Class MediaStreamInterface This class declares an abstract interface to MediaStream which, typically but not necessarily, represents streams of media data of audio and/or video content. Each MediaStream object can contain zero or more tracks, in particular audio and video tracks. All tracks in a MediaStream are intended to be synchronized when rendered. Different MediaStreams do not need to be synchronized. class MediaStreamInterface : public talk_base::RefCountInterface, public NotifierInterface { public: virtual std::string label() const = 0; virtual AudioTracks* audio_tracks() = 0; virtual VideoTracks* video_tracks() = 0; enum ReadyState { kInitializing, kLive = 1, // Stream alive kEnded = 2, // Stream have ended }; virtual ReadyState ready_state() = 0; protected: virtual ~MediaStreamInterface() {} }; MediaStreamInterface::label Returns a label that is unique to this stream, so that streams can be recognized after they are sent through the PeerConnection APIs. Syntax virtual std::string label() const = 0; MediaStreamInterface::audio_tracks Returns a pointer to a list of objects of type MediaStreamTrack representing the audio tracks associated with this MediaStream. Syntax virtual AudioTracks* audio_tracks() = 0; MediaStreamInterface::video_tracks Returns a pointer to a list of objects of type MediaStreamTrack representing the video tracks associated with this MediaStream. Syntax virtual VideoTracks* video_tracks() = 0; MediaStreamInterface::ready_state Returns the current ready state of the MediaStream. Syntax virtual ReadyState ready_state() = 0; Class LocalMediaStreamInterface The LocalMediaStreamInterface derives from the MediaStreamInterface with extra interfaces to add media tracks to the MediaStream. class LocalMediaStreamInterface : public MediaStreamInterface { public: virtual bool AddTrack(AudioTrackInterface* track) = 0; virtual bool AddTrack(VideoTrackInterface* track) = 0; }; PeerConnection APIs (peerconnection.h) Class StreamCollectionInterface This class declares a MediaStream container interface. class StreamCollectionInterface : public talk_base::RefCountInterface { public: virtual size_t count() = 0; virtual MediaStreamInterface* at(size_t index) = 0; virtual MediaStreamInterface* find(const std::string& label) = 0; protected: ~StreamCollectionInterface() {} }; StreamCollectionInterface::count Returns the number of MediaStreams in the collection. Syntax size_t count() = 0; StreamCollectionInterface::at Returns a reference to the MediaStream at the position index in the collection. Syntax MediaStreamInterface* at(size_t index) = 0; Parameters index [in] Position of a MediaStream in the collection. StreamCollectionInterface::find Searches the collection for a MediaStream with given label and returns a reference to it if found, otherwise it returns NULL. Syntax MediaStreamInterface* find(const std::string& label) = 0; Parameters label [in] The label value to be searched for. Class PeerConnectionObserver This class declares an abstract interface for a user defined observer. It is up to the PeerConnection user to implement a derived class which implements the observer class. The observer is registered when the PeerConnection is created using the PeerConnectionFactoryInterface. class PeerConnectionObserver { public: enum StateType { kReadyState, kIceState, kSdpState, }; virtual void OnError() = 0; virtual void OnMessage(const std::string& msg) = 0; virtual void OnSignalingMessage(const std::string& msg) = 0; virtual void OnStateChange(StateType state_changed) = 0; virtual void OnAddStream(MediaStreamInterface* stream) = 0; virtual void OnRemoveStream(MediaStreamInterface* stream) = 0; protected: ~PeerConnectionObserver() {} }; PeerConnectionObserver::StateType This enumerator is used to specify the type of the state machines. Syntax enum StateType { kReadyState, kIceState, kSdpState, }; PeerConnectionObserver::OnError This method will be called when an error occurs in PeerConnection. Syntax void OnError() = 0; Remarks Not implemented. PeerConnectionObserver::OnMessage This method will be called when a text message is received from the remote peer. Syntax void OnMessage(const std::string& msg) = 0; PeerConnectionObserver::OnSignalingMessage This method is called once a signaling message is ready. Syntax void OnSignalingMessage(const std::string& msg) = 0; Parameters msg [in] A ROAP format signaling message. Remarks The user should send the signaling message from the callback to the remote peer. PeerConnectionObserver::OnStateChange This method is called when any of the 3 state machines (ReadyState, SdpState or IceState) has a state change. Syntax virtual void OnStateChange(StateType state_changed) = 0; Parameters state_changed [in] Specify which state machine’s state has changed. Remarks The IceState is not implemented. PeerConnectionObserver::OnAddStream This method is called when media is received on a new stream from remote peer. Syntax virtual void OnAddStream(MediaStreamInterface* stream) = 0; Parameters stream [in] The handler to the remote media stream. Remarks The user may use this event to set the renderer for the received media stream. PeerConnectionObserver::OnRemoveStream This method is called when the remote peer closes a stream. Syntax virtual void OnRemoveStream(MediaStreamInterface* stream) = 0; Parameters stream [in] The handler to the closed remote media stream. Class PortAllocatorFactoryInterface This class declares a factory interface for creating the cricket::PortAllocator, which is used for the ICE negotiation. The PeerConnection factory will use this interface (if supplied) to create the PortAllocator for PeerConnection. The application may provide its own implementation of the PortAllocator by implementing the PortAllocatorFactoryInterface - basically the CreatePortAllocator method. class PortAllocatorFactoryInterface : public talk_base::RefCountInterface { public: struct StunConfiguration { StunConfiguration(const std::string& address, int port) : server(address, port) {} talk_base::SocketAddress server; }; struct TurnConfiguration { TurnConfiguration(const std::string& address, int port, const std::string& user_name, const std::string& password) : server(address, port), username(username), password(password) {} talk_base::SocketAddress server; std::string username; std::string password; }; virtual cricket::PortAllocator* CreatePortAllocator( const std::vector<StunConfiguration>& stun_servers, const std::vector<TurnConfiguration>& turn_configurations) = 0; protected: PortAllocatorFactoryInterface() {} ~PortAllocatorFactoryInterface() {} }; PortAllocatorFactoryInterface::CreatePortAllocator This method returns an instance of the PortAllocator class. Syntax virtual cricket::PortAllocator* CreatePortAllocator( const std::vector<StunConfiguration>& stun_servers, const std::vector<TurnConfiguration>& turn_configurations) = 0; Parameters stun_servers [in] A configuration list of the STUN servers. turn_servers [in] A configuration list of the TURN servers. Remarks TURN is currently not implemented. Class PeerConnectionFactoryInterface The PeerConnectionFactoryInterface is the factory interface used for creating PeerConnection, media stream and media tracks. class PeerConnectionFactoryInterface : public talk_base::RefCountInterface { public: virtual talk_base::scoped_refptr<PeerConnectionInterface> CreatePeerConnection(const std::string& config, PeerConnectionObserver* observer) = 0; virtual talk_base::scoped_refptr<LocalMediaStreamInterface> CreateLocalMediaStream(const std::string& label) = 0; virtual talk_base::scoped_refptr<LocalVideoTrackInterface> CreateLocalVideoTrack(const std::string& label, cricket::VideoCapturer* video_device) = 0; virtual talk_base::scoped_refptr<LocalAudioTrackInterface> CreateLocalAudioTrack(const std::string& label, AudioDeviceModule* audio_device) = 0; protected: PeerConnectionFactoryInterface() {} ~PeerConnectionFactoryInterface() {} }; PeerConnectionFactoryInterface::CreatePeerConnection Create an instance of the PeerConnection. Syntax virtual talk_base::scoped_refptr<PeerConnectionInterface> CreatePeerConnection(const std::string& config, PeerConnectionObserver* observer) = 0; Parameters config [in] The configuration string gives the address of a STUN or TURN server to use to establish the connection. The format is defined in the webrtc-api. observer [in] A pointer to an instance of the PeerConnectionObserver derived class. Remarks TURN is currently not supported. The allowed formats for the config string are: "TYPE 203.0.113.2:3478" Indicates a specific IP address and port for the server. "TYPE relay.example.net:3478" Indicates a specific host and port for the server; the user agent will look up the IP address in DNS. "TYPE example.net" Indicates a specific domain for the server; the user agent will look up the IP address and port in DNS. The "TYPE" is one of: STUN Indicates a STUN server STUNS Indicates a STUN server that is to be contacted using a TLS session. TURN Indicates a TURN server TURNS Indicates a TURN server that is to be contacted using a TLS session. PeerConnectionFactoryInterface::CreateLocalMediaStream Create an instance of a local media stream. Syntax virtual talk_base::scoped_refptr<LocalMediaStreamInterface> CreateLocalMediaStream(const std::string& label) = 0; Parameters label [in] Desired local media stream label. PeerConnectionFactoryInterface::CreateLocalVideoTrack Create an instance of a local video track. Syntax virtual talk_base::scoped_refptr<LocalVideoTrackInterface> CreateLocalVideoTrack(const std::string& label, cricket::VideoCapturer* video_device) = 0; Parameters label [in] Desired local video track label. video_device [in] Pointer to the video capture device that is going to associate with this track. PeerConnectionFactoryInterface::CreateLocalAudioTrack Create an instance of a local audio track. Syntax virtual talk_base::scoped_refptr<LocalVideoTrackInterface> CreateLocalAudioTrack(const std::string& label, AudioDeviceModule* audio_device) = 0; Parameters label [in] Desired local audio track label. audio_device [in] Pointer to the audio device that is going to associate with this track. Function CreatePeerConnectionFactory Create a new instance of PeerConnectionFactoryInterface. Syntax talk_base::scoped_refptr<PeerConnectionFactoryInterface> CreatePeerConnectionFactory(); Remarks The PeerConnectionFactoryInterface instance generated by this function will create required resources internally - including libjingle threads, socket and network manager factory classes for networking. Function CreatePeerConnectionFactory Create a new instance of PeerConnectionFactoryInterface with the given libjingle threads and portallocator factory. Syntax talk_base::scoped_refptr<PeerConnectionFactoryInterface> CreatePeerConnectionFactory(talk_base::Thread* worker_thread, talk_base::Thread* signaling_thread, PortAllocatorFactoryInterface* factory, AudioDeviceModule* default_adm); Remarks This function can be used when the application wants to provide its own implementation of the threads and the portallocator. Ownership of the arguments are not transferred to this object and must remain in scope for the lifetime of the PeerConnectionFactoryInterface. Class PeerConnectionInterface class PeerConnectionInterface : public talk_base::RefCountInterface { public: enum ReadyState { kNew, kNegotiating, kActive, kClosing, kClosed, }; enum SdpState { kSdpNew, kSdpIdle, kSdpWaiting, }; virtual void ProcessSignalingMessage(const std::string& msg) = 0; virtual bool Send(const std::string& msg) = 0; virtual talk_base::scoped_refptr<StreamCollectionInterface> local_streams() = 0; virtual talk_base::scoped_refptr<StreamCollectionInterface> remote_streams() = 0; virtual void AddStream(LocalMediaStreamInterface* stream) = 0; virtual void RemoveStream(LocalMediaStreamInterface* stream) = 0; virtual void CommitStreamChanges() = 0; virtual void Close() = 0; virtual ReadyState ready_state() = 0; virtual SdpState sdp_state() = 0; protected: ~PeerConnectionInterface() {} }; PeerConnectionInterface::ReadyState This enumerator is used to specify the type of the ready state. kNew - The object was just created and its ICE and SDP Agent have not yet been started. kNegotiating - The peerConenction object is attempting to get to the point where media can flow. kActive - A connection has been formed and if any media streams were successfully negotiated, any relevant media can be streaming. kClosing - The object is starting to shut down after the close() method has been invoked. kClosed - The shutting down is completed. Syntax enum ReadyState { kNew, kNegotiating, kActive, kClosing, kClosed, }; PeerConnectionInterface::SdpState This enumerator is used to specify the type of the SDP state. kSdpNew - The object was just created and SDP Agent have not yet been started. kSdpIdle - A valid offer answer pair has been exchanged and the SDP Agent is waiting for the next SDP transaction. kSdpWaiting - The SDP Agent has sent an SDP offer and is waiting for a response. Syntax enum SdpState { kSdpNew, // TODO(ronghuawu): kSdpNew is not defined in the spec. kSdpIdle, kSdpWaiting, }; PeerConnectionInterface::ProcessSignalingMessage Handle the signaling message from the remote peer. Syntax virtual void ProcessSignalingMessage(const std::string& msg) = 0; Parameters msg [in] A ROAP format signaling message. Remarks The order of messages is important. Passing messages to the PeerConnection in a different order than they were generated by the remote peer can prevent a successful connection from being established or degrade the connection's quality if one is established. PeerConnectionInterface::Send Send a text message to the remote peer over the data stream. Syntax virtual bool Send(const std::string& msg) = 0; // TODO(ronghuawu): This is not defined in the spec. Parameters msg [in] The text message to be sent to the remote peer. Remarks Not currently implemented. PeerConnectionInterface::local_streams Returns a live array containing the streams that the user agent is currently attempting to transmit to the remote peer (those that were added with A ddStream()). Syntax virtual talk_base::scoped_refptr<StreamCollectionInterface> local_streams() = 0; PeerConnectionInterface::remote_streams Returns a live array containing the streams that the user agent is currently receiving from the remote peer. This array is updated when OnAddStream and OnRemoveStream callbacks are fired. Syntax virtual talk_base::scoped_refptr<StreamCollectionInterface> remote_streams() = 0; PeerConnectionInterface::AddStream Add a local stream to the array of the streams that the user agent is currently attempting to transmit to the remote peer. This function only adds the stream to the array and does not trigger any changes until CommitStreamChanges is called. Syntax virtual void AddStream(LocalMediaStreamInterface* stream) = 0; Parameters stream [in] Pointer to the local media stream to be added. PeerConnectionInterface::RemoveStream Remove a local stream from the array of the streams that the user agent is currently attempting to transmit to the remote peer. This function only removes the stream from the array and does not trigger any changes until CommitStreamChanges is called. Syntax virtual void RemoveStream(LocalMediaStreamInterface* stream) = 0; Parameters stream [in] Pointer to the local media stream to be removed. PeerConnectionInterface::CommitStreamChanges Commit Stream changes made by the AddStream and RemoveStream. This will start sending media on new added streams and stop sending media on removed streams. Syntax virtual void CommitStreamChanges() = 0; PeerConnectionInterface::Close Close the current session. This will trigger a Shutdown message to be sent and the ready state will change to kClosing. Syntax virtual void Close() = 0; PeerConnectionInterface::ready_state Return the PeerConnection object's readiness state, represented by enum ReadyState. Syntax virtual ReadyState ready_state() = 0; PeerConnectionInterface::sdp_state Return the state of the PeerConnection SDP Agent, represented by enum SdpState. Syntax virtual SdpState sdp_state() = 0; Reference The current HTML5 specification for WebRTC is here: http://dev.w3.org/2011/webrtc/editor/webrtc.html The source code of the WebRTC Native API is here: https://code.google.com/p/libjingle/source/browse/trunk/talk/app/webrtc/ Client and server sample apps can be found here: https://code.google.com/p/libjingle/source/browse/trunk/talk/#talk%2Fexamples%2Fpeerconnection 原文链接：http://www.webrtc.org/reference/native-apis

[/b]