Android Opencore Framework

Sasken Confidential

2007 Sasken Communication Technologies

In this session
Media Player Architecture
 Media player engine  State machine  Usage scenario command flow graphs

Media Capture Architecture

 Media capture engine  State machine  Usage scenario command flow graphs

PVMF Node Architecture

 Nodes - Media player & Media capture scenarios  Node interface class  MIO Node

2007 Sasken Communication Technologies

Media player architecture

APPLICATIONS Media Player Application UI APPLICATION FRAMEWORK Media Player Engine Wrapper NATIVE LIBRARIES Media Player Engine JNI Player Engine Interfaces PV Media Player Engine Data Source File Recognizer Streaming Node RTSP, RTP, RTCP payloads Format Parser Node Format Parsers PV OMX AV Codec Node OMX Core OMX Codec Component Data Sink AV Media Output Node Surface Flinger Audio Flinger

PV OSCL LINUX KERNEL Display Driver Audio Driver

2007 Sasken Communication Technologies

Media player architecture

Media Player Engine JNI
 Exposes the media player APIs in the native layer to application layer

(Contd)

Media Player Engine

 Establishes the data path for media decoding & rendering  Maintains state machine  Handles events and information generated during multimedia playback operations.

Data Source
 Implements file parser nodes, streaming node & file recognizer

Data Sink
 Implements decoder nodes, Media output nodes  Sends the decoded data to rendering devices  Takes care of AV synchronization

2007 Sasken Communication Technologies

Media player engine

2007 Sasken Communication Technologies

Media player engine

(Contd)

PVPlayerFactory handles the instantiation and destruction of PVPlayerEngine PVPlayerInterface provides all player APIs
 Playback specific APIs like Start, Stop, Seek  Query and retrieve any extension interfaces

PVCommandStatusObserver, PVInformationalObserver, and PVErrorEventObserver are the callback handlers

 Notifies the application of any asynchronous command completion,unsolicited error and informational events

2007 Sasken Communication Technologies

Media player engine State machine

2007 Sasken Communication Technologies

Media player engine Usage Scenarios

player_usage_scen arios

2007 Sasken Communication Technologies

In this session
Media Player Architecture
 Media player engine  State machine  Usage scenario command flow graphs

Media Capture Architecture

 Media capture engine  State machine  Usage scenario command flow graphs

PVMF Node Architecture

 Nodes - Media player & Media capture scenarios  Node interface class  MIO Node

2007 Sasken Communication Technologies

Media capture architecture

APPLICATIONS Media Capture Application UI APPLICATION FRAMEWORK Media Author Engine Wrapper NATIVE LIBRARIES Media Author Engine JNI Author Engine Interfaces PV Media Author Engine Data Sink Format Composer Node Format Composers PV OMX AV Codec Node OMX Core OMX Codec Component Data Source AV Media Input Node

PV OSCL LINUX KERNEL Camera Driver Audio Driver Display Driver

2007 Sasken Communication Technologies

10

Media capture architecture

Media Author Engine JNI
 Exposes the media author APIs in the native layer to application layer

(Contd)

Media Author Engine

 Establishes the data path for media capture & encoding  Maintains state machine  Handles events and information generated during multimedia record operations

Data Source
 Implements audio/video media input node  Passes the captured raw media data to encoder nodes

Data Sink
 Implements encoder nodes, format composer nodes  Writes the encoded media data into file

2007 Sasken Communication Technologies

11

Media capture engine

2007 Sasken Communication Technologies

12

Media capture engine

PVAuthorEngneFactory handles the instantiation and destruction of PVAuthorEngine PVAuthorEngineInterface provides all engine APIs
 Recording specific APIs like Record, Pause, Stop  Query and retrieve any extension interfaces

(Contd)

PVCommandStatusObserver, PVInformationalObserver, and PVErrorEventObserver are the callback handlers

 Notifies the application of any asynchronous command completion,unsolicited error and informational events

2007 Sasken Communication Technologies

13

Media capture engine State machine

2007 Sasken Communication Technologies

14

Media capture engine Usage Scenarios

capture_usage_sce narios

2007 Sasken Communication Technologies

15

In this session
Media Player Architecture
 Media player engine  State machine  Usage scenario command flow graphs

Media Capture Architecture

 Media capture engine  State machine  Usage scenario command flow graphs

PVMF Node Architecture

 Nodes - Media player & Media capture scenarios  Node interface class  MIO Node

2007 Sasken Communication Technologies

16

PVMF Nodes Media player scenario

NATIVE LIBRARIES PV Player Node Registry PV Media Player Engine PV Node Command Status Observer PV Node Error Event Observer PV Player Data path PV Player Recognizer Registry PVMF Recognizer Registry File Recognizer Format Parser Node PV Node Info Event Observer

PV OMX Audio Decode Node MIO Audio Output Node

PV OMX Video Decode Node

Format Parser

MIO Video Output Node

Audio Flinger

Surface Flinger

LINUX KERNEL Audio Driver Display Driver

2007 Sasken Communication Technologies

17

PVMF Nodes Media player scenario

Media Data Flow

2007 Sasken Communication Technologies

18

PVMF Nodes Media capture scenario

NATIVE LIBRARIES PV Player Node Registry PV Author Engine PV Author Engine Node Utility Observer PV Node Command Status Observer PV Node Error Event Observer

PV Author Engine Node Utility

PV Node Info Event Observer Format Composer Node PV OMX Audio Encode Node MIO Audio input node PV OMX Video Encode Node

Format Composer

MIO Video input node

Android Audio Input LINUX KERNEL Audio Driver

Android Camera Input

Camera Driver

2007 Sasken Communication Technologies

19

PVMF Node Interface class -PVMFNodeInterface

Init() GetCapability() GetPorts () RequestPort () Prepare() Start() Pause() Stop() Reset() ReleasePort () CancelCommand() CancelAllCommands() Initializes the node. The node should be ready to returns its capabilities once init is complete. Retrieves the capability of the node used Returns all available ports for data flow Returns an already created unused port or dynamically creates one The node should be able to start after this Starts servicing the connected ports Pauses servicing all connected ports and retains the the unprocessed data Stops servicing and discards unprocessed data Resets the node Releases a connected port Cancels a particular command which is pending and yet to be processed Cancels all the commands to be processed and also aborts the command which is currently processing

2007 Sasken Communication Technologies

20

PVMF Node Interface Observer Classes

PVMFNodeCmdStatusObserver
 Notifies the status of issued command messages.

PVMFNodeInfoEventObserver
 Notifies the informational messages to the client

PVMFNodeErrorEventObserver
 Notifies the unsolicited error events back to the user

2007 Sasken Communication Technologies

21

Media Input-Output Node Playback scenario

2007 Sasken Communication Technologies

22

Media Input-Output Node Capture scenario

2007 Sasken Communication Technologies

23

MIO Node Interface Classes

PvmiMIOControl
 Allows other PV modules to issue control commands to MIO component

PvmiCapabilityAndConfig
 Facilitates capability and configuration exchange between other modules and MIO

PvmiMediaTransfer
 Facilitate to exchange media data  Also allows in-data commands transaction e.g end of data notification

PvmiClockExtensionInterface
 Allows other modules to provide a clock object to MIO component, to enable media synchronization during rendering

2007 Sasken Communication Technologies

24

MIO Node Sequence Diagram Media player

mio_player

2007 Sasken Communication Technologies

25

Thank You

Sasken Confidential

2007 Sasken Communication Technologies