SYSTEM AND METHOD FOR SOCIAL VIEWING OF AUDIO/VIDEO STREAMS Embodiments of the invention provide software-based schemes for

aggregating and synchronizing the viewing of audio/video streams from various sources in a multi-user networked 5 environment that functions as a virtual living room. Such schemes permit a plurality of users at a plurality of distributed geographic locations to listen to the same audio stream and/or watch the same video stream while the users exchange communications with one another in real time, thereby simulating the experience of the plurality of users being at the same physical location. While the following description characterizes embodiments of the invention as permitting users to 10 share the same video streams, it should be recognized that the present invention is also applicable to the sharing of other media streams, including audio streams. In this regard, the term video, as used herein, should be understood to include streamed audio content, streamed video content, and/or a combination of streamed audio content and streamed video content, in various embodiments of the present invention. FIG. 1 shows an exemplary screen view of a user interface in a system consistent with one 15 embodiment of the invention. As shown, the user can select video and/or audio content to view from a variety of sources, including, e.g., Netflix, Ustream, YouTube, Vimeo, and Sirius XM, by selecting from one of a plurality of tabs. At right, a window has tabs labeled chat, activity, and invites. The chat selection permits users to communicate with one another while watching video streams. The activity selection permits users to see what other users are viewing. The invites selection permits 20 users to receive invitations from other users to share the other users video streams. Embodiments of the invention permit a users live video-streaming session to be broadcast to the users friends via a real-time feed. A user runs a web-based software program that provides the user with (i) choices of his or her own video stream to view, and (ii) a real-time catalog showing the video stream that each of the users friends are currently viewing, as well as a list of friends of that friend who are 25 viewing the friends video stream. The user can either select his or own video stream to view or select a friends video stream from the catalog to view. Each video stream in the catalog could be represented by hyperlinks presented using various indicia including, e.g., one or more of a live video feed, a still-screen view, a title-screen view, a text hyperlink, and the like. The catalog shows the user the relevant parts of the users friends social viewing profiles 30 including the indicia to indicate in real-time which video streams those friends are currently viewing and the hyperlinks to select those video streams. In some embodiments, only a users friends can view and select to join the users video session, and in other embodiments, any member of the public can view and select to join those video sessions as well.

1234.001PROV

 For example, if a users friend is watching a television show, then identification of that show will appear in the catalog as being the currently-watched video stream. The user can click to select the friends video stream to join and watch the show with the users friend in the friends virtual living room. In certain embodiments, notifications such as RSS feeds are used to inform a user about the video 5 streams the users friends are currently viewing. Such notifications can be limited to only video streams being viewed by more than one person or video streams that have been viewed for longer than a threshold period of time, to avoid the user being inundated with the users friends frivolous personal searches or channel-surfing activity. An exemplary RSS notification might indicate: Join Kathleen Gilbert and two other friends in watching Sugar Rush Mountain II. 10 Instead of being directly invited, a user can also join by showing interest either in the video or in the specific individual by entering the individuals virtual living room, i.e., viewing session. When a user joins their friends virtual living room, content begins to stream directly to the users media player. However, in some embodiments, the content that the users friend is sharing with the user will not be added to the activity stream. However, once the user invites a third-party friend to join and the third-party 15 friend responds, then everyone who is viewing the users activity feed will be able to join in watching the video stream as well, e.g., by clicking the word Join in the RSS notification. In certain embodiments, when clicking on the title of a particular video selection in the RSS notification (e.g., Sugar Rush Mountain II), the user will not enter the session with the individual who was listed on the activity feed, but instead, the video selection will be streamed only to the users own media player. In certain 20 embodiments, by clicking the friends name in the RSS notification (e.g., Kathleen Gilbert), the user is taken to the friends FaceBook or other social-media page. In certain embodiments, when clicking on the number of other friends in the RSS notification (e.g., two), the identity of those friends is displayed to the user. To join a friends viewing session, there is no need for a user to copy and paste a URL. The user 25 can invite friends from his or her social networks, and once the friends join, their social-viewing session is then listed as an activity for all of his or her online friends to see, including their status, or alternatively, e.g., via a Twitter feed. Once a user has been invited to a friends video session, the user can not only join the session, but also see what the friend is watching and with whom. 30 Interface Overview In one embodiment, the system employs two distinct user interfaces and roles. The first role is that of owner. His user interface consists of a video player, video controls (pause, play, etc.), a capability for selecting the video to watch and a chat window. The owner uses this interface as one would expect a

1234.001PROV

 host to: he can choose any video to watch, pause and play the video, and perform other common video activities. The second role is that of guest. The guest interface is simpler than that of the owner. It consists of a video player and a chat window. The video player has no controls, nor any way to select a video. 5 Instead, the video player on the guests screen remains in sync with the video selection of the owner. As the owner plays, pauses and selects videos, these changes immediately display on the guest's interface. The owner and all guest users share a common chat window. This allows for discussion, much like might happen in a living room. 10 Server-Side Environment A centralized database store maintains a table of all Rooms in use by owners and guests. The following exemplary data is tracked for a room:

Attribute Id

Description The unique ID for the room

Example 45 8A6BEF 9B863C youtube|vSYadh2xmcI|playing|34.4

owner_slug A globally unique, unguessable string used by an owner to identify a room guest_slug A globally unique, unguessable string used by a guest to identify a room

player_state A description of what the owner is currently viewing on his media player 15

The owner and guest slug values allow for browser URLs and JavaScript code to easily access the room's data storage while doing so only through a specific role. In other words, if the guest slug is known (the example above being: 9B863C), then the current player_state can be read, yet the system will trivially be able to reject any attempt at setting this state. The player_state is a string composed of the following values:

20

Provider: the video provider, in the above case, YouTube.com Video ID: the unique identifier that specifies the video in the provider's context. Player status: this value is either playing or paused and describes the current mode of the video player. Timestamp: the time stamp specifies how many seconds the owner is into playing the video

25

1234.001PROV

 By combining and tracking these 4 values, a system consistent with embodiments of the invention can synchronize videos from a wide variety of video providers. Along with the shared storage, the server provides a facility by which owners and guests can regularly interact with the server. This is known as pinging. 5

In the case of the owner, a ping submits the following data via an HTTP POST: owner_slug player_state That is, the owner regularly submits a unique (and private) identifier back to a room, along with

10

the current state of the video the user has achieved (e.g., user has paused a video at 3 minutes and 10 seconds of video X). The server responds with an OK message, implying that the ping was received. The guest submits a ping request via HTTP POST that consists only of a guest_slug value. In response, the server provides the current player_state value associated with slug. The regular pinging performed by owners and guests is what allow the system to remain in sync.

15 Client-Side Environment In addition to the server side component, there is an owner and guest user interface that leverages JavaScript. The owner interface displays, among other things, a video player that the user can interact with. 20 Behind the scenes, the interface creates a JavaScript variable to store the current state of the video player, and sets up a background thread to publish this variable using the owner's ping method described above. The ping functionality should be invoked approximately every 200 milliseconds. As the user interacts with the video player change, the JavaScript variable representing the state of the player should be updated. This state will then be broadcast to the server by means of the ping 25 method. The guest interface displays a video player, but it lacks any controls. It, too, sets up a JavaScript variable to track the state of the player and a background thread to invoke the relevant ping method. It too, tracks the current state of the player. The ping method should be invoked approximately every 200 milliseconds. When the owner's state is received via the ping, it is compared to the current player state. If 30 it differs, the changes found are applied to the current player, and the received state becomes the new state of the guest player. Consider the following example:

1234.001PROV

Owner's Current State A provider: none video: none status: none time: none

Owner's Action

Guest's Current State provider: none video: none status: none time: none provider: none video: none status: none time: none provider: youtube video: abc status: playing time: 4.2 ... provider: youtube video: abc status: playing time: 29.2

Guest's Received State provider: none video: none status: none time: none provider: youtube video: abc status: playing time: 3.3 provider: youtube video: abc status: playing time: 4.2 ... provider: youtube video: abc status: paused time: 31.1

Owner opened up a blank page

Owner started viewing B provider: youtube video abc and is 3.3 video: abc seconds into it status: playing time: 3.3 Owner continues to allow C provider: youtube the video to play video: abc status: playing time: 4.2 D ... E provider: youtube video: abc status: paused time: 32.1 ... The Owner paused the video 32.1 seconds into it

In this case, the owner started watching a movie [A], and the guest was able to follow along. In rows [A] and [B] the state of the player the guest receives via the ping is different than its local state, 5 therefore, an update to the player is necessary. In row [C] both the owner and guest are in sync, therefore, the guest receiving state from the ping invocation does not cause the player to be modified. One of the advantages of the approach defined above is that if at any time the guest falls out of sync with the owner, the background ping will nudge it back into sync. For example, consider the guest temporarily losing connectivity. When connectivity returns, the current state of the owner will be received 10 via the ping functionality, and will differ from the guest's current (and now stale) state. The guest will force its player to reflect this new state, and the users will be back in sync. One area of the synchronization that does involve special handling is the time component of the player state. Because of the delay involved in the owner posting its state to the server, and the guest pulling this state in, it is quite possible that the time will regularly be off by a second or two. Forcing the 5
1234.001PROV

 guest's player to match the time exactly would mean that the player would regularly be skipping ahead by one to two seconds, resulting in an unacceptable video playing experience. To address this, the time component isn't required to match exactly, but instead, is maintained within a particular threshold. The default threshold is 3 seconds. This threshold allows for relatively close synchronized viewing, but 5 doesn't put an impossible burden on the guest's browser to keep in perfect sync with the owner. Additional features in both the owner and guest user interface can be added, such as the chat window described above. These features are not directly related the video synchronization scheme, though they can leverage either the owner_slug or guest_slug as needed. 10 Social Viewing Store A feature that might be included in a system consistent with embodiments of the invention is a social viewing store. In this scenario, points (or other virtual currency) are awarded for various events, including, e.g., awarding points to a user every time a friend joins a users virtual living room, or 15 awarding points to a user for every 15 minutes of video shared with friends (either based on viewing time 35 alone or based also on the number of friends with whom the video is being shared). The points can be used, e.g., to purchase themes, wallpaper, consoles (content), and pay-per-view consoles. This feature incentivizes sharing video in exchange for receiving points to design ones virtual living room, including adding exclusive consoles (content) that the users friends may not have. However, the user can allow friends from the users social network to access such consoles while online. 20 Advertising Revenue By identifying each individual viewers IP address, a system consistent with certain embodiments of the invention permits a single video stream to generate revenue based on multiple viewers who are watching a given video stream or advertisement. In certain embodiments, a user has the choice to 25 upgrade his or her account, e.g., by paying monthly subscription fees, to eliminate advertisements altogether. In this case, subscription fees could be calculated based on the average amount of advertising that a frequent user watches. To calculate the subscription rate, the users prior month of ad exposure can be examined. For example, the number of hours of shows viewed can be multiplied by six, and accounting for any console switch undertaken by the user, the product can further be multiplied by a 30 predetermined primetime advertisement cost per view, e.g., $0.20. The result is the cost that the user should have to pay for the privilege of having the subsequent month be free of ads. The subscription rate can change monthly on such a basis. For example, a user watching 15 hours of video a month, or 90 advertisements, might pay $1.80 for a commercial-free month. Basically, the user is pre-paying for their advertisements that the user will not have to watch. In some embodiments, all content viewed with a 6
1234.001PROV

Kevin M. Drucker 6/9/11 1:49 PM

Deleted: a user invites

Kevin M. Drucker 6/9/11 1:49 PM

Deleted: to the

Kevin M. Drucker 6/9/11 1:49 PM

Deleted: streaming session

 users friends is considered when calculating the fee. For example, if a user plans on watching content with a specified number of friends, the individual fee could be multiplied by the number of other users. Average virtual living-room size can be used as a proxy for the number of other users. In this manner, all parties or friends invited into the users living room would also have ad-free content. If a user does not 5 specify that he or she will watch with friends, then the user will be shown advertisements while viewing videos with friends. User-to-User Chat Functionality While sharing a video stream, appropriate functionality can be provided so that the virtual living 10 room resembles an actual living room, where users can communicate with one another while watching the same video stream. Such communications can be via a chat or instant message-type window, or alternatively, via voice-over-IP and/or videoconferencing communications implemented in a system consistent with embodiments of the invention. 15 Alternative Embodiments It should be understood that various changes in the details, materials, and arrangements of the parts which have been described and illustrated in order to explain the nature of this invention may be made by those skilled in the art without departing from the scope of the invention. For example, it should be understood that the inventive concepts of embodiments of the invention may be applied not only in 20 systems for detecting fraud in medical documents, but also for other image-comparison and documentcomparison purposes. The present invention can be embodied in the form of methods and apparatuses for practicing those methods. The present invention can also be embodied in the form of program code embodied in tangible media, such as magnetic recording media, optical recording media, solid state memory, floppy 25 diskettes, CD-ROMs, hard drives, or any other non-transitory machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing embodiments of the invention. The present invention can also be embodied in the form of program code, for example, stored in a non-transitory machine-readable storage medium including being loaded into and/or executed by a machine, wherein, when the program 30 code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing embodiments of the invention. When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits.

1234.001PROV

 It will be appreciated by those skilled in the art that although the functional components of the exemplary embodiments of the system of the present invention described herein may be embodied as one or more distributed computer program processes, data structures, dictionaries and/or other stored data on one or more conventional general-purpose computers (e.g., IBM-compatible, Apple Macintosh, and/or 5 RISC microprocessor-based computers), mainframes, minicomputers, conventional telecommunications (e.g., modem, T1, fiber-optic line, DSL, satellite and/or ISDN communications), memory storage means (e.g., RAM, ROM) and storage devices (e.g., computer-readable memory, disk array, direct access storage) networked together by conventional network hardware and software (e.g., LAN/WAN network backbone systems and/or Internet), other types of computers and network resources may be used without 10 departing from the present invention. One or more networks discussed herein may be a local area network, wide area network, internet, intranet, extranet, proprietary network, virtual private network, a TCP/IP-based network, a wireless network (e.g., IEEE 802.11 or Bluetooth), an e-mail based network of e-mail transmitters and receivers, a modem-based, cellular, or mobile telephonic network, an interactive telephonic network accessible to users by telephone, or a combination of one or more of the foregoing. 15 Embodiments of the invention as described herein may be implemented in one or more computers residing on a network transaction server system, and input/output access to embodiments of the invention may include appropriate hardware and software (e.g., personal and/or mainframe computers provisioned with Internet wide area network communications hardware and software (e.g., CQI-based, FTP, Netscape NavigatorTM, Mozilla FirefoxTM, Microsoft Internet ExplorerTM, or Apple SafariTM HTML Internet20 browser software, and/or direct real-time or near-real-time TCP/IP interfaces accessing real-time TCP/IP sockets) for permitting human users to send and receive data, or to allow unattended execution of various operations of embodiments of the invention, in real-time and/or batch-type transactions. Likewise, the system of the present invention may include one or more remote Internet-based servers accessible through conventional communications channels (e.g., conventional telecommunications, broadband 25 communications, wireless communications) using conventional browser software (e.g., Netscape NavigatorTM, Mozilla FirefoxTM, Microsoft Internet ExplorerTM, or Apple SafariTM). Thus, the present invention may be appropriately adapted to include such communication functionality and Internet browsing ability. Additionally, those skilled in the art will recognize that the various components of the server system of the present invention may be remote from one another, and may further include 30 appropriate communications hardware/software and/or LAN/WAN hardware and/or software to accomplish the functionality herein described. Each of the functional components of the present invention may be embodied as one or more distributed computer-program processes running on one or more conventional general purpose computers networked together by conventional networking hardware and software. Each of these functional 8
1234.001PROV

 components may be embodied by running distributed computer-program processes (e.g., generated using full-scale relational database engines such as IBM DB2TM, Microsoft SQL ServerTM, Sybase SQL ServerTM, or Oracle 10gTM database managers, and/or a JDBC interface to link to such databases) on networked computer systems (e.g., including mainframe and/or symmetrically or massively-parallel 5 computing systems such as the IBM SB2TM or HP 9000TM computer systems) including appropriate mass storage, networking, and other hardware and software for permitting these functional components to achieve the stated function. These computer systems may be geographically distributed and connected together via appropriate wide- and local-area network hardware and software. In one embodiment, data stored in the database or other program data may be made accessible to the user via standard SQL queries 10 for analysis and reporting purposes. Primary elements of embodiments of the invention may be server-based and may reside on hardware supporting an operating system such as Microsoft Windows NT/2000TM or UNIX. Components of a system consistent with embodiments of the invention may include mobile and non-mobile devices. Mobile devices that may be employed in the present invention include personal 15 digital assistant (PDA) style computers, e.g., as manufactured by Apple Computer, Inc. of Cupertino, California, or Palm, Inc., of Santa Clara, California, and other computers running the Android, Symbian, RIM Blackberry, Palm webOS, or iPhone operating systems, Windows CETM handheld computers, or other handheld computers (possibly including a wireless modem), as well as wireless, cellular, or mobile telephones (including GSM phones, J2ME and WAP-enabled phones, Internet-enabled phones and data20 capable smart phones), one- and two-way paging and messaging devices, laptop computers, etc. Other telephonic network technologies that may be used as potential service channels in a system consistent with embodiments of the invention include 2.5G cellular network technologies such as GPRS and EDGE, as well as 3G technologies such as CDMA1xRTT and WCDMA2000, and 4G technologies. Although mobile devices may be used in embodiments of the invention, non-mobile communications devices are 25 also contemplated by embodiments of the invention, including personal computers, Internet appliances, set-top boxes, landline telephones, etc. Clients may also include a PC that supports Apple Macintosh TM, Microsoft Windows 95/98/NT/ME/CE/2000/XP/Vista/7TM, a UNIX Motif workstation platform, or other computer capable of TCP/IP or other network-based interaction. In one embodiment, no software other than a web browser may be required on the client platform. 30 Alternatively, the aforesaid functional components may be embodied by a plurality of separate computer processes (e.g., generated via dBaseTM, XbaseTM, MS AccessTM or other flat file type database management systems or products) running on IBM-type, Intel PentiumTM or RISC microprocessor-based personal computers networked together via conventional networking hardware and software and including such other additional conventional hardware and software as may be necessary to permit these 9
1234.001PROV

 functional components to achieve the stated functionalities. In this alternative configuration, since such personal computers typically may be unable to run full-scale relational database engines of the types presented above, a non-relational flat file table (not shown) may be included in at least one of the networked personal computers to represent at least portions of data stored by a system according to the 5 present invention. These personal computers may run the Unix, Microsoft Windows NT/2000TM or Windows 95/98/NT/ME/CE/2000/XP/Vista/7TM operating systems. The aforesaid functional components of a system according to the present invention may also include a combination of the above two configurations (e.g., by computer program processes running on a combination of personal computers, RISC systems, mainframes, symmetric or parallel computer systems, and/or other appropriate hardware 10 and software, networked together via appropriate wide- and local-area network hardware and software). A system according to the present invention may also be part of a larger system including multidatabase or multi-computer systems or warehouses wherein other data types, processing systems (e.g., transaction, financial, administrative, statistical, data extracting and auditing, data transmission/reception, and/or accounting support and service systems), and/or storage methodologies may be used in 15 conjunction with those of the present invention to achieve additional functionality (e.g., as part of a system operated by a content provider such as a cable television operator). In one embodiment, source code may be written in an object-oriented programming language using relational databases. Such an embodiment may include the use of programming languages such as C++ and toolsets such as Microsofts .NetTM framework. Other programming languages that may be used 20 in constructing a system according to the present invention include Java, HTML, Perl, UNIX shell scripting, assembly language, Fortran, Pascal, Visual Basic, and QuickBasic. Those skilled in the art will recognize that the present invention may be implemented in hardware, software, or a combination of hardware and software. Accordingly, the terms computer or system, as used herein, should be understood to mean a 25 combination of hardware and software components including at least one machine having a processor with appropriate instructions for controlling the processor. The terms computer or system can be used to refer to more than a single computing device, e.g., multiple personal computers, or one or more personal computers in conjunction with one or more other devices, such as a router, hub, packetinspection appliance, firewall, etc. 30 It should also be appreciated from the outset that one or more of the functional components may alternatively be constructed out of custom, dedicated electronic hardware and/or software, without departing from the present invention. Thus, the present invention is intended to cover all such

10

1234.001PROV

 alternatives, modifications, and equivalents as may be included within the spirit and broad scope of the invention. It should be understood that the steps of the exemplary methods set forth herein are not necessarily required to be performed in the order described, and the order of the steps of such methods 5 should be understood to be merely exemplary. Likewise, additional steps may be included in such methods, and certain steps may be omitted or combined, in methods consistent with various embodiments of the present invention. Reference herein to one embodiment or an embodiment means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one 10 embodiment of the invention. The appearances of the phrase in one embodiment in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments. Although the invention has been set forth in terms of the exemplary embodiments described herein and illustrated in the attached documents, it is to be understood that such disclosure is purely 15 illustrative and is not to be interpreted as limiting. Consequently, various alterations, modifications, and/or alternative embodiments and applications may be suggested to those skilled in the art after having read this disclosure. Accordingly, it is intended that the invention be interpreted as encompassing all alterations, modifications, or alternative embodiments and applications as fall within the true spirit and scope of this disclosure.

11

1234.001PROV

FIG. 1
12
1234.001PROV