Genn.

ai ()
Attached realtime marketing platform

Who am I?
Takeshi NAKANO
Senior Researcher / Architect. Co-authered Getting started with Solr in Japanese. Co-authered Hadoop Hacks in Japanese. Writing Getting started with Kafka in Japanese.

http://genn.ai/ 2

Company Name

Recruit Holdings Co., Ltd

Founded

March 31, 1960 (incorporated: August 26, 1963 governance changed from Recruit Co., Ltd since October 1st, 2012 Tokyo, Japan

Headquarters

Number of Group Employees 22,368 as of the end of March,2012 Number of Group Companies 102 Consolidated Subsidiaries/as of the end of June,2012 Capital 3,002.64 million yen since March 1st, 1995 806,661 million yen from April 1st 2011 to March 31st 2012 117,617 million yen from April 1st 2011 to March 31st 2012)

Consolidated Sales

Consolidated Ordinary Income

http://genn.ai/ 3

Operating Companies

Domain

Recruit Career Co., Ltd

Recruit Jobs Co., Ltd

Recruit Holdings Headquarters Function Global Business Business R&D

Recruit Sumai Company Ltd

Recruit Marketing Partners Co., Ltd

Recruit Lifestyle Co., Ltd

Recruit Stafng Co., Ltd

STAFF SERVICE HOLDINGS CO., LTD

Recruit Administration Co., Ltd Recruit Technologies Co., Ltd

Recruit Communications Co., Ltd

http://genn.ai/ 4

What is our challenge?

Clients can get the information about their customers

checking their items on our web sites. Providing a printed report monthly. Send them more detailed report in real time.
In real time fashion, the value of the information will be

dramatically increased!

http://genn.ai/ 5

Agenda
Genn.ai core details What is genn.ai?
Overview How to gather clicks 0) Injecting our javascripts Structure 1) Setting up lters 2) Filtering customers 3) Visualizing them

Future
Dealing with historical streams Genn.ai wants friends.

http://genn.ai/ 6

Overview of using Genn.ai

Customer A

Web Sites

Customer B

Customer C

Marketer Analyst

Genn.ai Realtime Analysis Platform

Client (Hotels)

http://genn.ai/ 7

Demo videos!
They are based on a hotel reservation site like
Expedia.com.

An animation and a screencapture what the

marketting sta see using Genn.ai.
At this moment ..

What prefectures are all my users searching for? (0:26) Comparing the number of searches in today s and
yesterday s regarding places.
(2:50)

http://genn.ai/ 8

Overview of using Genn.ai

Customer A

Web Sites

Customer B

Customer C

Marketer Analyst

Genn.ai Realtime Analysis Platform

Client (Hotels)

http://genn.ai/ 9

Overview of using Genn.ai

Customer A

Web Sites

Customer B

Customer C

Marketer Analyst

Genn.ai Realtime Analysis Platform

Client (Hotels)

http://genn.ai/ 10

How to gain access to Genn.ai

Genn.ai will be able to be attached!

Customer A

(ex.Tomcat)

App

Apache

Customer B

Customer C

http://genn.ai/ 11

How to gain access to Genn.ai

Genn.ai will be able to be attached!
A custom apache module for embeding our

code on returning html sources from App.

(ex.Tomcat)

App

Apache

Customer B

mod

Genn.ai
http://genn.ai/ 12

How to gain access to Genn.ai

The embedded values and js on a html.

http://genn.ai/ 13

Structure

http://genn.ai/ 14

Steps for getting results

There are 3 steps! (after getting clicks)

0
gather clicks

3
Visualizing them

Pouring clicks into Kafka

Recording them as behaviors

http://genn.ai/ 15

Structure (Pouring into Kafka)

Query
genn.ai/web) Click Streams

Frontend for Genn.ai Genn.ai Front (Based on Finagle)

Backend for Genn.ai Storm

Topology (Filter)

Kafka

MongoDB

Visualizer for Genn.ai iPad (APP) Tomcat Zoomdata

kakan.pu

http://genn.ai/ 16

Structure (Recording as behaviors)

Query
genn.ai/web) Click Streams

Frontend for Genn.ai Genn.ai Front (Based on Finagle)

Backend for Genn.ai Storm

Topology (Filter)

Kafka

MongoDB

Visualizer for Genn.ai iPad (APP) Tomcat Zoomdata

kakan.pu

http://genn.ai/ 17

Structure (Visualizing customers)

Query
genn.ai/web) Click Streams

Frontend for Genn.ai Genn.ai Front (Based on Finagle)

Backend for Genn.ai Storm

Topology (Filter)

Kafka

MongoDB

Visualizer for Genn.ai iPad (APP) Tomcat Zoomdata

kakan.pu

http://genn.ai/ 18

How is Genn.ai composed?

Genn.ai Frontend! Genn.ai Backend! Genn.ai Visualizer!

: Gathering clicks. : Recording behaviors. : Visualizing customers attrs.

Kafka MongoDB Storm

: Used for message bus : Storing Additional tables : Processing click streams
http://genn.ai/ 19

Details of recording behaviors.

http://genn.ai/ 20

Structure (Recording behaviors)

Query
genn.ai/web) Click Streams

Frontend for Genn.ai Genn.ai Front (Based on Finagle)

Backend for Genn.ai Storm

Topology (Filter)

Kafka

MongoDB

Visualizer for Genn.ai iPad (APP) Tomcat Zoomdata

kakan.pu

http://genn.ai/ 21

Calculated values
Recording behaviors
Useful values for each sites
Ex.)Hotel
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" }

reservation site,

http://genn.ai/ 22

Calculated values
Recording behaviors
Useful values for each sites
Ex.)Hotel
Where
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" }

reservation site,

is the favorite place?

http://genn.ai/ 23

Calculated values
Recording behaviors
Useful values for each sites
Ex.)Hotel
Where Where
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" }

reservation site,

is the favorite place? is the this searching place?

http://genn.ai/ 24

Calculated values
Recording behaviors
Useful values for each sites
Ex.)Hotel
Where Where What etc..
{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" }

reservation site,

is the favorite place? is the this searching place?

price range has he booked?

http://genn.ai/ 25

Functions for recording behaviors

1 2 3

Time shift function

Time sensitive counter (old item will be expired and deleted)

Facet ( like Solr )

Categorized counter

Range Facet ( like Solr )

Dynamic categorized counter

http://genn.ai/ 26

Functions for recording behaviors

1 2 3

Time shift function

Time sensitive counter (old item will be expired and deleted)

Facet ( like Solr )

Categorized counter

Range Facet ( like Solr )

Dynamic categorized counter

http://genn.ai/ 27

Time Shift function?

We make 3 min time window and now start.

( no clicks here )
Now

http://genn.ai/ 28

Time Shift function?

In our 3 min time window, 1.5 min are captured.

2 clicks have been recorded.

Now
1.5 mins Click Click
Time window

t
2 clicks are captured.

http://genn.ai/ 29

Time Shift function?

There are 3 clicks in the past. The length of the capturing window becomes

max.
Now
3 mins Click Click Click Time window

http://genn.ai/ 30

Time Shift function?

There are 4 clicks in the past. Ignore the rst 2 clicks
Now
1.5 min Click Click 3 mins Click Time window

Click

http://genn.ai/ 31

Time Shift function?

There are 4 clicks in the past. Ignore the rst 3 clicks
Now
3 mins Click Click 3 mins Click

Click Time window

http://genn.ai/ 32

Functions for recording behaviors

1 2 3

Time shift function

Time sensitive counter (old item will be expired and deleted)

Facet ( like Solr )

Categorized counter

Range Facet ( like Solr )

Dynamic categorized counter

http://genn.ai/ 33

Functions for recording behaviors

1 2 3

Time shift function

Time sensitive counter (old item will be expired and deleted)

Facet ( like Solr )

Categorized counter

Range Facet ( like Solr )

Dynamic categorized counter

http://genn.ai/ 34

Counting the frequency

This function is under the eect of the Time

shift function.

{ "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" } http://genn.ai/ 35

Counting the frequency

This function is under the eect of the Time

shift function.
You have booked 1 stay in Berlin and 3 stays

in Tokyo in this 3 weeks.

Now
1.5 week
L.A.

3 weeks
Berlin

S.F.

Tokyo Tokyo Tokyo

t
"largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }
http://genn.ai/ 36

Time window

Details of visualization.

http://genn.ai/ 37

Visualizing the customers

Query
genn.ai/web) Click Streams

Genn.ai Front (Based on Finagle)

Backend for Genn.ai Storm

Topology (Filter)

Kafka

MongoDB

Visualizer for Genn.ai iPad (APP)

Tomcat

kakan.pu

http://genn.ai/ 38

Visualizing the customers

Query
genn.ai/web) Click Streams

Genn.ai Front (Based on Finagle)

Backend for Genn.ai Storm

Topology (Filter)

Kafka

MongoDB

Visualizer for Genn.ai iPad (APP)

Tomcat

kakan.pu

http://genn.ai/ 39

Kafka consumer
Kakan.pu
The program for relaying clicks from Kafka to the

other systems.
The enriching function (joining the MongoDB data) The converting function (for following systems)

http://genn.ai/ 40

Visualizing the customers

Query
genn.ai/web) Click Streams

Genn.ai Front (Based on Finagle)

Backend for Genn.ai Storm

Topology (Filter)

Kafka

MongoDB

Visualizer for Genn.ai iPad (APP) Tomcat Zoomdata

kakan.pu

http://genn.ai/ 41

Visualizing tool
1 2 3

Zoomdata
A product containing a set of server app and ipad

app for visualizing event data.

http://genn.ai/ 42

Zoomdata
1 2 3

Zoomdata has the functions like..

WebAPI for streaming data. Own event processor for vizualizing. The bar and bubble graph on default setting. Customizable visualizing templates.

http://genn.ai/ 43

Feature plan.

http://genn.ai/ 44

Genn.ai Evolution
Adding functions on Genn.ai core.
Genn.ai should have ltering / aggrigation functions

for federating with other systems.

Historical rewinding function will be implemented with

the Hadoop cluster.

http://genn.ai/ 45

Future plan
Genn.ai wants friends!
We

are looking for co-challenging partners for tuning and improving Genn.ai. can use Genn.ai on your web sites for totally free (infrastructure etc..) if we could share the dream. codes will be opensourced in the future.

You

Genn.ai

Please feel free to contact us!

http://genn.ai/

http://genn.ai/ 46

Thanks!

http://genn.ai/ 47

48