Assignment 2: A Doc2Vec Model for Rating Prediction
Steven Hill
University of California, San Diego
San Diego, California
PID: A53040946
s5hill@eng.ucsd.edu
ABSTRACT
We design a joint model that combines Word2Vec’s exten-
sion Doc2Vec with latent models for predicting user-item
ratings. We explore this task by look at the BeerAdvocate
dataset, a large corpus containing user-item written reviews
and corresponding ratings. By using user and item embed-
dings generated during training jointly as input to a neural
network regression model, we obtain improved performance
over the popular rating prediction approaches.
Categories and Subject Descriptors
[Recommendation Systems]: Latent Factor Model
; [Distributed Word Representations]: Word2Vec
1. INTRODUCTION
For this assignment we will look at the task of predicting
beer ratings for given users and beers from the BeerAdvocate
dataset. Our task is as follows: given an item i (a beer
brand) and a user u (the drinker), our goal is to develop a
function r(u, i) which tries to predict the rating that user u
would give to item i.
To help us with this task, we look at the BeerAdvocate
dataset. As the name implies, the reviews and ratings here
cover mainly beers. This data spans more than 10 years
from 1998 to 2011. It consist of more than 1.5 million re-
views, with over 30,000 users and 66,000 beers. Each review
includes ratings on appearance, aroma, palate, taste, and
overall impression (which is what we care about for this
assignment). While the dataset includes a lot potentially
useful information, for this task I only use the user id, beer
id, beer rating and text included in the review.
A great characteristic of this dataset is that the median
number of words per review is 126, meaning that we want a
model that extracts information from this abundant amount
of text to better its prediction. This promotes our design of
a model that makes use of review text at training time to
better predict user-item ratings at test time.
So as stated, at training time we have access to the user-
item reviews along with ratings. However at test time we
only have access to user and item id’s. We present a model
that capitalizes on the text in the reviews to gain better
rating predictions.
The remainder of the paper is structured as follows: we
briefly cover related work in the area of user-item rating
prediction and review text, cover the model in whole, and
present experimental results using this model for this very
task.
2. RELATED WORK
One of the most crucial works in recent years for user-item
rating prediction is the latent factor model described in [4].
A function for rating prediction r is defined as r(u, i) =
α + βi + βu + γi · γu , where γ is a multi-dimensional latent
representation for user u and item i. This model is com-
parable to most state of the art and generalizes well with
proper regularization.
The joint model we will look at can be viewed as a follow-
up to [6]. They combine the latent factor model with Latent
Dirichlet Allocation (LDA) defined in [1]. LDA is a model
that discovered hidden dimensions in text. Every document
trained on by LDA will get a distribution off “topics” that
are discovered automatically. This combination is referred
to as Hidden Factors as Topics (HFT). By using the in-
teraction between a user’s (or item’s) topic vector and an
item’s (or user’s) latent vector, they achieve improved per-
formance over the baseline latent factor model. The topic
vectors are learned from the review text corresponding to
a user or item. Later will use this model in a comparison
against our own. [2, 11, 3] are a small collection of works
that deal with model-based rating prediction.
The BeerAdvocate dataset has been used in past work
when studying review text. [7] try to model the various
aspects that come from beer reviews (like feel, look smell,
taste, and overall rating). [8] tries to model how users’ beer
taste change over time and makes recommendations based
on this temporal change. [6] uses the beer dataset for rating
prediction; much like we are for this assignment.
In this work we will use Doc2Vec as means of extracting in-
formation for text. Word2Vec ([10, 9]) deals with efficiently
representing word vectors in low dimensional space (often
referred to as embeddings). Doc2Vec ([5]) is an extension of
Word2Vec that produces document vectors. Similar to how
LDA was used previously to produce latent vectors, we will
be using Doc2Vec for a similar task, hoping to gain more
semantics in our vectors.
3. MODEL
Before going into the details of the joint model, it is im-
portant understand each component individually and how
they come together. We review how Doc2Vec (and it’s base
Word2Vec) work.
3.1 Word2Vec
A well known framework for learning word vectors is shown
in Figure 1. The main idea to predict what word occurs
given the surrounding (context) words. In this model, every
word is mapped to a unique vector, represented by a column
in weight matrix W (note that W also corresponds to the
embedding layer of neural network).
There are actually two flavors of Word2Vec. Continuous-
Bag-of-Words (CBOW) tries to predict a single target word
given multiple context words, while Skip-gram (SG) tries
to do the opposite; predicting multiple target words given
a single context word. They both produce similar results
(at least for our use case), and thus we will not distinguish
between the two going forward.
More specifically, given a sequence of training words, w1 ,
w2 , · · · , wT , we want to optimize the following:
T −k
1 X
log p(wt |wt−k , · · · , wt+k )
T
t=k
This corresponds to maximize the average log probability.
The predicted output word is decided based on a multiclass
classifier. Using softmax, we have:
eywt
p(wt |wt−k , c . . . , wt+k ) = P yw
ie
i
These models are trained using stochastic gradient de-
scent with back propagation. These are commonly known
as neural language models. The output is dependent on
the weights from the first layer and second layer to define
the multinomial distribution of the output. After training
converges, the weights corresponding to each input word and
output word are used as vectors for word embeddings. Word
embeddings with similar semantics are mapped to similar
positions in vector space. These vectors can be used for a
wide variety of tasks, including word algebra and language
translation.
But because this learning is done through back propa-
gation, it’s not tractable to do softmax classification given
the size of the word vocabulary (which can be in the mil-
lions). This is a problem for the output end of the model.
In practice either negative sampling or hierarchical softmax
schemes are used to speed up training. We will forego an
indepth review of alternative training methods for brevity.
In the negative sampling case, logistic regression is used to
compare a positive outcome to sampled “negative” cases. In
hierarchical softmax, the probability distribution is encoded
in a binary tree, where the left and right probabilities gen-
erate the probability of seeing a word given context.
3.2 Doc2Vec
The authors of [10] use an extension of this to learn para-
graph vectors [5]. Paragraph ids are used in a similar manner
to predict the target word given context words, except now
there are additional vectors representing the paragraphs.
Every paragraph is mapped to a unique vector, represented
by a column in matrix D, and every word is mapped to
a unique vector, represented by a column in matrix W .
The paragraph vectors and word vectors are either directly
summed, averaged, or concatenated at the hidden layer to
predict a target word given context words. The only real
change from the Word2Vec to this model is how to handle
the hidden layer. The summed hidden layer model is shown
in Figure 2.
The paragraph (or document) variables can thought of as
an additional context words trying to predict a target word.
Training happens in the same way as Word2Vec; back propa-
wM
wM
wM-1
wM-1
... ...
...
w2 w2
w1 w1
Figure 1: Word2Vec. This (CBOW) model tries to
capture the probability of a target word being at
the center of input context words; predicting which
word should appear given surround words. For each
word in the input context words, the word is repre-
sented as a one-hot encoding of that context word
i, and the output node corresponds to the probabil-
ity of that target word appearing given the context
word. During training, the weights are adjusted try
to push the probability of the target word appearing
with the context word to 1.
gation with approximate methods for weight training. After
training, there are vector embeddings for every word and
document (if concatenation is used they can be of different
lengths). The authors of this methodology claim to have
gained state-of-the-art results on sentiment analysis and in-
formation retrieval.
3.3 Multi-layer Perceptron Regression
Lastly, before we discuss the joint model for prediction,
it will also help to look at regression in a neural network
setting (shown in Figure 3). The main difference between a
neural network regression and classification model is that the
output is no longer non-linear. With proper regularization
techniques, these standard models can be trained to have
weights to generalize very well on test data.
3.4 Joint Doc2Vec Model for Predicting Rat-
ings
In this section we describe the joint model that combines
both Doc2Vec and a neural regression model to predict user-
item ratings. Figure 4 shows an example of such a joint
model.
During training, the model takes in three inputs in the
form of three one-hot vectors per training instance. These
inputs correspond to some item i, user u, and word w for
each w in the review text for item i and user u. The joint
model has two objective functions it is trying to optimize.
On one side, we want we have function related to the per-
formance of the regression model F1 . This tries to minimize
the mean square error for user-item pairs. On the other side
is F2 , a function related to the Doc2Vec side. We want to
maximum the average log probability of seeing target words
given context words, user id, and item id. We want to min-
 w uK

... w

u1
dN ru,i wM
... ...
iN
... wM-1
wM ...
... ...

d1 i1
wM-1 w2

wM
wM ... ... w1

wM-1
w2 ...
wM-1
... w2
w1
w1

w2

Figure 4: A neural network model for jointly pre-

w1
Figure 2: Doc2Vec. Similar to Word2Vec, except
there is an additional binary document variables.
There are multiple ways to process the addition
document information; either document vectors can
be summed (shown above), averaged into the word
vectors, or more preferred, concatenating document
vectors to the word vectors. Shown above is a sim-
plified version of Doc2Vec that ignores concatena-
tion and averaging of input vectors. Aside from the
hidden layer decision, the output and back propaga-
tion remain the same.
i1
i2
h1
i3
h2 o
i4
h3
i5
i6
Figure 3: A simple feed-forward neural network for
regression. ij is j-th input, hk is a k-th hidden node
which applies a nonlinear transform to the sum of
the input, and o is output of the model; a linear sum
of the hidden nodes.
dicting user-item ratings and producing word vec-
tors. The input (the middle column) corresponds to
the concatenation of one-hot encodings of users (u),
items (i), and words (w). The first layer (the encod-
ing layer) of both sides of the network represent an
embedding of the input users and words. The right
side is trained on the user-item review text using
a softmax output scheme for predicting neighbor-
ing context words, while the left side is trained for
predicting the rating using regression.
imize the following:
F = F1 − µ · F2
µ is a hyperparameter for how much weight we want to
give the Doc2Vec model in comparison to the regression
mode. We want the embeddings for each side to influence
eachother, so the part of W corresponding the documents in
the Doc2Vec model is the same as the W used for the regres-
sion. This means the embeddings used for word prediction
and rating prediction are the same.
For every word of every review in the training set, we
perform an update to the weights of the model. The model
can be fit to optimize F1 then −µ·F2 (or vice versa) for every
example, or after every batch. The algorithms for training
either individual model can be used in an alternating fashion
to update the weights for the full model. F1 just uses user i
and item j, while F2 will additionally use the review text.
Intuitively, these models can be thought of as using low di-
mensional embeddings of the input for their predictive tasks.
Word2Vec uses embeddings of each word to predict a tar-
get word. Doc2Vec uses the concatenation of document and
word embeddings to predict a target word. This joint model
uses the concatenation of user, item, and word embeddings
to predict a target word, while simultaneously predicting a
user-item rating using a concatenation of the user and item
embeddings.
A typical issues with these models is the number of param-
eters and how they tend to overfit. To help this, we apply
very small L2 penalty on the user and item vectors, and
apply a dropout scheme to the remainder of the regression
part of the model. Dropout [12] is a method of regulariza-
tion that has proved extremely effective in neural models in
recent year. The idea is that for each weight update instance Method Avg. MSE
during training each node has a chance of “turning off” with Latent Factor 0.371
probability p. The nodes do not influence the output at that Hidden Factor as Topics 0.367
time, and thus the weights are not updated. The intuitive MLP Regression 0.385
reason that this is effective is it uses the idea of majority MLP Regression w/ Dropout 0.369
influence. It’s hard to overfit when told a model with nearly Joint Doc2Vec 0.365
p of its nodes gone has to perform similarly to the full archi- Joint Doc2Vec w/ Dropout 0.362
tecture. In the results section we illustrate the importance
of this regularization. Table 1: Results from experiments. The MSE is
We will see some of the following pros and cons in the suc- averaged over 10 trials.
ceeding sections, but it will help mention them here clarity.
The major pro is that with dropout regularization the model
has consistently strong performance. The negatives are that
training is very slow (training takes around 5 hours), and 0.9
there are many hyperparameters. Another issue is the cold Joint Doc2Vec
start problem, in which a user and/or item have never been Joint Doc2Vec w/ Dropout
0.8
seen before. In this case, we use averages of user ratings,
otherwise set to global average.
0.7

MSE on Testing Set

4. EXPERIMENTS AND RESULTS
0.6
As per our task, we will use the BeerAdvocate beer dataset
to evaluate our joint Doc2Vec model. But in order to eval-
uate our model, we need to decide which hyperparameters 0.5
to use as there is a lot of flexibility in the architecture. We
will use the following: 0.4

• Concatenation of user, item, and word vectors 0.3

6 8 10 12 14 16 18 20
• 5 dimensional user and item vectors; 200 dimensional # Epochs on Training Set
word vectors
Figure 5: Performances provided by regulariza-
• Training with negative sampling
tion using dropout scheme on the regression side
• The input layer of the regression is 10 (user + item of model. The joint Doc2Vec model with dropout
dimensionality) and L2 input regularization shows improved per-
formance, even many iterations beyond the point
• The hidden layer of the regression is 5, using Rectified where L2 regularization starts to increase on the test
Linear Units (ReLUs). dataset.

• Let µ = 10

• Implemented in Tensorflow using Adam optimizer
• Uses dropout with probability p = 0.9
The hyperparameters were chosen from various criteria.
Word vector sizes were chosen from what has worked heuris-
tically well in past word embedding findings. The user and
item dimensionality were picked to keep in line with past
work done by [6]. The architecture and µ were selected on
results from random subsets of the data. Little was gained
from increasing the size of the hyperparameters, and not
enough to decide they were a major factor.
We run 10 trials of experiments, each of which randomly
chooses 80% for training, 10% for validation, and 10% for
testing. For each iteration we stop training once mean square
error has been minimized on the validation set, The results
from the 10 trials are then averaged and presented in Ta-
ble 1. Similar to results we have seen previously, the Topic
Model based model that also used review text outperformed
a baseline latent factor model. Interestingly enough, what
ended up being a major influence over the results was the
regularization. In all models we use L2 regularization, but
the models that employed a dropout scheme had the best
and most consistent results.
These results are significant because they show us that
the joint Doc2Vec embeddings convey useful information
(at least more so than previous models). According to [5],
Doc2Vec embeddings capture document semantics just as
Word2Vec embeddings does for words. A potential rea-
son that our joint model outperforms HFT is that user and
item vectors might contain more semantic information than
the document topic distributions achieved through HFT and
LDA. It’s hard to say whether the performance gain comes
from the different training methodology, or whether to con-
tribute it to architecture and regularization.
Our joint Doc2Vec model creates a nonlinear function that
determines the “agreeance” between user and item embed-
dings. The Doc2Vec part of the model is pushing these
vectors towards some semantically meaningful positions in
vector space, while the regression part is pushing the seman-
tics to correspond to beer ratings. All the parameters in the
network are working towards balancing the two tasks, with
the hopes that one influences the other in a positive way. At
least the for the task of rating prediction, we can see this
goal has been accomplished.
Earlier we mentioned how important dropout regulariza-
tion is to the model. Figure 5 show the training on one trial
of the joint Doc2Vec model with and without dropout.
 Although not shown in the table 1, in all trials the joint [11] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl.
Doc2Vec model performed similarly and outperformed the Item-based collaborative filtering recommendation
other models. algorithms. In Proceedings of the 10th international
conference on World Wide Web, pages 285–295. ACM,
2001.
5. CONCLUSION
[12] N. Srivastava, G. E. Hinton, A. Krizhevsky,
We looked at a joint model that uses both Doc2Vec and I. Sutskever, and R. Salakhutdinov. Dropout: a simple
neural regression to achieve state-of-the-art performance (at way to prevent neural networks from overfitting.
least for the BeerAdvocate dataset and up until 2013). This Journal of Machine Learning Research,
approach was easily implemented in Tensorflow, and even 15(1):1929–1958, 2014.
though the hyperparameters were mostly heuristically cho-
sen, still outperformed similar model-based methods.
There are still many questions remaining in regards to
the model we have presented. Would these results hold for
other datasets? With better regularization could the HFT
model achieve better results? It there an ideal setting for
hyperparameters? Although the model is simple in theory,
it has a lot of weights that need to be adjusted.
With all the recent work in vector encodings of words and
text, it would be interesting to see how this model could be
adjusted to further improve upon results.

6. REFERENCES
[1] D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent
dirichlet allocation. Journal of machine Learning
research, 3(Jan):993–1022, 2003.
[2] J. S. Breese, D. Heckerman, and C. Kadie. Empirical
analysis of predictive algorithms for collaborative
filtering. In Proceedings of the Fourteenth conference
on Uncertainty in artificial intelligence, pages 43–52.
Morgan Kaufmann Publishers Inc., 1998.
[3] A.-L. Deng, Y.-Y. Zhu, and B. Shi. A collaborative
filtering recommendation algorithm based on item
rating prediction. Journal of software,
14(9):1621–1628, 2003.
[4] Y. Koren and R. Bell. Advances in collaborative
filtering. In Recommender systems handbook, pages
77–118. Springer, 2015.
[5] Q. V. Le and T. Mikolov. Distributed representations
of sentences and documents. In ICML, volume 14,
pages 1188–1196, 2014.
[6] J. McAuley and J. Leskovec. Hidden factors and
hidden topics: understanding rating dimensions with
review text. In Proceedings of the 7th ACM conference
on Recommender systems, pages 165–172. ACM, 2013.
[7] J. McAuley, J. Leskovec, and D. Jurafsky. Learning
attitudes and attributes from multi-aspect reviews. In
Data Mining (ICDM), 2012 IEEE 12th International
Conference on, pages 1020–1025. IEEE, 2012.
[8] J. J. McAuley and J. Leskovec. From amateurs to
connoisseurs: modeling the evolution of user expertise
through online reviews. In Proceedings of the 22nd
international conference on World Wide Web, pages
897–908. ACM, 2013.
[9] T. Mikolov, K. Chen, G. Corrado, and J. Dean.
Efficient estimation of word representations in vector
space. arXiv preprint arXiv:1301.3781, 2013.
[10] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and
J. Dean. Distributed representations of words and
phrases and their compositionality. In Advances in
neural information processing systems, pages
3111–3119, 2013.