Heavy Duty

Infrastructure
for the Adaptive
World
TABLE OF CONTENTS
2 The Knewton Platform
3 The Science Behind Knewton
7 Features of the Knewton Platform
11 Big Data and Education
12 Knewton in the Classroom
www.knewton.com
AN INTRODUCTION TO KNEWTON
2
www.knewton.com
The Knewton Platform
Knewton is an infrastructure platform that makes it possible for anyone to build the worlds most
powerful adaptive learning applications.
CHAPTER 1
Soon, every education application will be adaptive. There will be adaptive quizzing
apps, ashcard apps, textbook apps, simulation apps, and many more. Knewton
technology allows partners to build powerful adaptive experiences for any type of
content, in nearly any subject matter, for every student.
The Knewton platform consolidates data science, statistics, psychometrics,
content graphing, machine learning, tagging, and infrastructure in one place
in order to enable personalization at massive scale. In addition to powerful
adaptivity, the Knewton platform can also provide concept-level analytics for
students and teachers, pinpoint student prociency measurement, content efcacy
measurement, student engagement optimization, and more.
Knewton adaptivity is distinguished by its scientic rigor and tremendous scope.
Using advanced data science and machine learning, Knewtons sophisticated
technology identies, on a real-time basis, each students strengths, weaknesses,
and learning patterns. In this way, the Knewton platform is able to take the
combined data of millions of other students to help each student learn every single
concept he or she ever encounters.
This introduction to Knewton explores the science behind the Knewton platform,
explains what makes Knewton adaptivity diferent, and describes Knewtons impact
in a classroom setting.
3
www.knewton.com
Knewton addresses this challenge head-on, using educational path planning
technologies and advanced models of student ability. These technologies and
models ensure that every student progresses through the course material in a way
that maximizes his or her learning.
When most people use the phrase adaptive learning, what they have in mind is
simple diferentiated learning apps. Most of these apps use single-point adaptivity,
which evaluates a students performance at one point in time to determine the
level of instruction or material she receives from that point on. An example of this
would be a course that includes a diagnostic exam, the results of which determine
subsequent course content, with little or no further data mining or personalization.
When Knewton refers to adaptive learning, it means a system that is continuously
adaptive that responds in real-time to each individuals performance and
activity on the system and that maximizes the likelihood a student will obtain his or
her learning objectives by providing the right instruction, at the right time, about
the right thing. Upon completion of a given activity, Knewton directs the student
to the next activity. For example, when a student struggles with a particular set
of questions, Knewton will know where that particular students weaknesses lie
in relation to the concepts assessed by those questions and can deliver content
to increase the students prociency on those concepts. In this way, Knewtons
continuously adaptive system provides each student with a personalized syllabus
at every moment.
Proficiency Data
In order to do this, Knewton captures concept-level prociency data. Prociency
data goes way beyond observable data like test scores or time taken. Capturing
a students performance on a test or assignment does not take into account the
difculty of the material, the concepts to which it has relevance, or a students
The Science Behind Knewton
No two students are identical they learn and forget at diferent rates, come from diferent
educational backgrounds, and have diferent intellectual capabilities, attention spans, and modes of
learning. Designing an adaptive learning infrastructure that takes into account the characteristics of
each student is an immensely complex task.
CHAPTER 2
FI GURE A.
A Knewton recommendation
4
www.knewton.com
prior experience on similar content. A true model of prociency can estimate
not just what a student did, but what that student knows, at a granular level.
Prociency data can also show how prepared students are for further instruction
or assessment, and estimate how their abilities evolve over time.
To get this kind of prociency data, it is essential to have large pools of normed
content. To get normed content, there must be infrastructure to passively,
algorithmically, and inexpensively norm content at scale. Then, there must be
infrastructure to make sense of and take action on the resulting data. There are
no shortcuts. Knewton has built the necessary infrastructures to gather, multiply,
process, and action student prociency data. Anyone who wants to builds true
adaptive learning apps, without doing all the painful and expensive work on a one-
of basis, can use the Knewton infrastructure to do so.
Heavy Duty Infrastructure
The Knewton infrastructure comprises three main parts:
DATA INFRASTRUCTURE
Collects and processes huge amounts of prociency data.
Adaptive ontology: Maps the relationships between individual concepts, then
integrates desired taxonomies, objectives, and student interactions.
Model computation engine: Processes data from real-time streams and parallel
distributed cluster computations for later use.
INFERENCE INFRASTRUCTURE
Further increases data set and generates insights from collected data.
Psychometrics engine: Evaluates student prociencies, content parameters,
efcacy, and more. Exponentially increases each students data set through
inference.
Learning strategy engine: Evaluates students sensitivities to changes in
teaching, assessment, pacing, and more.
Feedback engine: Unies this data and feeds results back into the adaptive
ontology.
PERSONALIZATION INFRASTRUCTURE
Takes the combined data power of the entire network to nd the optimal strategy
for each student for every concept she learns.
CHAPTER 2: THE SCIENCE BEHIND KEWTON
5
www.knewton.com
Recommendations engine: Provides ranked suggestions of what a student
should do next, balancing goals, student strengths and weaknesses, engagement,
and other factors.
Predictive analytics engine: Predicts student metrics such as the rate and
likelihood of achieving instructor-created goals (e.g., how likely is a student
to pass an upcoming test with at least a 70%?), expected score, prociency on
concepts and taxonomies (e.g., state standards), and more.
Unied learning history: A private account that enables students to connect
learning experiences across disparate learning apps, subject areas, and time
gaps to allow for a hot start in any subsequent Knewton-powered app.
Heres a quick look at some of the theories and approaches behind the Knewton
infrastructure.
Item Response Theory (IRT)
Imagine that youre teaching a math remediation course full of fourth graders.
Youve just administered a test with 10 questions. Of those 10 questions, two
questions are very simple, two are incredibly hard, and the rest are of medium
difculty. Now imagine that two of your students take this test. Both answer
nine of the 10 questions correctly. The rst student answers an easy question
incorrectly, while the second answers a hard question incorrectly. Which student
has demonstrated greater mastery of the material?
Under a traditional grading approach, you would assign both students a score of 90
out of 100, grant both of them an A, and move on to the next test. But this approach
illustrates a key problem with measuring student ability via testing instruments:
test questions do not have uniform characteristics. So how can Knewton measure
student ability while accounting for diferences in questions?
IRT models student ability using question level performance instead of aggregate
test level performance. Instead of assuming all questions contribute equivalently
to our understanding of a students abilities, IRT provides a more nuanced
view on the information each question provides about a student. It is founded
on the premise that the probability of a correct response to a test question is a
mathematical function of parameters such as a persons latent traits or abilities
and item characteristics (such as difculty, guessability, and specicity to topic).
Figure B shows two item response function curves generated by an IRT model.
The curves illustrate how an IRT model relates a students ability with the
CHAPTER 2: THE SCIENCE BEHIND KEWTON
I TEM RESPONSE FUNCTI ONS FOR
VARI OUS DI FFI CULTY VALUES
P
R
O
B
A
B
I
L
I
T
Y

O
F

A
N
S
W
E
R
I
N
G

C
O
R
R
E
C
T
L
Y
STUDENT ABI LI TY
FI GURE B.
Item Response Theory
6
www.knewton.com
probability of answering a question correctly, given that questions difculty and
discrimination levels and guessability. While IRT models are atemporal and
reliant upon a single measure of ability (and thus reect only one facet of the
science behind Knewton recommendations), they help Knewton better understand
how a students performance on testing relates to his or her ability.
Probabilistic Graphical Models (PGMs)
This framework, which encompasses statistical methods such as Bayesian networks
and Markov random elds, allows data scientists to code and manipulate probability
distributions over multi-dimensional spaces in which there are hundreds or even
thousands of variables at play. In other words, PGMs allow Knewton analysts to
build complex models one efect at a time, relating the many learning activities
they observe to estimations that are useful for recommendation.
One of the ways in which Knewton applies PGMs is by using a students known
prociencies to determine which other topics he may be ready to master. For
instance, such a model might help the platform discover to what degree a mastery
of fractions helps students master decimals and to what degree a mastery of
decimals helps students master exponentiation. Knewton data scientists can
thus determine the relationship between mastery of fractions and mastery of
exponentiation. Ultimately, the discovery of these types of relationships allows
the Knewton platform to continually rene its recommendations.
CHAPTER 2: THE SCIENCE BEHIND KEWTON
FI GURE C.
Probabilistic Graphical Model
X2 X1 X3
Y2 Y1 Y3 Y4
7
www.knewton.com
Knewton-enhanced products are linked by the Knewton knowledge graph, a
cross-disciplinary graph of academic concepts. The knowledge graph takes into
account these concepts, dened by sets of content and the relationships between
those concepts. Knewton recommendations steer students on personalized and
even cross-disciplinary paths on the knowledge graph towards ultimate learning
objectives based on both what they know and how they learn. The more content
that teaches or assesses each concept that is added to the system, the more precise
the adaptive course experience becomes.
When visualized, the knowledge graph can provide a sense of a students potential
ow through the course material.
Within the knowledge graph, concepts have prerequisite relationships that
contribute to dening a students path through the course. Special relationships
that dene content as either instructional or assessment determine what kind
of content to deliver to students at any given point.
Network efects are a natural consequence of the knowledge graph and the models
Knewton uses to determine its recommendations. In isolation, each students
response to each question is only a tiny scrap of information, but when propagated
through the entire system and understood in context, the value of that information
is amplied tremendously. Every student action and response around each content
item increases the systems understanding not only of the student and the content
item, but also, by extension, of all the content in the system and all the students
in the network.
The more students who use the Knewton platform, the more rened the relationships
between content and concepts and the more precise the recommendations delivered
through the knowledge graph. For each student learning each individual concept,
the system nds a population of extremely similar students who have already
FI GURE D.
Laurens personalized learning pathway
FI GURE E.
Williams personalized learning pathway
Features of the Knewton platform
Knewton provides partners with an infrastructure platform that leverages advanced technology and
data science to improve learning experiences. A cross-disciplinary knowledge graph, prociency-
based adaptivity, lifelong student learning proles, and vast network efects combine to produce
powerfully personalized learning for every student who takes a Knewton-enhanced course.
CHAPTER 3
8
www.knewton.com
learned that concept. It then asks, Who among those similar students learned
that concept the best, and what did they do that worked so well? The system then
allows this proven efective learning path to inform the students learning path
going forward. In this sense, a student experiencing a particular challenge (for
example, spatial skills as they relate to math word problems) need not be limited
by the fact that no one else in his or her class is experiencing the same difculty.
The Knewton adaptive learning platform is able to take the combined data of
millions of other students to help each student learn every single concept she ever
encounters.
Recommendations
As students work through online lessons, Knewton analyzes vast amounts of
anonymized data to gure out what a student knows and how they learn best. Then
Knewton recommends what to study right now to help a student succeed.
Knewton recommendations guide each student toward learning goals set by
instructors or course administrators. Goals, which can apply to an individual or
to groups of learners, allow instructors to ensure recommendations are tailored to
every learner. An instructor can set multiple goals for a class or individual students.
For example, an instructor might set a goal for all students to be prepared for a
specic quiz or exam by a particular date and at a minimum score. The same
instructor might also set an intermediate goal for an individual to reach a specic
prociency level on a concept or assessment.
Recommendations allow students with diferent needs, interests, strengths, and
weaknesses to work toward common goals in a sequence and pace that continuously
adjusts to meet their needs. Each student advances as efciently as possible, while
also meeting prociency requirements along the way. Recommendations can work
within any pedagogy or course format.
Analytics
Knewton data scientists transform huge quantities of data into cognitive/
interaction models, estimators, rankers, data frameworks, and actionable insights.
This translates into instructional and content analytics for teachers, publishers,
and administrators.
One of the biggest challenges facing teachers and school administrators today is
the growing diversity of the students within their population. A greater diversity
of students means a greater diversity of needs to consider. Some struggle because
English is not their rst language; others have difculty with focus or organization.
CHAPTER 3: FEATURES OF THE KNEWTON PLATFORM
9
www.knewton.com
Others may be particularly weak in some area but possess unusual strengths in
another. Knewton analytics provide insight into the learning process, specically
in terms of efcacy, engagement, and knowledge retention, and help teachers
ensure that diverse needs are being met.
Knewton analytics provide insight into the learning process, help teachers make
sense of individual student learning patterns in the context of a whole class, and
anticipate what students will need in the near future. While teachers do this
already, it becomes more difcult to do manually as students pursue increasingly
varied paths. The metrics Knewton provides include:
PROFICIENCY
How procient is the student in the topics covered in this chapter?
READINESS FORECAST
Given multiple factors (e.g., the students current rate of learning, work completed,
and current understanding), is the student likely to be procient in this entire unit
in three weeks?
PREDICTED SCORE
What would the student score on Fridays quiz if he took it today?
ACTIVE TIME
How much productive time has the student spent working in the past week?
A teacher can use these metrics to coach his or her student with a precise
understanding of his particular weaknesses. Afer multiple years of teaching the
same course, teachers will be able to compare data from year to year. Knewton
analytics will allow them to access useful information (like which students are
struggling and require immediate attention) while leaving them free to interpret
the results. This capacity allows teachers to better understand their curriculum, so
that they can rene it from semester to semester and year to year.
Content analytics can help teachers, publishers, and administrators determine the
strongest and weakest aspects of their teaching materials and precisely what the
content they are working with teaches and assesses. This ensures that content can
be analyzed for ne-tuned improvements from year to year, and that students are
never stuck with outdated or inefective materials.
CHAPTER 3: FEATURES OF THE KNEWTON PLATFORM
10
www.knewton.com
Student Profile
With Knewton, students can maintain a continuously updated learning prole that
contains information on what the student knows and how she learns best. The
prole is progressive, which means it keeps getting smarter the longer the student
remains on the platform.
Today, students walk into classrooms each September as if they were just born.
Teachers must learn everything about them from scratch. Knewton-powered
apps change this, allowing each student to start courses warm by connecting
his or her learning history to every app. Instructors see students prociency in
individual concepts, how they study, and how well diferent strategies work for
them. Students get learning products that provide the exact material they need,
when they need it. This learning history stretches across grades and subjects, for
every Knewton-powered app helping all students maximize their potential.
The more ofen a student uses Knewton-enhanced courses, the more efective the
platform becomes at serving up targeted learning material.
The Knewton prole provides penetrating insight into students own understanding
of the material what they truly grasp and dont grasp, their misunderstandings
and misconceptions. Students develop a deeper, more nuanced understanding of
their learning style and strengths and weaknesses, helping them to maximize their
academic potential.
The implications of all this are straightforward: student engagement can be
strengthened if academic work is imbued with a sense of continuity. Nothing
is more dissatisfying to students than feeling like the challenges they face are
essentially arbitrary and culminate in nothing. The Knewton learning prole
answers the student need for continuity and meaning by afording students a sense
of long-term investment in the learning process.

CHAPTER 3: FEATURES OF THE KNEWTON PLATFORM
11
www.knewton.com
Big data unleashes
a range of productive
possibilities in the
education domain
in particular.
How Education Data is Different
The advent of big data in areas such as internet search and social media has
disrupted existing industries, created new industries, and led to the extraordinary
success of companies such as Google and Facebook. Big data unleashes a range
of productive possibilities in the education domain in particular, since data that
reects cognition is structurally unique from the data generated by user activity
around web pages, social proles, and online purchasing habits.
One feature that distinguishes the data produced by students is the fact that
academic study requires a prolonged period of engagement; students remain on the
platform for an extended length of time. There is a focus, intention, and intensity
to students activity; education is high-stakes. The sustained intensity of these
eforts generates vast quantities of meaningful data that can be harnessed to power
personalized learning for each individual.
Another feature that distinguishes the data produced by students is the very
high degree of correlation between educational data and the aggregated efect
of all those correlations. If, for example, a student has demonstrated mastery of
fractions, algorithms can reveal how likely it is that he will demonstrate mastery
of exponentiation as well and how best to introduce that concept to him or
her. If a student has demonstrated mastery of various grammatical concepts (say,
subjects, verbs, and clauses), educational data can optimize his learning path, so
that diferent sentence patterns will click for the student as quickly as possible.
The hierarchical nature of educational concepts means that they can be organized
in a graph-like structure, which means that the student ow from concept-to-
concept can be optimized over time, as Knewton learns more and more about
the relationships between them through data. Every student action and response
around each content item ripples out and afects the systems understanding of all
the content in the system and all the students in the network.

Big Data and Education

The term big data is used to describe the tremendous volume, velocity, and variety of data
generated by various technology platforms. Big data refers specically to data sets that are so large
and complex that they are challenging to work with using traditional database management tools.
CHAPTER 4
12
www.knewton.com

Knewton at ASU
Online adaptive learning makes it possible to implement mastery-based learning
in a scalable way. Knewton Math Readiness, for example, creates a guided, self-
paced environment in which live instruction is optimized around targeted group
sessions. The course is designed to present students with personal learning paths
as it continually assesses their mathematical prociency and adapts accordingly.
Lessons consist of videos, online textbook selections, and lesson quizzes. Early
efcacy reports reect the success of the program: afer two semesters of use with
over 2,000 developmental math students at Arizona State University, withdrawal
rates dropped by 56%, pass rates went from 64% to 75%, and 45% of the class
nished four weeks early.
This implementation model is ofen referred to as blended learning, a term which
describes any arrangement in which a student learns in part at a brick-and-mortar
facility and in part through online delivery with student control over time, place,
path, or pace.
Irene Bloom, a Senior Lecturer at ASU, was originally a skeptic of online learning.
But she says that the classroom dynamic has changed for the better since
introducing Knewton into her developmental math classes: I love looking around
the classroom and seeing them working in groups, talking to each other and
explaining things to each other Most of the time, diferent groups are working
on diferent things, depending on where they are in the course. This is very new
for me. Before this, I worked on the assumption that all students were at the same
place. Now, because they progress at diferent rates, I meet them where they are.
FI GURE G.
Knewton math readiness results at ASU
Before this, I worked
on the assumption that
all students were at
the same place. Now,
because they progress
at diferent rates, I meet
them where they are.
Knewton in the Classroom
Adaptive learning supports mastery-based learning, a school of teaching founded on the idea
that student progression through a course should be dependent on prociency as opposed to
the amount of time spent on academic work. Though it may not always be referred to by name,
mastery-based learning describes any situation in which one is given a set of problems, labs, or
activities, and in which progression through that material is dependent on successful completion of
various tasks rather than seat time.
CHAPTER 5
13
www.knewton.com
How Knewton Engages Students
Knewton can improve student engagement by increasing self-condence, decreasing
discomfort and frustration, and encouraging productive learning habits.
Students are less likely to lose focus if feedback is immediate and they can quickly
self-correct. A continuously adaptive learning system is able to deliver personalized
feedback to both multiple choice and free response questions quickly that is,
instantaneously or near-instantaneously. The result is pacing that is conducive to
risk-taking, experimentation, iterative development, and rapid learning.
COMMUNITY AND COLLABORATION
Isolation can exacerbate the challenges students experience in school. An adaptive
system can improve student engagement by weaving a social component into
coursework. Knewton Math Readiness, for instance, provides a dashboard that
allows teachers to group students who are working on the same material. Using the
reporting features, teachers can also arrange peer review opportunities and form
groups of students whose abilities complement each other.
GAMIFICATION
With countless opportunities for students to demonstrate skill and reect on
action and feedback, adaptive courses naturally have much in common with
games. Whats more, adaptive courses keep students in a game-like state of ow
by escalating the difculty of the work incrementally and unveiling levels one at a
time to increase suspense. These and other game elements can be heightened to
transform adaptive courses into truly gamied learning experiences.
FI GURE H
Study groups and peer review
FI GURE I.
Gamication visual elements
CHAPTER 5: KNEWTON IN THE CLASSROOM
The Knewton infrastructure makes it possible
for anyone to build adaptive learning
applications. Knewton can enable personalized
recommendations and predictive analytics for
new or existing products.
Interested in working with Knewton?
learn.knewton.com/signup-platform
14
www.knewton.com