Vous êtes sur la page 1sur 11

Running head: MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 1

Machine Learning; Neural Networks In Starcraft II

Jack T. Wagner

Liberty High School

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 2


Abstract

The purpose of this research is to create and analyze, drawing conclusions on how game oriented

entities of neural network learning systems operate and provide evidence of efficiency and

functionality. A game environment can be created with neural network learning type entities to

grow and advance the AI of the entities through repetitive trial; a new generation of the entity

created by each. The latter more advanced than the last. By using these methods a capable AI

was made, one capable of contesting an experienced player in a RTS game based on strategic

and tactical, decision and actions. The NN is further advanced through facing human opponents

and replaying mass game data, and playthroughs. Resulting in a neural network that can be

proven to have advanced in its skill of the game, drawing conclusions on the generational

growing of the neural nets intelligence and mistake ratios.

Keywords: AI (Artificial Intelligence), RTS (Real-Time-Strategy), NN (Neural Network)

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 3

Machine Learning; Neural Networks in Starcraft II


Artificial Intelligence (AI) is a quickly developing field in todays world; new

breakthroughs being made everyday. Especially that of entities that can interact with humans; but

importantly can learn and grow. Neural Networks are a technology that allows programs to

emulate the network of synapses in the human brain, allowing them to determine and make

decisions with a complicated series of inputs, hidden layers or synapses, and outputs. As well

as learn from their experiences through problem solving processes. More specifically, Neural

networks are a set of algorithms that are designed to recognize patterns. They interpret sensory

data through digital perception, labeling or raw input. The patterns they recognize are numerical,

stored by vectors, by which any data, be it images, sound, text or time series, must be translated.

Neural networks help to organize and sort by a sectional method. You can think of them as a

clustering and classification layer on top of data you store and manage. By similarities and trends

they sort and group unlabeled data among the inputs(Figure 1.), and they classify data when they

have a labeled data set to match. A game environment can be created with neural network

learning type entities to grow and advance the AI of the entities through repetitive trial; a new

generation of the entity created by each. The latter more advanced than the last. By using these

methods a capable AI can be made, one capable of contesting an experienced player in a

RTS(Real-Time-Strategy) game based on strategic and tactical, decision and actions.

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 4

Figure 1.
-Diagram of a Neural Network by: AstroML

Artificial intelligence has a come a long way as of late; especially notable in that of the

professional stage of strategy games. Researchers are looking to reach the next achievement in

AI development by defeating world champions or otherwise, the greatest of humanity. In 2016

Google created a Go playing AI called Alpha Go that defeated the world champion Aja Huang in

a ranked tournament. the mind behind the game was an artificial intelligence named AlphaGo,

and it was beating one of the best players of perhaps the most complex game ever devised by

humans(Metz). The defeat of the world champion Go player signified an achievement in the

field of artificial intelligence and a clear sign that the battlefield had moved to the next stage.

Games act like milestones; when a machine is superior to humans at playing difficult games,

its a sign that its neural net has reached a new level of intelligence(Quach). IBMs

supercomputer, Deep Blue, conquered chess in the late 1990s. Last year, Google DeepMind

gained attention for

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 5


creating Alpha Go, an AI that beat Go master Lee Sedol. The next game in researchers sights is

StarCraft II. Due to the advancing technologies for creating and developing artificial

intelligence, humanity and machines now have the capability to evolve and learn from each

other.

A concept and proven practice of developing artificial intelligence is through the use of

neural networks. Training neural networks effectively produce evolving entities of artificial

intelligence and products of machine learning. Inside the DeepMind offices near Kings Cross

station in London, the team fed 30 million human Go moves into a deep neural network, a

network of hardware and software that loosely mimics the web of neurons in the human

brain(Metz). The training that Google's Go bot underwent prepared it for the match against the

worlds national Go Champion. If you feed a neural net enough photos of your mom, it can

learn to recognize her. Feed it enough speech, it can learn to recognize what you say. Feed it 30

million Go moves, it can learn to play Go(Metz). Neural networks have the capability to

emulate the network of synapses in the human brain; giving them the ability to learn by

processing sets of data fed to them. This can applied to training any sort of machine. Modern

ATM machines use neural network reading systems to process checks, reading the alphabetic

and numeric characters on them with more than 99.9% accuracy. This is due to how extensively

their respective neural networks were trained. By having millions of samples of hand-written

character and numbers fed to them they effectively learned how to identify each character and

number with incredible

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 6


accuracy. Neural networks and machine learning open new possibilities not only in more

advanced scientific research, but also apply to day to day machines and processing systems.

The battleground for AI research is moving on to conquer more complex games and

expand the scope of the quickly developing field. Like seen with Go in 2016. Games provide

a perfect platform for AI research and developing the field of machine learning. By

definition, games are constrained. Theyre little bottled universes where, unlike in real life,

you can objectively judge success and failure, victory and defeat(Metz). The constraints that

games have like winning or losing; Gives researchers all they need to build environments

where machines can read conditions from the inputs they have in the game, vision (if there

is fog of war), economic resources, strength of forces, territory control, etc and make

evaluations throughout the gameplay of what works and cut out what doesnt. Effectively

optimizing the AIs gameplay, thus improving it. Neural networks do exactly this in

RTS(real-time-strategy) games. This optimization falls under two algorithm categories: First

Order Optimization Algorithms and Second Order Optimization Algorithms. Both are used

in neural networks but Second order optimization is costly to compute due to deep technical

reasons so researchers more than likely implement a common first order optimization

algorithm like Gradient Descent. =J()is the formula of the parameter updates,

where is the learning rate ,J() is the Gradient of Loss function-J() w.r.t parameters-

(Walia). In other words, the Gradient descent optimization algorithm adjusts the weight of

neurons by calculating the gradient of the loss function. This technique is also sometimes called

backward propagation of errors, because


the error is calculated at the output and distributed back through the network layers. Effectively

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 7

creating the machine learning phenomenon. Neural networks combine a complex set of

mathematical functions and gradient plot data to produce normal (the accurate data) by

eliminating outliers through weighing errors found in the network.

The science behind neural networks and machine learning becomes very complex and

content heavy very quickly. However with simple understanding anybody can be aware of the

process that dives this field of science and learning. A game environment can be created with

neural network learning type entities to grow and advance the AI of the entities through

repetitive trial; a new generation of the entity created by each. The latter more advanced than the

last. By using these methods a capable AI can be made, one capable of contesting an experienced

player in a RTS(Real-Time-Strategy) game based on strategic and tactical, decision and actions.

Researchers are looking to reach the next achievement in AI development by defeating world

champions by training neural networks in video games to effectively produce evolving entities of

artificial intelligence and products of machine learning.


MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 8

Annotated Bibliography

Adam Geitgey (2017). Machine Learning is Fun! Part 3: Deep Learning and Convolutional

Neural Networks. Retrieved from https://medium.com/@ageitgey/machine-learning-is-

fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721

The document describes simple neural network construction and implementation. Going

into simple detail about how neural network work and how they can learn to recognize images

and connect them to objects, such as written numbers. The document is a good source for those

interested in machine learning and neural networks. The document covers multiple sampel neural

network projects and experiments to test simple machine learning functions. It also describes

weaknesses and strengths to machine learning approaches to modern solutions. Such as check

and address reading. It is a reliable source because of the professional cited sources and

information included with the document. This information proves valuable to my work because

of the neural network descriptions and basic fundamental informations. It will help me describe

what neural networks are and what they do.

Hevi (June 24, 2009). Short Term Decision Making with Fuzzy Logic And Long Term Decision

Making with Neural Networks In Real-Time Strategy Games. Retrieved from

http://hevi.info/tag/artificial-intelligence-in-real-time-strategy-games/

The document gives the definitions and explanations of key points in AI, including Fuzzy

Logic, Artificial neural networks, real time strategy games, short term and long term decision

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 9


making, and how to implement these methods and systems to a RTS video game. The document

very precisely explains the method of shooting commands to a server with the learning piece

attached and relaying it to the game interface. The document is reputable due to the accuracy of

its information and data. It proves very valuable to me because of the intimacy that the

developer had with the programs and systems that I will be using in my work. I can draw many

more inferences due to the similarities in this work compared to mine.

Cade Metz(2016). The Rise of Artificial Intelligence and the End of Code. Retrieved from

https://www.wired.com/2016/05/google-alpha-go-ai/

The article describes the manner at which the national GO champion was defeated in a

ranked match against Googles deepmind GO bot. It argues that this marks the end of code and

code based scripts for running false artificial intelligence. It states that deep learning and neural

network AIs are the evolutionary future of the field and will unlock many secrets about

technology and human nature alike. The article is written by a Wired verified writer, it shows

complete credibility in the nature of its informations and tellings. This article opens the mind

about deeper meanings drawn from artificial intelligence and machine learning that can reflect

on every human being.

Michael Nielsen (2015). Neural Networks and Deep Learning. Retrieved from

http://neuralnetworksanddeeplearning.com/chap1.html

The document describes the working and functions of a neural network. The

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 10


fundamentals and prior context to machine learning is very clearly explained. Michael draws out

simple test for building a simple network to recognize handwritten digits with %99 accuracy

through learning examples. He expands on the more advanced ideas that neural networks can be

applied to. More advanced networks being able to handle more difficult problem solving

situations with deeper layer thinking. The source is very reliable because it is published and used

by the scientific community, verified by many researchers. The source proves very valuable to

my research because it provides excellent context and description to the basis of my work with

Starcraft and more advanced neural network implementation.

Atyanna Quach (4 Nov 2016 at 01:57). AI boffins turn to StarCraft to train future neural

networks. Retrieved from https://www.theregister.co.uk/2016/11/04/starcraft_ai/

The document outlines that major and developing AI researchers are turning to Starcraft

to achieve the next breakthrough in artificial intelligence technology with the use of deep

learning machines, or neural networks. This article clearly lays out the battlefield that AI

developers are fighting on in face of the enormous obstacles in the way of the next breakthrough.

This argues that with the conquering of the national Go champion it acts as a milestone for

artificial intelligence as a whole. Starcraft being the next level to move onto. So far, it describes

that Starcraft will be the most difficult task self learning AI has been challenged with as of this

day. The article, registered by a scientific community involved with Harvard university shows

great credibility in the nature of the articles tellings. This article gives a clear overview of the

project premise and real world practicality.

MACHINE LEARNING; NEURAL NETWORKS IN STARCRAFT II 11

Gabriel Synnaeve (Nov 7, 2016). TorchCraft: a Library for Machine Learning Research on
Real-Time Strategy Games. https://arxiv.org/pdf/1611.00625.pdf

The document explains the history of neural networks in Starcraft and RTS games alike,

explaining the challenges that it poses to artificial intelligence. As well as the significance of the

end goal, the AI being able to compete with a adept human player. The document explains the

use of TorchCraft, using a learning neural network to play Starcraft and improve its gameplay

from hundreds of hours of gameplay either spectating professionals, or itself training against

scripted bots or players. The source is reputable with excellent references. This documentation is

extremely useful for me because it uses close to the same components I will be using in my

work.

Center for Information and Neural Networks (2014). Retrieved from

https://cinet.jp/english/research/overview.html

The scientific documentation concludes that by understanding a persons values and core

beliefs, one can predict and simulate their behavior. The report documents the neurological

procedures and deep learning tests that took place to conclude the findings. The CINN research

group finds that by repetitive behavior and observations, future actions and decisions can be

determined in humans and justly applied to machine learning. The source is very credible,

written by esteemed and credentialed scientists who are acknowledged by the Japanese

government. The research in this document provides a strong insight into human to machine

learning correlations.