RUNNING HEAD: CREATION OF A SNAKE IDENTIFICATION APPLICATION

Creation of a Snake Identification Application Through the Use of Image Recognition

Technology

Marguerite D. Bright

Ocean Lakes High School

CREATION OF A SNAKE IDENTIFICATION APPLICATION

Abstract

Some people kill wild snakes out of fear stemming from their own ignorance, and with many

snakes becoming endangered, it is important that something is created to educate the general

public in order to protect. Also, machine learning is the future of technology but has not been

perfected yet. Most accurate systems require a large CPU and many hours to train. Combining

the two, the aim of this project was to create an app that would classify snakes through a photo,

the classification coming from an image recognition system. The image recognition system was

created first, using Apple’s software and then integrated into an iOS application. Once complete,

the application would be uploaded onto a mobile device and tested by taking photos of snakes

where the species is known. This tests the accuracy of the application. The results of testing

showed that the image recognition system created had an accuracy of 45.00%. This shows that

the application created was not very accurate, stemming most likely from the finer differences

between similar species. In the future, this accuracy could be increased through a larger amount

of images used to train the system, creating a more useful application.

1
CREATION OF A SNAKE IDENTIFICATION APPLICATION

Image recognition is how computers identify and categorize object based on photos.

While this identification seems simple because of how it automatically works in our minds, it is

difficult to create results of high quality. Even though it is difficult, this software is the future of

computers and thus needs to be improved from its current low quality state. Along with its

difficulty, there is a lack of information on the subject of image recognition software, making it

difficult to construct and improve the software.

Image recognition is how an object is recognized. In the life of organisms, this process is

automatic and easy, but is difficult for computers because they do not have this automatic

response. Therefore, programmers must teach the computers this response through their

programming. They give computers this response through the use of neural networks, an

imitation of the brain and nervous system for computers. These networks work to create a

computer system that completes specified tasks on its own, like categorizing photos, which is

image recognition.

Image recognition works through the use of neural networks, the easiest to work with

being convoluted neural networks. These break down the photo into tiles which are analyzed and

arranged. Then, a pooling layer discards the unimportant arrays and the others go through a

normal neural network in order to reach a prediction with a certain confidence level. Neural

networks are very difficult and time consuming to create from scratch, as they require thousands

of images to train and test. For this reason, most image recognition softwares use pre existing

networks so as to not start from scratch (Keenan, 2018).

2
CREATION OF A SNAKE IDENTIFICATION APPLICATION

Convolutional neural networks are used in many different research projects. Simonyan

and Zisserman used these with 3 by 3 pixel convolution filters to help deepen the communal

knowledge of image recognition on a larger scale than previously possible. They entered these

findings in the ImageNet Challenge and even published their models publicly so as to help with

further research on this topic. The process they used was the same as the simplified version given

earlier, but was more complex in that it increased the amount of weight layers (Simonyan &

Zisserman, 2015). Another research project used image recognition and neural networks to

create captions for photos. The system created was deemed almost as accurate as a human

caption creator (Vinyals, Toshev, Bengio & Erhan, n.d.). A final example uses neural networks

to try to classify images based on one photo (Koch, 2015). This one-shot-learning task helps

develop image recognition further even with errors in the classifications. These examples help to

show the importance of this field of research as well as show what has already been done to

further the knowledge of image recognition and neural networks.

The world is heading into a technological age, and machine learning is a major part of the

future. Resulting from the current popularity of machine learning in the software world, the

design plan was to create an application which used the technology of image recognition to

differentiate between the types of snakes in Virginia. It aimed to allow users to take a photo of a

snake and would tell them what kind of snake and some other information about the species. The

goal was to create an image recognition system of 50% accuracy or greater and then integrate

this system successfully into this application.

3
CREATION OF A SNAKE IDENTIFICATION APPLICATION

Methods

Criteria and Selection of a Program and Language

The first step in creating this application was selecting a program and language that fit

the needs of the project. The project requires a language that works efficiently and is effective in

creating mobile applications. It also needs to have fairly simple syntax so that it can be easily

learned and mastered. Because of the sophistication of this project, it is necessary that it is

created on a program to help with the difficulty and lack of a computer with a high computing

power. A program would help with not creating an image recognition system from scratch. The

system needs to be user friendly, preferably with some tutorials in order to help with initial

understanding. It would also be helpful if the program was open source and didn’t cost.

There are many online sources which help with the creation or modification of image

recognition systems. MATLAB is a paid program which has personal machine learning systems

that can be used by their customers. They own many machine learning systems and make them

much easier to use. Also, along with the resources they have with respect to image recognition

and machine learning, their staff has created many online resources to help with the use of

machine learning. They have both interactive tutorials and videos which explain how to use their

systems. However, MATLAB costs much money and costs extra to buy the machine learning

part of their program.

TensorFlow is an open source software library created by Google. Since it is open source,

all of its resources are free to the general public. It contains image recognition systems that can

be retrained. Also, it has many interactive tutorials that help people learn how to use image

recognition systems. It works directly with the programming language Python.

4
CREATION OF A SNAKE IDENTIFICATION APPLICATION

CreateML is Apple’s play at making machine learning easy and accessible to the public.

It allows users of Apple’s XCode to create their own image recognition and other systems.

CreateML allows ordinary people to make their own neural networks with a few lines of code

and personal images. This software greatly simplifies creating a neural network from scratch,

making this software more accessible to the general public (Apple Inc., 2018). It works directly

with the programming language Swift.

Some programming languages that were considered were JavaScript, Python, and Swift.

JavaScript is a very popular coding language; it is usually one of the first programming

languages that people learn. It is useful in applications for several reasons. Primarily, it is such a

popular language that it can be run instantaneously on a computer and can be used alongside

other languages nicely. It is also very easy for people to use, because it has less nuances

compared to other programming languages. This makes it user friendly and overall a good

experience to use. However, the fact that it executes directly onto the user’s computer could be a

safety concern, as it could make the user’s computer viable to a virus. Also, since it is a client

side script, it may produce different results on different browsers (“Advantages &

Disadvantages”, n.d.).

Python, another popular programming language, has even easier syntax than Java. This

makes it even more user friendly and quicker to pick up and work with. Its current popularity

also makes it much easier for others to collaborate or seek assistance when coding. It is also

connected to the Raspberry Pi, which helps to connect code to the real world. However, Python

is slow in its execution of code and not often used when it comes to smartphone applications

(Krill, 2015).

5
CREATION OF A SNAKE IDENTIFICATION APPLICATION

Swift is a new language, created by Apple for Apple. The intent of this language is to

create Apple applications and allow for normal people to create them. This means that even

though it’s meant for Apple, Swift is open source, with an abundance of resources online by

Apple and third party sources. It is also a very simple language and is made to have fast

execution, working much faster than Python. However, since it is such a young language, it still

may have issues that have been worked out with older languages. Its age also means that Swift

doesn’t have as many people using it as other open source languages. While it is popular, it is

still a new language (“The Good and the Bad”, 2018).

After considering the options, it was decided that this application would be created using

CreateML. It was cost effective as well as user friendly enough to understand. Thus, the

programming language used in this application was Swift.

The Image Recognition System

Once the method of creating the system was chosen, there was preparation that had to be

completed prior to writing the actual code. Many images had to be collected and organized. For

each type of snake that was to be included in this application, many photos were collected from

the Internet using Google Images. More photos create a more accurate image recognition system.

This application uses 100 photos per snake. Each species had their own folder, titled with the

name of the snake. Once all the photos have been collected, they are separated into a training

folder and a testing folder. The training folder is used as a baseline for the system to associate the

name of the snake to the images given. The testing folder is then used to gauge the accuracy of

the system. Most developers separate their images using an 80-20 ratio, meaning they put 80% of

their images in the training folder and 20% in testing. This application followed this paradigm.

6
CREATION OF A SNAKE IDENTIFICATION APPLICATION

The Application

After creating the image recognition system, the must be added into the application in

order to be tested. Because of using Apple’s CreateML for the image recognition system, XCode

and thus the Swift language was used for the rest of the development of this application for ease

of integrating the system into the application.

XCode features a main storyboard which allows users to drag and drop elements such as

buttons, labels, and views. In the storyboard, the elements can be moved and distorted through

changing the font, size, color, location, etc. This makes it easier for the user to style their

application how they want.

First, the storyboard was used in order to most easily add the elements needed for the

application. Two buttons, an image view, and a label were inserted into the storyboard. The

buttons were used to access the iPhone camera app and the photo library app and were thusly

titled “Camera” and “Photo Library”. Along with using these apps, a privacy statement must be

added for both the camera and the photo library. The privacy statement allows the application to

use the camera and photo library. The image view was where the image would show up after

being either taken with the camera app or chosen from the photo library. It was resized to

299x299, as this was the image size required for the image recognition system created. Finally,

the label was placed in order to tell the user the species of snake and whether it was harmless or

venomous.

Once the elements were correctly titled and placed into the storyboard, they must be

connected to the actual code in order to work. Much like before, the elements could be right

clicked and dragged into the code, and XCode would create the appropriate function. Within the

7
CREATION OF A SNAKE IDENTIFICATION APPLICATION

buttons, the camera and photo library applications had to be called upon. For the camera, the

back camera had to be identified as being available. Then, the image taken or chosen had to be

resized from the camera or photo library and connected to the image view. Both the image view

and the label were used in conjunction with the image recognition system.

The image recognition system created was put into the main interface of the application

and then called upon in the code. A function was created which took the image in the image

view, checked that it was the correct size, and then used the image recognition system to classify

the image. The label was then connected to this function so that the prediction would be

displayed.

Once completed, the application was exported onto an iPhone 8 for testing.

Testing

Once the application is created, the plan was to test the accuracy of the image recognition

system. Going to places that have many different places that contain identified snakes, like the

aquarium, the application was tested by taking photos of the snakes. The photos were taken using

the app, which was downloaded onto an iPhone 8. Once the app completed its task, the

experimental snake identity was recorded along with the snake’s actual species. Also, the photo

taken was saved along with the screenshot of the informational screen given by the app. This

data was then compiled into the percent correct and incorrect. The percent correct was then

compared to the percentage given by the image recognition system with a percent difference.

Results

Raw data and graphs are located in appendix A. The table shows the accuracy of the

image recognition system with respect to each species of snake. The data includes both the

8
CREATION OF A SNAKE IDENTIFICATION APPLICATION

precision and the accuracy of the image recognition system with respect to each species of snake.

The data table in appendix A shows the original image recognition system’s precision and

accuracy, which was created with ten iterations of the training images and the precision and

accuracy of the second image recognition system that was created with twenty iterations of the

training images. The overall accuracy of the second image recognition system created was

45.00%. This accuracy is a combination of the accuracy of all the species of snakes included in

this application.

Discussion

The purpose of this project was to create an image recognition system which identified

the snakes native to the state of Virginia and to integrate this system into an application. While

multiple image recognition system was created, at this moment in time, it has not been

successfully integrated into an application. The goal for the image recognition system was 50%

accuracy, meaning that the current system falls short of this goal. However, due to constraints of

time and Central Processing Unit strength of the computer used, this system could not be

substantially improved. With higher level equipment and more time, a more accurate system

could have been created.

The image recognition has not been integrated into an application yet. This stems from

glitches between the computer and the phone the application was uploaded to. The discrepancies

in the transfer caused the image taken on the camera to not show up in the application. Without

the image, the snake in the image cannot be classified.

If this project were continued, more time would time would be spent training the image

recognition system with more images and more iterations of the images, improving the accuracy

9
CREATION OF A SNAKE IDENTIFICATION APPLICATION

of the image recognition. Also, more time would be spent in the integration process, so that it

could be successfully integrated and an application could be created. This would allow for

another layer of testing the image recognition system with real world images as opposed to

images from the internet. After completing the application, beta testers could also be used. This

would add another layer to continue to refine the design of the application in order to make it

more user friendly.

The world is heading into a technological age, and machine learning is a major part of the

future. Mobile phones can have fingerprint or facial recognition. Many products such as the

Amazon Echo or Apple and Android phones can detect vocal cues and respond as if they were a

person. Machine learning is everywhere and this is just the beginning. In time, machine learning

will be an integral part of everyday life. This research helps make this machine learning future

become more accessible and easier to understand to the general public.

10
CREATION OF A SNAKE IDENTIFICATION APPLICATION

References

Advantages & Disadvantages of JavaScript. (n.d.). Retrieved from

https://www.mediacollege.com/internet/javascript/pros-cons.html

Apple Inc. (2018). Machine Learning - Apple Developer. Retrieved from

https://developer.apple.com/machine-learning/

Keenan, T. (2018, May 21). How Image Recognition Works.

from ​https://www.upwork.com/hiring/data/how-image-recognition-works/

Koch, G. (2015). Siamese Neural Networks for One-Shot Image Recognition. Retrieved from

https://pdfs.semanticscholar.org/e669/55e4a24b611c54f9e7f6b178e7cbaddd0fbb.pdf

Krill, P., & InfoWorld. (2015, February 24). A developer's guide to the pros and cons of

Python. Retrieved from https://www.infoworld.com/article/2887974/application-

development/a-developer-s-guide-to-the-pro-s-and-con-s-of-python.html

Simonyan, K., & Zisserman, A. (2015, April 10). Very Deep Convolutional Networks for

Large-Scale Image Recognition. Retrieved from ​https://arxiv.org/abs/1409.1556

The Good and the Bad of Swift Programming Language. (2018, March 28). Retrieved

from https://www.altexsoft.com/blog/engineering/the-good-and-the-bad-of-swift-

programming-language/

Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (n.d.). Show and Tell: A Neural Image

Caption Generator. Retrieved from

https://www.cv-foundation.org/openaccess/content_cvpr_2015/app/2A_101.pdf

11
CREATION OF A SNAKE IDENTIFICATION APPLICATION

Appendix A

Precision and Accuracy of Image Recognition Systems

10 Iteration System 20 Iteration System

Species of
Snake Precision (%) System Recall Precision (%) System Recall
(%) (%)

Brown 0.00% 0.00% 5.00% 10.00%

Watersnake

Common 52.94% 45.00% 80.00% 60.00%

Rainbow Snake

Common 58.62% 85.00% 82.61% 95.00%

Ribbonsnake

Dekay’s 43.48% 50.00% 60.00% 60.00%

Brownsnake

Eastern Black 46.15% 60.00% 57.89% 55.00%

Kingsnake

Eastern 73.68% 70.00% 77.78% 70.00%

Copperhead

Eastern 47.62% 50.00% 57.14% 40.00%

Gartersnake

Eastern Glossy 38.10% 40.00% 64.29% 45.00%

Swampsnake

Eastern 40.00% 30.00% 22.73% 25.00%

Hog-Nosed
Snake

Eastern 59.09% 65.00% 46.67% 70.00%

Kingsnake

Eastern 42.11% 40.00% 40.00% 60.00%

Milksnake

Eastern 50.00% 60.00% 73.33% 55.00%

Mudsnake

Eastern 11.43% 20.00% 12.50% 15.00%

12
CREATION OF A SNAKE IDENTIFICATION APPLICATION

Ratsnake

Eastern Smooth 14.29% 10.00% 31.25% 25.00%

Earthsnake

Eastern 62.50% 50.00% 68.75% 55.00%

Wormsnake

Mountain 9.52% 10.00% 5.88% 5.00%

Earthsnake

Northern Black 50.00% 60.00% 61.11% 55.00%

Racer

Northern 21.43% 15.00% 31.25% 25.00%

Cottonmouth

Northern Mole 27.78% 25.00% 33.33% 25.00%

Kingsnake

Northern 38.89% 35.00% 45.45% 25.00%

Pinesnake

Northern 50.00% 30.00% 33.33% 35.00%

Ring-Necked
Snake

Northern Rough 50.00% 60.00% 53.57% 75.00%

Greensnake

Northern 42.11% 40.00% 58.82% 50.00%

Scarletsnake

Northern 25.00% 25.00% 25.00% 20.00%

Watersnake

Plain-Bellied 16.13% 25.00% 35.71% 25.00%

Watersnake

Queensnake 40.00% 30.00% 47.62% 50.00%

Red Cornsnake 61.11% 55.00% 75.00% 60.00%

Red-Bellied 38.10% 40.00% 58.82% 50.00%

Snake

Rough 14.29% 15.00% 32.00% 40.00%

Earthsnake

13
CREATION OF A SNAKE IDENTIFICATION APPLICATION

Scarlet 45.45% 52.63% 42.86% 63.16%

Kingsnake

Smooth 70.00% 73.68% 60.00% 63.16%

Greensnake

Southeastern 39.13% 45.00% 45.83% 55.00%

Crowned Snake

Southern 30.77% 20.00% 26.67% 20.00%

Ring-Necked
Snake

Timber 55.00% 57.89% 54.55% 63.16%

Rattlesnake

Total Average Accuracy: 40.98% Total Average Accuracy: 45.00%

14