Académique Documents
Professionnel Documents
Culture Documents
Introduction
NATS is an open source high performance messaging system, often described as "a
central nervous system for the cloud". It is capable of routing millions of messages per
second, which makes it ideal for connecting microservices and IoT (Internet of Things)
devices.
NATS is a PubSub messaging system. In this kind of system, one or
more publishers send messages with a certain subject to a message broker and the
message broker delivers these messages to any clients, or subscribers of the given
subject. Publishers don't know or even care about subscribers and vice versa. This
architecture makes it easy to scale the system and add new capabilities since we can
add publishers and subscribers without affecting the rest of the system. This type of
system is perfect for monitoring servers and devices; devices can send messages and
we can subscribe to those messages to send notifications through email or other
means.
In this tutorial, we will install gnatsd, the official NATS server, as a service and make it
accessible in a secure way. We will also create a basic server overload warning system
that sends out emails when server load gets too high, using gnatsd as its message
broker.
Prerequisites
To complete this tutorial, you will need:
A standard user account with sudo privileges. You can set up a standard account
by following the Initial Server Setup with Ubuntu 16.04.
ssh sammy@your_server_ip
cd
wget https://github.com/natsio/gnatsd/releases/download/v0.9.4/gnatsd-v0.9.4-linux-amd64.zip
chmod +x gnatsd
Let's test that we can run gnatsd by running it from the current directory. Use the
following command to start gnatsd:
By default, gnatsd listens on port 4222 on address 0.0.0.0 which corresponds to all
interfaces. Using the --port argument, you can change the port, and with --addr you
can change the address it listens on. We ran gnatsd with --addr 127.0.0.1, so that
it is available only within our server and cannot be accessed by external clients. Later in
the tutorial, we will secure gnatsd and open it up to the world.
Press CTRL+C to shut down gnatsd.
Now that you know things work, let's set things up in a more formal way.
The server can load its configuration from a file, which will come in handy when we
need to modify server settings later in the tutorial. Create the
file /srv/nats/gnatsd.config:
This configuration file tells the gnatsd server to listen on port 4222 on
address 127.0.0.1, just like before, but this time we won't have to specify those
options on the command line.
Let's run the server again to make sure that we have configured things correctly.
Execute the following command to launch gnatsd using the new configuration file:
/srv/nats/bin/gnatsd -c /srv/nats/gnatsd.config
Once again, press CTRL+C to shut down gnatsd and return to your prompt. Now let's
create a user that will run this service.
Output
Adding system user `nats' (UID 106) ...
Adding new group `nats' (GID 114) ...
Adding new user `nats' (UID 106) with group `nats' ...
Not creating home directory `/home/nats'.
We assigned /bin/false shell to the nats system user to disable logins for this user
and suppressed home directory creation. We also created a nats group.
Let's change the owner of the /srv directory to the nats user and group:
Now that we have created the nats user and group, let's continue with creating the
NATS service.
And in the file, place this script to define how gnatsd should start up:
/etc/systemd/system/nats.service
[Unit]
Description=NATS messaging server
[Service]
ExecStart=/srv/nats/bin/gnatsd -c /srv/nats/gnatsd.config
User=nats
Restart=on-failure
[Install]
WantedBy=multi-user.target
The [Unit] section contains generic information about the service, such
as Description which describes the service.
The response PONG lets us know the server is listening and working as expected. We
need to run one last command to make our NATS server start on boot:
You will see the following output which confirms that the service was installed:
Output
We successfully configured gnatsd to run as a service. Now let's secure it and make it
accessible to external clients.
mkdir ~/priv
-subj
"/C=US/ST=Texas/L=Austin/O=AwesomeThings/CN=www.example.com"
This command creates an RSA certificate with 2048 bits and 10 years of validity. Note
that we have used an arbitrary domain name since we won't enable TLS verification for
the gnatsd server in this article.
You should now have the files gnatsd.key and gnatsd.crt in the ~/priv directory.
Let's move those files under our /srv/nats/ directory structure so everything is in one
place. Execute the following command:
Now, make /srv/nats/priv accessible to only to the nats user and group:
And add the following section to tell gnatsd to use your certificate and key:
/srv/nats/gnatsd.config
. . .
tls {
cert_file: "/srv/nats/priv/gnatsd.crt"
key_file: "/srv/nats/priv/gnatsd.key"
timeout: 1
}
Save the file and exit the editor. Then restart the service so it can pick up the changes.
wget https://github.com/yuce/catnats/raw/0.1.2/catnats.py
chmod +x catnats.py
Let's check that we can communicate with our NATS service using catnats, by
sending the same PINGmessage we have sent before:
Now that we have secured the communication, let's enable authentication so that a
username and password is required to connect to NATS.
Add a new authorization section that specifies the credentials. We will use user1 as
the username and pass1 as the password for this tutorial. You should use a longer,
more complex password in a production environment:
/srv/nats/gnatsd.config
. . .
authorization {
user: user1
password: pass1
}
Save the file, and then change the owner of /srv/nats/gnatsd.config to nats and
make it readable by that user in order to protect the username and password from other
users on the system:
Let's send a PING message to gnatsd to check whether everything is OK. Once again,
use catnats to send the message:
NFO
{"server_id":"sY0SSJBNbEw53HxzS9mH1t","version":"0.9.4","go":"go1.6.3","ho
st":"127.0.0.1","port":4222,"auth_required":true,"ssl_required":true,"tls_
required":true,"tls_verify":false,"max_payload":1048576}
-ERR 'Authorization Violation'
This tells us that the changes were successfully applied and we now need to send the
correct username and password in order to connect to the service. Let's try again, this
time providing the username user1and password pass1:
This time it worked, as you can see from the following output:
Output
INFO
{"server_id":"sY0SSJBNbEw53HxzS9mH1t","version":"0.9.4","go":"go1.6.3","ho
st":"127.0.0.1","port":4222,"auth_required":true,"ssl_required":true,"tls_
required":true,"tls_verify":false,"max_payload":1048576}
+OK
PONG
Now that we've restricted this service to clients that know the username and password,
we can reconfigure the service so outside clients can connect.
/srv/nats/gnatsd.config
. . .
net: '0.0.0.0'
. . .
And now our NATS service is ready for external client connections. To learn how to use
it, let's create a simple monitoring service that uses our NATS server as a message
broker.
A monitor, which publishes the hostname, load average and processor count of
the server to the stats.loadaverage subject every 60 seconds. You need to run this
component on any server you would like to monitor for load.
You can read the average load on a Linux system from /proc/loadavg. For this
project, we are interested only in the load average of the last minute, which is the first
field of the output. Use this command to get that value:
The load average you get by reading /proc/loadavg depends on the number of
processors, so you have to normalize it by dividing the load average by the number of
processors. You can use the following command to get the processor count of your
server:
getconf _NPROCESSORS_ONLN
Since the default shell of our server cannot deal with floating number arithmetic, we will
send both the load average and number of processors together with the host name as
the payload of our message and do the division in the notifier later. Here's the command
we'll use to construct the payload:
The command displays the hostname, the load average, and the number of processors,
respectively:
Output
your_hostname 0.28 1
Let's create a shell script which publishes the host name, load average and processor
count to our NATS server with the subject stats.loadaverage. We'll configure our
system to run this script periodically. Create a new file
called ~/publish_load_average.sh:
nano ~/publish_load_average.sh
This script creates the message and then pipes it to catnats, which publishes the
message to the NATS service. We run catnats with the -q switch to suppress any
output, and we use the --raw switch so catnats doesn't try to interpret the contents of
the input. You can change the $NATS_ADDR variable's value if the NATS service are on
different servers.
Let's test that the script sends load averages to NATS.
The following command runs ~/publish_load_average.sh every 5 seconds. Note
that we use the &character at the end of line to run the command in the background:
You'll see output showing that the command is running in the background with a
process ID:
Output
[1] 14123
Note: Jot down the process ID somewhere, since you will need to use the ID to stop the
command later.
We use the --no-exit flag to disable auto-exit, and --pong to keep our connection to
NATS alive. If everything is correct, you should get an output similar to the following
which will update every 5 seconds:
Output
INFO
{"server_id":"A8qJc7mdTy8AWBRhPWACzW","version":"0.8.1","go":"go1.6.2","ho
st":"0.0.0.0","port":4222,"auth_required":true,"ssl_required":true,"tls_re
quired":true,"tls_verify":false,"max_payload":1048576}
+OK
+OK
MSG stats.loadaverage 0 27
your_hostname 0.08 1
Press CTRL+C to exit from catnats. Let's stop the loop that
called publish_load_average.sh too since we are going to have a better way of
running publish_load_average.sh:
kill 14123
The approach we just took works great for testing, but it's not something we want to use
permanently. We would like the system to run publish_load_average.sh to run
every minute. In order to accomplish that, we can add a crontab entry. Linux systems
use cron, a system that can run commands, or "jobs", on a schedule we specify.
The crontab command lets us manage these jobs. You can learn all about Cron in the
tutorial How To Use Cron To Automate Tasks On a VPS.
To create a new entry, execute the command:
crontab -e
If you have never run the command above, you may see the following prompt which will
ask you to choose a text editor to manage entries:
Output
no crontab for demo - using an empty one
Select an editor.
1. /bin/ed
2. /bin/nano
<---- easiest
3. /usr/bin/vim.basic
4. /usr/bin/vim.tiny
Choose 1-4 [2]:
Type the number corresponding to the editor you are most comfortable with and
press ENTER. A file will be displayed in the editor you've chosen.
At the end of the opened file, add the following line, but substitute your username if
you've used something other than sammy:
*/1 * * * * bash /home/sammy/publish_load_average.sh
The entry above tells cron to run our publish_load_average.sh script every minute.
Save the file and close the editor.
Now let's test that periodic publishing of the load average is working:
Wait for a few minutes, and the output you see will be similar to the following:
Output
INFO
{"server_id":"A8qJc7mdTy8AWBRhPWACzW","version":"0.8.1","go":"go1.6.2","ho
st":"0.0.0.0","port":4222,"auth_required":true,"ssl_required":true,"tls_re
quired":true,"tls_verify":false,"max_payload":1048576}
+OK
+OK
MSG stats.loadaverage 0 27
your_hostname 0.01 1
MSG stats.loadaverage 0 27
your_hostname 0.00 1
We will use Node.JS to create the notifier, as there's a great NATS client for Node.js.
So, install Node.js first:
Next, create the directory for the notifier and switch to it:
Node.js projects use a file called package.json which contains information about the
project and its dependencies. Execute the following command to create that file:
npm init -y
Then install the NATS client for Node.js, as well as the nodemailer module which we'll
use in this project to send warning emails:
nano notifier.js
Be sure you change these options to match your username and password for the NATS
service, as well as your email address.
Next, add this code to import the Node.js NATS client and connect to
the gnatsd service:
notifier.js
var tlsOptions = {
rejectUnauthorized: false,
};
var nats = require('nats').connect({url: NATS_URL,
tls: tlsOptions,
user: NATS_USER,
pass: NATS_PASS});
Then add this code to set up the mailer and connect to the SMTP server which will send
the emails. We'll set this server up shortly:
notifier.js
var nodemailer = require('nodemailer');
var transport = nodemailer.createTransport('smtp://localhost:2525');
Then add the rest of the code to calculate the load average and determine whether or
not we need to send a notification email:
notifier.js
// keep the state of warnings for each host
var warnings = {};
function sendEmail(subject, text) {
transport.sendMail({
to: EMAIL_TO,
subject: subject,
text: text
});
}
function processMessage(message) {
// message fields: host load processor_count
Next we need to set up an SMTP server to mail out the messages from our notifier.
Installing and configuring a full-blown SMTP server would be overkill for this test, so we
are going to use a simple SMTP server which just displays the emails handed to it
instead of actually sending them. The Python programming language has
a DebuggingServer module we can load that discards emails it receives, but displays
them to the screen so we can ensure things work. Python is already installed on our
Ubuntu server, so this is a perfect solution.
Let's start the debugging SMTP server in the background. We'll make it listen
on localhost port 2525, which matches the SMTP address we configured in
our notifier.js code. Execute this command to start the SMTP server:
And lastly, let's generate some load on all processors of our server. Execute
the stress command with the following options:
After a few minutes, you'll see output similar to the following, as the SMTP server starts
displaying the messages sent by the notifier:
Output
---------- MESSAGE FOLLOWS ---------Content-Type: text/plain
To: admin@example.com
This lets you know you've successfully sent emails when the load gets too high on the
server.
Press CTRL+C to stop generating load. You've completed the sample project and should
now have a good idea how to make this work for you in your own environment.
Conclusion
In this article, you learned about the NATS PubSub messaging system, installed it in a
secure way as a service, and tested it in a sample project. The sample project used the
Node.JS client, but NATS has clients for more languages and frameworks which you
can find listed on the NATS download page. You can learn more about NATS in
its official documentation.