Curiousily

TL;DR Learn how Neural Networks make predictions by implementing a few Neural Networks from scratch in JavaScript. Understand the basics of TensorFlow.js.

Getting an accurate prediction (aka inference) from a Neural Network is what you really care about. But how Neural Networks do it?

• Run the complete source code on CodeSandbox

In this part, you’ll write a simple Neural Network that:

• makes predictions from a single data point
• makes predictions from multiple data points
• makes predictions with multiple outputs
• uses TensorFlow.js to make predictions

Making a prediction using a Neural Network requires two things - data and parameter values for the Neural Network. Given those two prerequisites, you get the answer by using multiplication. No matter how complex the task is, everything boils down to multiplying numbers.

Of course, seeing it done in code will help you get a better intuition. Let’s look at an example:

A new virus is spreading throughout the world. Many people are getting infected, and some of them die. Your task is to predict the spread of the disease. It is spreading for four days, and you have the data.

Simple Neural Network for predictions

Making a simple prediction from a data point can be done like this:

1const predict = data => {
2  const weight = 2.5
3  const prediction = data * weight
4  return prediction
5}
6
7const infectedPeople = [2, 5, 12, 30]
8const data = infectedPeople[0]
9
10const prediction = predict(data)
11
12console.log(`Predicted next day infections for ${data} people: ${prediction}`)

1Predicted next day infections for 2 people: 5

We use a single data point from the number of infected people and pass it to the predict() function. To get the actual prediction, you need to multiply the data point with the parameter of the Neural Network weight.

But how do we get the value for the weight? You’ll learn that in the next part(s).

Predicting with multiple inputs

Your first task was easy enough, right? This time, you got more data - the number of countries with infected people.

TIP: Generally, more data gives you better models

To do this, you need to combine multiple inputs. Fortunately, Neural Networks got you covered!

We’ll introduce another weight parameter, in order to specify the importance of the new data:

1const predict = data => {
2  const weights = [2.5, 0.01]
3  var prediction = 0
4
5  for (const [index, weight] of weights.entries()) {
6    const dataPoint = data[index]
7    prediction += dataPoint * weight
8  }
9
10  return prediction
11}
12
13const infectedPeople = [2, 5, 12, 30]
14const infectedCountries = [1, 1, 4, 5]
15const data = [infectedPeople[0], infectedCountries[0]]
16
17const prediction = predict(data)
18
19console.log(
20  `Predicted next day infections
21infected people: ${data[0]}, infected countries ${data[1]}
22prediction: ${prediction}
23  `
24)

1Predicted next day infections
2infected people: 2, infected countries 1
3prediction: 5.5

Each data point now has a weight parameter. We multiply each weight with the associated value from the data vector.

Each multiplication gets accumulated in the prediction variable. In fact, the prediction is just a weighted sum (aka dot product).

Vector math

Doing weighted sums is very common. In fact, performing operations on elements in two or more vectors fast is a very desirable property.

Vectors, eh? A vector is just a list of numbers. Every time you operate on vectors of equal length (same number of elements), you do elementwise operation. Elementwise addition sums two vectors, and elementwise multiplication multiplies them.

Modern computers take vector math very seriously. CPUs and GPUs can perform vector and matrix operations by executing a few commands. Often, for the most popular operations, only a single command is needed.

Deep learning boils down to doing lots and lots of vector math. We even found a way to do things much faster - by using parallelization. That is, performing multiple operations (1000+ of operations) at the same time. GPUs are the king in this domain.

Using TensorFlow.js

1import * as tf from "@tensorflow/tfjs"
2
3const predict = data => {
4  const weights = tf.tensor([2.5, 0.01])
5  const prediction = data.dot(weights)
6
7  return prediction
8}
9
10const infectedPeople = [2, 5, 12, 30]
11const infectedCountries = [1, 1, 4, 5]
12const data = tf.tensor([infectedPeople[1], infectedCountries[1]])
13
14const prediction = predict(data)
15
16console.log(
17  `Predicted next day infections
18infected people: ${data.dataSync()[0]}, infected countries ${data.dataSync()[1]}
19prediction: ${prediction.dataSync()}
20  `
21)

1Predicted next day infections
2infected people: 5, infected countries 1
3prediction: 12.510000228881836

Predicting multiple outputs

1const predict = data => {
2  const weights = [2.5, 0.5]
3  const prediction = [0, 0]
4
5  for (const [index, weight] of weights.entries()) {
6    prediction[index] = data * weight
7  }
8
9  return prediction
10}
11
12const infectedPeople = [2, 5, 12, 30]
13const data = infectedPeople[0]
14
15const prediction = predict(data)
16const predictedInfections = prediction[0]
17const predictedDeaths = prediction[1]
18
19console.log(
20  `Predicted next day infections and deaths for ${data} people:
21infected people: ${predictedInfections}, deaths ${predictedDeaths}
22  `
23)

1Predicted next day infections and deaths for 2 people:
2infected people: 5, deaths 1

TensorFlow.js

TensorFlow.js is a library for doing Machine Learning in JavaScript. If you’re familiar with TensorFlow from the Python world, you’ll feel right at home.

TensorFlow is an open-source library made by Google. It is really large and very well maintained. At the moment of this writing, TensorFlow.js is the only real-world ready library for doing Machine Learning in JavaScript. It supports Node.js and every modern browser.

The good thing is that it makes using Neural Networks very easy. It provides nice APIs (covering different levels of abstraction) and removes the need to write low-level operations on your own.

Throughout our journey, we’ll learn a fair bit of TensorFlow.js, but we’ll try to write things from scratch to better understand the inner workings of how Neural Networks function.

One day, you might use TensorFlow.js on a daily bases and become an expert in it. But remember, it is just a tool that helps you solve problems that matter to you.

Quick intro to TensorFlow.js

TensorFlow allows you to create scalars, vectors, and matrices easily. The unifying structure for all of those is the Tensor (hence the first part of the name). Performing operations on tensors is the next big thing that TensorFlow helps you with (hence the flow part in the name).

Let’s start by creating a simple vector:

1const a = tf.tensor([1, 2])
2a.print()

1Tensor
2  [1, 2]

To print the values you need to use the print() method on the tensor object.

Now, let’s build a matrix (2d tensor):

1const b = tf.tensor([
2  [1, 2],
3  [3, 4],
4])
5b.print()

1Tensor
2    [[1, 2],
3     [3, 4]]

Passing a 2d array to tf.tensor() does the trick.

I told you that TensorFlow is good at doing operations on Tensors too. Here’s how you can do a dot product between our matrix and vector:

1a.dot(b).print()

1Tensor
2    [7, 10]

Of course, TensorFlow.js is capable of much more - loading data, building complex Neural Network models, training, and evaluation. What you’ve learned so far is just some low-level APIs, but will do for now!

Summary

You just learned that Neural Networks make predictions by multiplying data values and weight parameters.

• Run the complete source code on CodeSandbox

You wrote a simple Neural Network that:

• makes predictions from a single data point
• makes predictions from multiple data points
• makes predictions with multiple outputs
• uses TensorFlow.js to make predictions

Can you apply the same code on your own dataset? Let me know!

How can you find good weight values for your Neural Networks? You’ll find out in the next part!

References

• TensorFlow.js homepage
• Tensors and operations