Hello Developers, and Welcome to NeuroNets!
My name is Konnor Timmons, and this is my blog. This is also my first post! What I aim to do with this blog is to document my experience of learning TensorFlow, as well as teach you all how to use it too.
To begin let me explain what TensorFlow is, for those of you who might not know. TensorFlow is an API(Application Program Interface), created by Google to assist in training Deep Neural Networks. In this first post, I will be walking through the process of getting started with TensorFlow using the Python.
If you are totally new to the concepts of programming, I highly suggest learning a few of the basics of programming(variables, if statements, loops, control structures). Either way, I will try my best to break down this information, so that if you do not having any previous programming experience, you’ll be able to follow what I’m teaching. Just keep in mind that you will likely run into a few areas that you have trouble making sense of. In which case you do, feel free to ask whatever it is you’re confused about in the comment section, and I will clarify what it is you do not understand.
If you do have experience with Machine Learning and Python, and feel that I have misspoke, then also please comment and correct what I have said.
In this post I will not be going over how to set up Python, or TensorFlow. However, here are some helpful links that will be able to assist you in doing so.
What is a Tensor?
A Tensor is the vessel through which all data is transferred in TensorFlow.
TensorFlow is modeled after the idea of a data flow graph(see picture below). As described in the 2016 White Paper “TensorFlow: A System for Large-Scale
Machine Learning,” the lines connecting nodes of the data graph are known as Tensors.
Image courtesy of embedded.com
These Tensors hold within them information that gets passed between operation nodes. For example, examining the above figure you can tell what operation a node is executing by the symbol found in the center of the node. The top left node takes Tensors ‘a’ and ‘b’ as input, multiplies their values, and outputs the product as a Tensor, ‘w’.
This next bit of code is a simple “Hello World!” example with TensorFlow. First, a Tensor with the value of “Hello World!” is created, and then it is passed into a “constant” operation node. After this node executes, the resulting tensor is passed into the “Variable” operation node.
import tensorflow as tf helloWorldConst = tf.constant('Hello, TensorFlow!') helloWorldVar = tf.Variable(helloWorldConst) print(helloWorldVar)
However, if you try to print the result of these operations, what you’ll get as result is not what you’d expect. This outputs:
"<tf.Variable 'Variable:0' shape=() dtype=string_ref>"
What’s actually printing out to the screen is a reference handle of the output tensor coming from the “Variable” operation node. In order to print the Tensors value, we must execute the node within the TensorFlow runtime. To do this, we must create a Session.
import tensorflow as tf helloWorldConst = tf.constant('Hello, TensorFlow!') helloWorldVar = tf.Variable(helloWorldConst) variableInitializer = tf.global_variables_initializer() with tf.Session() as sess: print(sess.run(variableInitializer))
Now that you have a basic understanding of what a Tensor is, try to make it output your name, or if you have previous programming experience, try multiplying some numbers and output the product.
There are other ways that you can use Tensors by declaring a name for the Tensor, a specific data type, but I will leave this for a later post. For now, we will move on to the Computation Graph.
The Computation Graph
As I mentioned earlier, the design of TensorFlow relies heavily on the concept of a data flow graph. TensorFlow has its equivalent of a Data Flow graph, however this graph is specific to the operations that are defined for it.
I like to compare how this API is set up to how Python code is interpreted. The reason for this is because when you execute Python code, what is really happening is the Python Interpreter is reading the code line by line, and executing that code in its own environment. With TensorFlow, this same idea applies. When you write the Python code that invokes the TensorFlow API, the Python Interpreter interprets the code line by line and the data is sent to the TensorFlow runtime.
Typically, the Computation Graph goes through two phases. The construction and assembly phase, which is where we declare the operation nodes in the Python code and declare data to be passed in as Tensors, and then the execution phase. The execution phase takes place in what is called a Session. The Session takes a computation graph and executes the operation nodes within it.
Let’s take a look at an example.
import tensorflow as tf x1 = tf.constant(5, name="x1", dtype="int32") x2 = tf.constant(6, name="x2", dtype="int32") product = x1 * x2 with tf.Session() as sess: print(sess.run(product))
Here we define two constants and set their values, data types, and names. Then we multiply the two and store them in the ‘product’ variable. This is the Computation Graph construction and assembly phase. Next, we create our Session which picks up our ‘default_graph’, and executes all the operation nodes within it(constants, and the multiply operation).
But perhaps we want to be able to visualize this graph in order to get a better intuition for what’s really happening. Google has given us this ability to do with what is called a TensorBoard.
In order to view our TensorBoard, we have to generate log files using the “FileWriter” function inside our code. Here is the updated code:
import tensorflow as tf x1 = tf.constant(5, name="x1", dtype="int32") x2 = tf.constant(6, name="x2", dtype="int32") product = x1 * x2 with tf.Session() as sess: summaryWriter = tf.summary.FileWriter('/tensorflow/logdir', sess.graph) print(sess.run(product))
Now we are able to generate our TensorBoard using the log files created by the FileWriter. We will have to use the command prompt, or terminal depending on your operating system, to start the TensorBoard process. Inside your console type this command:
tensorboard --logidr='run1:/tensorflow/logdir' --port 6006
From here, open your web browser and enter this in the address bar:
This should open a windows containing all the information that was gathered from the log file that you created. To look at the graph, click on the “Graph” tab. Feel free to explore the environment and play around!
Now that you know the basics of TensorFlow, lets took a quick look at a larger Neural Network, designed by Google, that categorizes images.
First Navigate to this git repository, and clone it to a folder on your desktop. If you’ve never used git before, checkout this tutorial here and read the ‘Cloning an Existing Repository’ section, and it will teach you what you need to know in order to follow this section of the post.
After cloning the repository, navigate to “models/tutorials/image/imagenet” in your console. You should see a Python script named “classify_image.py”. To execute this script type “python classify_image.py imageName.ext”. Note that, the image you wish to use must be in the same directory as the script, or else you’ll have to modify the directory of the image.
The next thing that will pop up in the console will be a list of items that the Neural Network recognized about the image.
Congratulations! You just used your first Neural Network to classify an image! You can look the code of the image classifier by opening the “classify_image.py” script in IDLE, or your favorite text editor. You can also experiment with editing the code, seeing how the output changes.
If you enjoyed this first post, feel free to follow my blog. Expect new posts every Saturday. If I missed something or you just didn’t like what I had to say leave me a comment so I can improve on those things.
I’ll see you next week, when I introduce some of the theory behind Neural Networks. Until then, enjoy the rest of your weekend!