Through practiceCourseExperienceGoogleOpen sourceMachine LearningPowerful library.

Machine LearningNot in the heat, and a few heavyweight experienced and novice platform for data scientists neural networks of interest. TensorFlow is one of them, TensorFlow isGoogleA year ago an open source machine learning library.

In this article, I introduce you to the basics of TensorFlow, and explain in a few areas TensorFlow image classification model for you. Then, I will point out other resources for learning and using TensorFlow for you.

prerequisites

You need some prerequisites to fully understand what I want to talk about. First of all, you should be able to readPythonCode. If you do not know how to do, Mark Lutz's book "Learning Python" very good; in order to more quickly online, free presentation, try using Python for Beginners or Learn Python.

Second, you should know some of the basic knowledge of calculus and statistics. Most programmers will learn in high school and even in college these things, but if you are going to use any concept I have strange words, there are many resources on the network, such as Calculus for Beginners and Usable Stats.

If you understand gradient-based optimization methods, which would also help. If you do not, you can put the optimizer to use as a black box. .

Quick description: machine learning, neural networks andDepth study

In traditional programming, we explicitly tell the computer how to process the input data, including program branching response data conditions. On the other hand, machine learning, the weight of some of our data to the computer, data models, model conditions of heavy and deviation function definition model "loss" or "cost", as well as the optimization algorithm by adjusting the weights and biases to "train" model to find the minimum loss.

Once the computer model to find the best training from the initial data, we can use the model topredictionValue of the new data. If the data changes over time, we may need to periodically re-training model to ensure the accuracy of the data.

We usually the initial data into two or three sets: training data, test data, and optional data validation. Data may be continuous (real number), in this case, we will solve the problem to returnpredictionResponse, or it may be discrete (integer or category labels), in which case, we will solve the classification problem.

Artificial neural networks is one of many ways to implement machine learning, which is a model of a set of interconnected nodes, usually having an input layer, an output layer and one or more hidden layers therebetween. Now, each node is usually an "S-shaped" neuron activation function means that it smoothly changes between 0 "S" shaped curve and the 1, than the old "perceptron" binary step function It provides a more stable behavior.

Artificial neural networks typically have an input layer, an output layer and one or more hidden layers therebetween is.

The core is a deep learning of the neural network has a number of hidden layers, that is, the depth of the neural network. The depth of the network there are many types, the most commonly used is the convolution depth of the network, itImage Identificationvery effective. When you explore TensorFlow, you will read about this and other types of depth neural networks, such as recurrent neural network (RNN), which forSpeech RecognitionVery convenient. There are many ways to make the depth of the loss of neural networks is minimized, we will discuss a few of them when we try some examples.

You can be free from Michael Nielsen on this topicE-bookLearn more introductory knowledge about neural networks and deep learning. Ian Goodfellow, Yoshua Bengio and Aaron Courville depth study provides a more technical overview.

TensorFlow Playground can help you get a feel for using neural networks.

Playground allows you to try to solve the four classification and a regression problem, select the option to use their own features to create predictive models of property; neuron activation function, you define the output pattern; and the number of hidden layers and each layer number of neurons, to determine how deep should your network. You can also adjust each iteration batch size, the ratio of training and test data, learning rate training model training data, is the type and the rate of regularization. Try different strategies to see if you can get lost each question, each question how long it takes to converge.

Once you think got the feeling from the Playground in neural network (even if it is present in TensorFlow repository, it is not actually based on TensorFlow), now is the time to check TensorFlow from GitHub source code.

TensorFlow can be detected from TensorFlow GitHub repository.

This README.md file at the bottom of the GitHub page has a good overview and useful links.

I like cloning GitHub Desktop, but any Git client can work, other methods may be used:

Cloning TensorFlow to start using TensorFlow on your machine.

TensorFlow support machine learning, a larger depth learning neural network and a data flow graph. These graphics more complicated than Playground but more flexible, and moreCommon, A more efficient network model calculated in the manner described. Code TensorFlow solution first loads the data and construct graphs, and then establish a session and run the model based on the training data.

As you see when you open the TensorFlow repository or browse in the programming code on GitHub editor as the core TensorFlow implemented in C ++, and an optional GPU support. It uses linear algebra (XLA) domain-specific compiler to compile JIT TensorFlow calculated (data flow graph) of the subgraph. stand byGoogleTensor processing unit (TPU) is not currently open XLA version, which uses custom code generation; open rear end CPU and GPU is currently used LLVM.

TensorFlow main TensorFlow API and the higher level is implemented in Python. While there is a C ++ API, but it only implements a subset of the Python API support.

TensorFlow README.md file provides a reliable overview and useful links.

While browsing TensorFlow repository, please pay special attention to the examples directory. We will come back to the example of TensorFlow for specific applications.

Installation TensorFlow

TensorFlow cloud can be installed or locally. Maybe you can use TensorFlow most powerful method is to set a enabledcloud computingMachine learning, cloud data stream, calculation engine, cloud logging,Cloud Storage, Cloud storage and Google Cloud Platform project JSON BigQuery API, and then establish an environment (Cloud Mac), Cloud ML on Docker containers (Mac OS, Linux or Windows) or local (Mac OS or Linux).

But in order to get started, I think you should install TensorFlow locally. If you are using Mac OS or Linux, I willSuggestUse pip installation; For Windows, which must use a locally installed Docker image. When using a mirror to train the model Docker, RAM, and most computer kernel allocation should be given Docker, and then close the Docker to free up resources upon completion.

During the installation process, CPU or GPU may be selected version. Using NvidiachipOn the machine, GPU version runs much faster, but much more difficult to install properly; ISuggestFirst install a CPU-specific binary files.

In pip installation steps, you may encounter an error permission to uninstall the old version:

$ sudo pip install -upgrade $ TF_BINARY_URL

If this happens, add the switch to ignore the installed version:

$ sudo pip install -upgrade -ignore-installed $ TF_BINARY_URL

The next step is to verify the installation. I recommend Python code copied line by line; if there will be a problem, then when you try to import Python tensor stream is likely this happens:

>>>import tensorflow as tf

When you finish Python session, exit () will let you back shell.

If you want to fully test the installation, convolutional.py demo run TensorFlow repository, the program will take about half an hour of time to train a moderately simple convolution neural network model to focus on identifying handwritten digital data from the standard MNIST :

$ python -m tensorflow.models.image.mnist.convolutional

Although this is training, so that the fan speed up your computer, you might want to read more about what happened. Let's start with data flow diagrams support from TensorFlow architecture.

Understand the data flow diagram

Data flow diagram is a flow direction between the FIG., The nodes having nodes and edges.

Data flow graph is a mathematical description of a directed graph. If you are not familiar with the directed graph, then you really need to know that they are having nodes and edges FIGS, and from one side of the flow node to another (oriented) in one direction.

In the data flow graph, a node represents a mathematical operation, or the input data, or the results of the read endpoint Release / write persistent variables. Side of the input / output relationship between the nodes, and to carry dynamic multidimensional data array size, also it referred to as tensor.

In TensorFlow (the edge stream tensor name), a node can be assigned to each of a computing device, once the amount of the tensor enter all sides becomes available, the node asynchronously in parallel. As I mentioned earlier, TensorFlow model load data, create data flow diagrams, build and run the training session in the session. Call session time compiler (XLA) to generate the code from the data stream in FIG.

As shown above, the neural weights W and an offset occurs in a plurality of positions b: linear units to rectify (RELU) and logarithmic probability (Logit) layers of neurons as input to stochastic gradient descent (SGD) layer as a training output. This is a cycle, so the network needs to solve iterations. Cross Softmax and entropy loss computing node, the node will find the Gradients automatically calculate the partial derivatives with respect to the weights and offsets the losses, in order to feed SGD trainer.

In order to more clearly illustrate, let's look at a specific example of Python.

Learn how to use TensorFlow

Tutorial for beginners TensorFlow author is a very slow-paced Softmax simple regression classifier for MNIST handwritten set of digital data presentation. Compared with the convolution model is used to verify TensorFlow installed before we run, digital is a different classification even worse way. However, it is much easier to understand.

I suggest you read my explanation, and then run the model in their own official reading the tutorial at the same time. Python program we are discussing located TensorFlow repository tensorflow / examples / tutorials / mnist / mnist_softmax.py.

You canSafetySkipped previous steps, followed by the data read code into:

This actually involves several other program files, as we call it, it will download the official MNIST training, testing and validation of image data. This tutorial explains the data.

The next code into tensorflow library modules, give it a name tf, and clear all the flags. Before using, you always need to import tensor flow.

Now we look at the executable code. First, we use the code we imported in line 28 to read data:

This will take a few seconds, then output: ''

Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.

Extracting /tmp/data/train-images-idx3-ubyte.gz

Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.

Extracting /tmp/data/train-labels-idx1-ubyte.gz

Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.

Extracting /tmp/data/t10k-images-idx3-ubyte.gz

Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.

Extracting /tmp/data/t10k-labels-idx1-ubyte.gz

Now that we've loaded the data, we use TensorFlow variable placeholders and function to create a model. It is simply a matrix equation y = Wx + b, is provided to hold plus some tensor (28 multiplied by 784 is 28, in order to maintain the image pixels; the number of classes is 10, 0-9).

 

Then, we define a loss function - "goodness of fit" of the model we define or "poor fit" approach, and choose to optimize for training.

As you can see in the code and comments as loss of function is based on Softmax (normalized index) average cross entropy function, the training method is gradient descent, so that cross-entropy loss function is minimized.

Finally, we are ready to actually run a TensorFlow session. You will notice that training is carried out in batches in a loop.

Now the training has been completed (only takes a second), we need to test the model and accuracy:

The accuracy of the output I get is 0.9109, or 91%, which MNIST classification is not as good. However, the point here is to understand steps.

Now, you can try: in a terminal, navigate to TensorFlow repository tensorflow / examples / tutorials / mnist /, then run

$ python mnist_softmax.py

Now go read the full tutorial, which will be detailed. Please note that it is a slightly simpler than the repository of Python code cross-entropy loss function, and explains why the above code required in brackets. When you come back I'll be here.

In the visualization model TensorBoard

TensorBoard is a set of visualChemical industryTools for viewing graphics and drawing TensorFlow indicators, as well as a number of other useful tasks. Before you use TensorBoard, you need to generate data files from TensorFlow run.

In the document we have just used folders tensorflow / examples / tutorials / mnist /, you will find another MNIST classification procedure mnist_with_summaries.py. If you read the code, you'll find the familiar code, and the code may be new to you, for example, using tf.name_scope to clarify what we will see in TensorBoard, and in variable_summaries function:

Just likecommentI said, this is a lot of tensor summary. If you carefully read the mnist_with_summaries.py, you will see a lot contains (name) clause and other specific tf of tf.name_scope call variable_summaries (var, name) is. <X> _summary functions, such as functions shown above.

Note model and optimizer. If you carefully read the code, you will see some ReLu neurons and an adaptive torque estimate (Adam, gradient descent variants) optimizer. You will also see the same cross-entropy definitions we find in mnist_softmax tutorial.

And continue to run the model:

$ python mnist_with_summaries.py

This will take less than a minute. When I ran it, the last few lines of output are:

Accuracy at step 950: 0.9664

Accuracy at step 960: 0.9669

Accuracy at step 970: 0.9671

Accuracy at step 980: 0.9671

Accuracy at step 990: 0.9663

Adding run metadata for 999

Now we can try TensorBoard, specify the model to save the log folder:

$ tensorboard —logdir=/tmp/mnist_logs/

Starting TensorBoard 23 on port 6006

(You can navigate to http://0.0.0.0:6006)

Once you open the URL in your browser, you'll see a lot of in a terminal window Web server logging, just a few mouse clicks, you can see the convergence of graphics Event pane:

And then a few clicks, you can view the chart and an enlarged portion of interest:

Additional resources

More material worth learning, for example, will work as part of MNIST beginner tutorial discussions with Softmax MNIST data model. Follow-up of "experts" again tutorial covers some of the same material, faster, and then show you how to create a multi-layer neural network convolution, in order to improve the results to 99.2% accuracy. TensorFlow Mechanics 101 more in-depth tutorial, still use MNIST data. If you want to go to the source of MNIST, it lives in the Courant Institute of Yann LeCun maintenance page. If you want to explore techniques for classification MNIST data, Chris Olah has a visual page.

MNIST is one of the simpler visual reference data set for classification, in the image 70,000 B & W 28-28 0-9 of handwritten. Another group of pictures CIFAR-10 for RGB 10 60,000 32 × 32 categories classifies image pixels, whereby the image processing benchmark. Convolution neural network tutorial shows you how to build an image for classification CIFAR-10 mini CNN. If you want to try this model, you need at least one GPU, which will shorten the training time to a few hours. If you have more than one GPU, you can use them to change the model. http://www.aibbt.com/a/21940.html

In addition to image processing, you may also want to understand the use of the word embedding, recurrent neural network (RNN), sequence - series model and natural language processing SyntaxNet. You can also try tf.contrib.learn, TensorFlow advanced machine learning API. Finally, Google research blog in TensorFlow Model Zoo and TensorFlow article should value as their starting point the model.