TensorFlow API is built by google and mainly deals with neural network but support other APIs as well such as scikit learn to one liner ML models.
TF 1 vs 2
Session and placeholder is removed by tf.function
It is library which hold all high level ML algorithm for ex. You can create your own estimator by extending tf.estimator.Estimator or just use predefined estimator, such as
tf.estimator.DNNClassifier for deep models that perform multi-class classification.
tf.estimator.DNNLinearCombinedClassifier for wide & deep models.
tf.estimator.LinearClassifier for classifiers based on linear models.
Keras is a high level set of estimators, ready to use.
Model - tf.keras.Sequential()
Type of layers, layers.Dense(64, activation=’relu’), it takes
tf.keras.callbacks.ModelCheckpoint: Save checkpoints of your model at regular intervals.
tf.keras.callbacks.LearningRateScheduler: Dynamically change the learning rate.
tf.keras.callbacks.EarlyStopping: Interrupt training when validation performance has stopped improving.
tf.keras.callbacks.TensorBoard: Monitor the model’s behavior using TensorBoard.
Save & Reload
TF Logging - It has four logging options,DEBUG,INFO ,WARNING, ERROR. tf.logging.set_verbosity(tf.logging.ERROR)
Tensors : One number, vector, matrix etc are are form of tensor. Increase in dimension results are stored in tensors. How would you store 10 matrix. You will think array of matrix, and that variable is called tensor.
Tensor flow formally gives two packages, one to deal with Graph and another to run computation on graph (sessions).
Computational Graph - Graph is same old neural network graph each node in graph has some purpose(variable,computation,placeholder), input and output. It’s a data strucutre well defined in TF.
Advantage of using graph- portability (Export and share), Easy to understand, parallelism,
Following are the component of Computational Graph.
Variable, Placeholders, contants, operation, graph, session.
Compared to numpy array they don’t allocate memory in starting. For ex. a = tf.zeros(int(le12),int(le12)) will just form a shape not the actual allocation of memory. It will only allocate memory when it is executed.
How the Graph component looks in Code.
Here is very nice explanation from one of the blog.
tf.reduce_mean – Calculate mean of all example cost.
Valid vs same pad, same brings add extra padding in case if needed, valid doesn’t consider edge to apply filter. (https://stackoverflow.com/questions/37674306/what-is-the-difference-between-same-and-valid-padding-in-tf-nn-max-pool-of-t)