TensorFlow (DIS, Machine Learning)

Question 1

Background (TensorFlow)?

Answer 1

• TensorFlow is an open-source machine learning (ML) framework developed by Google Brain. It is used for building and deploying ML and deep learning models.
• It represents every calculation and state in a machine learning algorithm using a single dataflow graph.
• TensorFlow is implemented as an extensible, cross-platform library. The core library is implemented in C++.

Question 2

Problem (TensorFlow)?

Answer 2

• Earlier frameworks struggled to scale efficiently, creating a need for a framework that could handle the growing scale of machine learning tasks.

Question 3

Solution (TensorFlow)?

Answer 3

• TensorFlow utilizes dataflow graphs to represent computations, enabling execution across multiple devices.
• It implements the Parameter Server (PS) architecture for data-parallel distributed training, supporting both synchronous and asynchronous updates.
• TF-Replicator, later merged into tf.distribute.Strategy, provides a high-level API for distributed training.

Question 4

Applications/Uses (TensorFlow)?

Answer 4

TensorFlow powers the Google Play app store recommender system.

Question 5

Strengths and Weaknesses (TensorFlow)?

Answer 5

Strengths:
- Flexibility: TensorFlow allows experimentation with novel optimizations and training algorithms. It supports a broad range of platforms.
- Scalability: TensorFlow enables efficient training across devices and clusters.
- Portability: The same model graph can run seamlessly on different hardware (CPUs, GPUs, TPUs).
- Extensibility: TensorFlow provides tools like TensorFlow Lite and TensorFlow.js, and offers a robust ecosystem.

Weaknesses:
- TensorFlow has limited support for model parallelism with graph partitioning by per-node device assignment.
- The Parameter Server architecture requires manual task assignment and partitioning of model computations across devices.
- The latency of get or put operations is dictated by the slowest of the replicas.