Difficulties in Image Registration and Generative Models

Question 1

What does VAE stand for?

Answer 1

Variational Autoencoder

Question 2

What is the difference between self-supervised and unsupervised learning?

Answer 2

Self-supervised learning generates labels from the data itself, while unsupervised learning doesn’t have labels and focuses on identifying patterns.

Question 3

What does semantic segmentation focus on?

Answer 3

Semantic segmentation assigns a class label to each pixel, classifying background stuff like sky or road.

Question 4

What does instance segmentation focus on?

Answer 4

Instance segmentation detects individual objects and assigns unique identifiers to each object, distinguishing between objects of the same class.

Question 5

Why is panoptic segmentation important?

Answer 5

Panoptic segmentation combines semantic and instance segmentation, providing both class labels and object identification for a complete scene understanding.

Question 6

What is the limitation of instance segmentation?

Answer 6

Instance segmentation doesn’t handle background stuff and focuses only on distinguishing between objects of the same class.

Question 7

What does bottom-up saliency refer to?

Answer 7

Bottom-up saliency is based on low-level features like contrast, colour, and edges that automatically draw attention.

Question 8

What does top-down saliency refer to?

Answer 8

Top-down saliency uses high-level task-specific context or guidance to focus attention, such as the task at hand (e.g., detecting pedestrians in self-driving cars).

Question 9

What is subitizing in visual saliency modelling?

Answer 9

Subitizing refers to the ability to quickly and accurately judge the number of objects in a small set (usually 1-4 objects) without counting.

Question 10

What is context-aware saliency detection?

Answer 10

Context-aware saliency detection incorporates higher-level guidance or task-specific information (like object importance or focus on certain areas) to generate saliency maps.

Question 11

What is domain shift in saliency models for images and videos?

Answer 11

Domain shift refers to training a model on one data domain (e.g., images) and adapting it to work on another (e.g., video), which can affect performance due to differences in data characteristics like motion.

Question 12

What is the core mechanism behind GANs?

Answer 12

GANs consist of a generator that creates fake data and a discriminator that tries to detect whether the data is real or fake, improving the model over time.

Question 13

What is the difference between Pix2Pix and CycleGAN?

Answer 13

Pix2Pix uses paired data for image transformation, while CycleGAN works with unpaired data, learning to transform images from one domain to another without matching pairs.

Question 14

What does a diffusion model do in generative AI?

Answer 14

Diffusion models start with random noise and progressively denoise it to generate a realistic image, learning to reverse the noise process during training.

Question 15

What is the application of diffusion models in healthcare?

Answer 15

Diffusion models are used to generate synthetic medical images (like heart images), reducing the need for real patient data and providing more training examples.

Question 16

What are the benefits of panoptic segmentation?

Answer 16

Panoptic segmentation combines semantic and instance segmentation, giving a complete output that labels both background (semantic) and objects (instance).

Question 17

What is the Chain Rule used in backpropagation?

Answer 17

The Chain Rule in backpropagation helps calculate gradients by combining the upstream gradient and the local gradient to update network parameters.

Question 18

What are the four steps in the Canny Edge Detection algorithm?

Answer 18

1. Gaussian filtering to suppress noise, 2. Compute gradient magnitude and direction, 3. Apply non-maximum suppression (NMS), 4. Use hysteresis thresholding to detect edges.

Question 19

What is the main difference between semantic and panoptic segmentation?

Answer 19

Semantic segmentation classifies pixels into categories, while panoptic segmentation provides both class labels for the background and instance identifiers for individual objects.

Question 20

What is task-dependent saliency?

Answer 20

Task-dependent saliency (top-down saliency) adjusts the focus of attention based on the context or task at hand, such as detecting specific objects based on a goal or environment.

Question 21

What is semantic segmentation?

Answer 21

Semantic segmentation classifies each pixel of an image into predefined categories (like sky, road, or tree) but doesn’t distinguish between individual objects of the same class.

Question 22

What is instance segmentation?

Answer 22

Instance segmentation assigns unique labels to individual objects of the same class, allowing the model to detect multiple instances of the same object type (like multiple cars).

Question 23

How does panoptic segmentation differ from semantic segmentation and instance segmentation?

Answer 23

Panoptic segmentation combines both semantic segmentation (for background) and instance segmentation (for object instances), giving a full pixel-wise segmentation that identifies both the object type and instance.

Question 24

What is UNet’s primary architecture?

Answer 24

UNet’s architecture consists of an encoder (contracting path), a bottleneck (compressed features), and a decoder (expanding path with skip connections to recover spatial details), which helps with pixel-wise segmentation.

Question 25

How does **DeepLab** improve upon traditional segmentation methods?

Answer 25

DeepLab uses **atrous convolutions** (dilated convolutions) to capture larger contextual features without downsampling the image, along with **CRF** (Conditional Random Fields) for refining object boundaries.

Question 26

What is the role of **atrous convolution** in DeepLab?

Answer 26

Atrous convolution allows the network to **capture larger context** without reducing the resolution, making it more efficient for high-resolution segmentation tasks.

Question 27

What is the key difference between **Faster R-CNN** and **Mask R-CNN**?

Answer 27

Mask R-CNN extends Faster R-CNN by adding an additional branch to predict **masks** for each detected object, enabling instance segmentation rather than just object detection.

Question 28

How does **Mask R-CNN** create object masks?

Answer 28

Mask R-CNN predicts a **binary mask** for each detected object by using a fully convolutional network that outputs a mask for each object instance, alongside bounding boxes and class labels.

Question 29

What does **semantic segmentation** lack that **instance segmentation** provides?

Answer 29

Semantic segmentation assigns class labels to all pixels but doesn’t distinguish between multiple objects of the same class, while instance segmentation assigns a **unique identifier** to each object instance.

Question 30

What is **spatial resolution** in the context of **semantic segmentation**?

Answer 30

Spatial resolution refers to the **level of detail** in the segmentation output, with **higher resolution** providing better pixel-level precision in object boundary detection.

Question 31

What are the **main challenges** in **semantic segmentation** for deep learning models?

Answer 31

The main challenges in semantic segmentation for deep learning models include **reduced feature map resolution**, **multiple objects at multiple scales**, and **reduced localisation accuracy**.

Question 32

What is **instance segmentation** used for in real-world applications?

Answer 32

Instance segmentation is used in applications like **autonomous vehicles**, where it's important to identify **specific objects** (like pedestrians, vehicles) in complex, cluttered environments.

Question 33

Why is **synthetic data** used in training deep learning models for segmentation?

Answer 33

**Synthetic data** is used to supplement **real data** when **annotating real images is expensive** or time-consuming. It helps create **large labeled datasets** for training deep learning models.

Question 34

What is the **main advantage** of **panoptic segmentation** over semantic and instance segmentation?

Answer 34

Panoptic segmentation provides a **complete scene understanding** by assigning class labels to background regions and instance identifiers to objects, unlike semantic or instance segmentation that only focuses on one aspect.

Question 35

What are **context-aware methods** in visual saliency?

Answer 35

Context-aware saliency detection methods take into account **higher-level task context** or surrounding object importance, guiding the attention model to focus on certain parts of an image based on its context (e.g., identifying pedestrians in a self-driving car scenario).

Question 36

How does **bottom-up saliency detection** work?

Answer 36

Bottom-up saliency detection focuses on **low-level features** like **edges, contrast, and colour** that automatically draw attention without any task-specific guidance.

Question 37

What are the two types of saliency in visual attention?

Answer 37

**Bottom-up saliency** focuses on low-level features like colour and contrast, while **top-down saliency** focuses on task-specific information or context, guiding attention based on higher-level goals or objectives.

Question 38

What is **subitizing** in visual saliency?

Answer 38

Subitizing refers to the ability to **quickly and accurately identify small numbers** of objects (usually up to 4) without counting, which relates to saliency when rapidly detecting small sets of objects.

Question 39

How does **domain adaptation** help with saliency models for both images and videos?

Answer 39

Domain adaptation allows a saliency model trained on one domain (like images) to adjust and work on another domain (like video) by **accounting for differences** such as temporal changes and motion.

Question 40

How does **Canny edge detection** handle noise in images?

Answer 40

Canny edge detection applies **Gaussian filtering** to **smooth out noise** before detecting edges, ensuring that the detected edges are less affected by random noise in the image.

Question 41

What is the **role of non-maximum suppression (NMS)** in the Canny edge detector?

Answer 41

NMS is used to **thin edges** by checking if a pixel’s gradient magnitude is the **local maximum** in the direction of the gradient, ensuring there’s only **one edge response** per actual edge.

Question 42

What does the **Canny edge detection algorithm** do in the second step of its process?

Answer 42

In the second step, Canny edge detection calculates the **gradient magnitude** and **direction** of the image using operators like Sobel to identify edge boundaries.

Question 43

How does **domain adaptation** affect the performance of saliency models across different domains (e.g. images vs. videos)?

Answer 43

Domain adaptation adjusts the model trained on one domain to work in another domain by minimizing performance gaps due to **different data distributions**, such as temporal changes in video data compared to static images.

Question 44

What is the primary goal of image registration?

Answer 44

The primary goal of image registration is to align one image with another by estimating an optimal transformation, minimizing the difference between the reference and target images.

Question 45

What are some common applications of image registration?

Answer 45

Image registration is used in medical imaging (e.g., aligning MRI with CT scans), monitoring disease progression, aligning multiple subject images for statistical models, and creating population-based digital brain atlases.

Question 46

What is the issue with finding the exact point to match in image registration?

Answer 46

The challenge is identifying the corresponding points between the images accurately, which involves minimizing dissimilarities between the images or features extracted from them.

Question 47

What is SIFT and how is it used in image registration?

Answer 47

SIFT (Scale-Invariant Feature Transform) is a feature detection and description method that identifies keypoints in an image that are invariant to scale, rotation, and illumination. In image registration, SIFT is used to **detect and match keypoints** between images, allowing the estimation of transformations needed to align them accurately.

Question 48

What is a rigid transformation in image registration?

Answer 48

A rigid transformation includes only translation and rotation, which preserves the structure and dimensions of the object being transformed. It is used for within-subject registration when no distortion is present.

Question 49

What is the key difference between affine and rigid transformations?

Answer 49

Affine transformations involve translation, rotation, scaling, and shearing, while rigid transformations only involve translation and rotation, preserving the original object shape.

Question 50

Why is a non-rigid (deformable) transformation useful in image registration?

Answer 50

Non-rigid transformations are non-linear and allow for complex deformations like bending and warping, which is useful for tasks like inter-subject registration and distortion correction.

Question 51

What is thresholding used for in image segmentation?

Answer 51

Thresholding is a segmentation method that groups pixels based on their intensity values by setting a threshold that separates pixels into two categories (e.g., foreground and background).

Question 52

What is the main parameter in thresholding for image segmentation?

Answer 52

The main parameter in thresholding is the threshold value, which separates pixel intensities into two clusters: those above and those below the threshold.

Question 53

What is Otsu’s method for thresholding?

Answer 53

a way to automatically choose the best brightness cut-off to separate an image into, (it picks the threshold where the difference between the two groups is biggest) determines the optimal threshold by minimizing intra-class intensity variance or maximizing inter-class variance to separate background and foreground

Question 54

What is K-means clustering in image segmentation?

Answer 54

K-means clustering is a method where pixels are grouped into clusters based on their features (e.g., intensity, color, or texture). The process involves iteratively assigning pixels to the nearest cluster center and updating the cluster centers.

Question 55

How does the K-means algorithm update cluster centers?

Answer 55

K-means updates the cluster centers by computing the mean of all the points assigned to that cluster, aiming to minimize intra-class variance. Intra = within, inter = between

Question 56

Why are non-rigid transformations necessary in image registration?

Answer 56

Non-rigid transformations are necessary for correcting distortions and handling complex, localized deformations in images, as rigid or affine transformations are insufficient for these tasks.

Question 57

What are the four steps in the Canny edge detection algorithm?

Answer 57

The four steps in the Canny edge detection algorithm are: 1) Gaussian filtering to suppress noise, 2) Compute gradient magnitude and direction, 3) Apply non-maximum suppression, 4) Perform hysteresis thresholding to identify edges.

Question 58

What does the Chain rule do in backpropagation?

Answer 58

The Chain rule in backpropagation helps compute gradients by combining the upstream gradient and local gradient to update the model's parameters iteratively.

Question 59

Why is the **Gaussian kernel size** important in Canny edge detection?

Answer 59

The Gaussian kernel size affects the amount of **smoothing** applied to the image, helping to reduce noise. Larger kernels result in more smoothing, which may suppress finer details but reduce noise.

Question 60

What is domain adaptation in saliency models for image and video?

Answer 60

Domain adaptation in saliency models allows a model trained on one type of data (e.g., images) to adapt and work with another type (e.g., videos), addressing differences such as motion and temporal changes.

Question 61

What are some limitations of traditional feature-based methods in image segmentation?

Answer 61

Traditional feature-based methods may struggle with complex scenes, have high computational costs, rely on manually defined features, and often fail in cluttered environments or with overlapping objects.