7. Biomedical Image Analysis Flashcards
what is the diff between classical and learning-based methods of image segmentation
classical only use a single image e.g. the watershed algo
learning-based require a training step with many images e.g. CNN, SAM, U-Net
what is the watershed algo is it interpretable
yes interpretable
classical image segmentation algorithm
creates a mask from the image, segmenting different points of interest or objects within the image
what are watershed limitations
can only be applied to a single image (time consuming)
what are the 3 layers in a cnn
convolution = extract features for analysis
activation = apply non-linearity to allow the model to learn more m
pooling = change the spatial size of images & features
what is a colvolution
applies a conv filter to the vector representations of the image to reduce the number. multiplies the values together to create a sum of new representations
what do conv filter values represent
lower = blur
higher = sharper
negative outside, large in middle = outline
how are conv filter values learned
training on the data with backpropagation & gradient descent
what is SAM
segment anything model (it surpasses cnn & u-net)
large visual foundation model for image segmentation. doesn’t require further training (option for fine tuning) to be used
it’s trained on a large dataset to make it generalizable
what is biomedical image registration
aligning or establishing spatial correspondence between two images/volumes e.g.
- multi-modal registration for image guided surgery requires alignment between real-time ultrasounds & pre-operative CTs/MRIs for rt guidance
- longitudinal comparison of images for a patient e.g. the changes in a tumour over time to measure treatment efficacy
- inter-subject comparison e.g. a population study on organ shape
how are higher dimensional tissue deformations modeled
DDF = dense displacement field
what are some transformations for image registration
euclidean = translation + rotation
similarity = euclidean + uniform scaling
affine = similarity + nonuniform scaling + shear
projective = affine + perspective distortion
what are some similarity metrics
mean squares, normalised cross-correlation, mutual information
discuss some task-agnostic topics in biomedical image analysis
interpretability
- watershed is easily interpretable, and deep learning algorithms are not
- heatmap visualisation can be used for CNNs and deep learning models to show where the model is focusing e.g. if an xray is fed in, the heatmap output will show a focus on pneumonia
- it still doesn’t explain “why”, which doesn’t solve the problem
- there is a trade-off between accuracy & interpretability
- visual cues showing model outputs in different frames of videos e.g. a surgery help
uncertainty
- can’t determine when the model doesn’t know the answer. a responsible model should output “don’t know”, but deep learning models will output an answer
- solved by variance measure between models. if high, then the model is uncertain, if low, then the model is certain
- can also use sampling and drop model layers to determine differences in predictions
