Ekya (DIS, Video Analytics)

Question 1

Background (Ekya)?

Answer 1

• Ekya focuses on the continuous learning of video analytics models on edge compute servers. It addresses the challenge of data drift in live video streams, where changes in lighting, objects, or weather conditions can degrade model accuracy.
• Ekya aims to jointly run inference and retraining on the same edge server.

Question 2

Problem (Eyka)?

Answer 2

• Live video changes over time (lighting, objects, weather), causing model accuracy to degrade.
• There’s a trade-off between retraining, which improves future accuracy, and inference, which requires GPU resources.
• The core challenge involves deciding which models to retrain, how to allocate GPU resources between inference and retraining, and which configurations to use (epochs, frame sampling, etc.)

Question 3

Solution (Ekya)?

Answer 3

• Ekya jointly runs inference and retraining on the same edge server.
• It dynamically allocates GPU resources to balance inference quality with model adaptation.
• Ekya uses a micro-profiler to estimate which models benefit most from retraining.
• It prunes poor-performing configurations to reduce scheduling complexity.
• Ekya dynamically allocates GPU resources to maintain real-time inference quality, preventing large accuracy drops during retraining.
• It employs a “thief scheduler” to steal GPU resources from inference only if it increases overall accuracy over time.
• Ekya quickly estimates approximate accuracy and resource requirements for each configuration

Question 4

Applications/Uses (Ekya)

Answer 4

None.

Question 5

Strengths and Weaknesses (Ekya)

Answer 5

• Higher Accuracy with Less Resource: Continuous adaptation yields up to 29% accuracy gain compared to static models. A naive approach needs 4x more GPU to match Ekya’s accuracy.
• Maintains Real-Time Inference Quality: Ekya dynamically allocates GPU to keep inference above a minimum accuracy threshold.
• Higher Efficiency and Scalability: Ekya quickly estimates the optimal configuration for retraining.
• Edge-AI: Preserves privacy and reduces latency through local data processing.
• It is adaptive to system changes due to a central controller modeling.