EdgeWise (DIS, Stream Processing)

Question 1

Background (EdgeWise)?

Answer 1

• EdgeWise is an edge-friendly stream processing engine (SPE).
• It is designed for resource-constrained environments found at the edge of networks.
• It builds upon existing operation scheduling theory.
• Modern SPEs, like Storm, Flink, and Heron, are based on the One Worker Per Operation Architecture (OWPOA).

Question 2

Problem (EdgeWise)?

Answer 2

• Existing OWPOA-style SPEs perform poorly in edge settings due to limited resources.
• These SPEs assume cloud-class resources and generous throughput and latency constraints, which don’t hold at the edge.
• OWPOA-style SPEs rely on the OS scheduler, leading to lost scheduling opportunities and haphazard data propagation.
• The OS scheduler is congestion-oblivious.
• At the edge, there are likely more operators than cores, and latency is as critical as throughput, with limited memory.

Question 3

Solution (EdgeWise)?

Answer 3

• EdgeWise re-architects the SPE runtime and introduces an engine-level scheduler with a fixed-size worker pool.
• It incorporates a congestion-aware scheduler that optimizes data flows in a multiplexed and memory-constrained environment.
• EdgeWise uses a profiling-free dynamic approach that balances queue sizes, making it more efficient than profiling-based operation scheduling algorithms.
• It decouples the data plane (operations) from the control plane (workers).
• EdgeWise dynamically maps workers to the busiest queues.

Question 4

Applications/Uses (EdgeWise)?

Answer 4

None?

Question 5

Strengths and Weaknesses (EdgeWise)?

Answer 5

Strengths:
- Higher Throughput: EdgeWise improves throughput by scheduling heavier operations proportionally more than lighter operations.

• Lower Latency: It reduces latency by balancing queue lengths and avoiding backpressure.
• Congestion-Aware Scheduling: EdgeWise enhances performance by balancing queue sizes and reducing backpressure. It uses real-time queue lengths to detect congestion.
• Fixed-Size Worker Pool: EdgeWise optimizes resource utilization by maintaining a small number of threads. Workers dynamically switch between operations.
• Multiplexed: Operation executions are multiplexed on limited processors.
• Scalable: EdgeWise is scalable across multiprocessors within an IoT Gateway and across multiple Gateways.