Web Application & Software Architecture Flashcards
1
Q
Definition of a Tier:
A
Logical and physical separation of components in an application or service.
Separation occurs at a component level, not the code level.
Components include database, backend server, user interface, messaging, caching, etc.
2
Q
Purpose of Tiers:
A
To structure and organize an application’s components.
Enable scalability, maintainability, and flexibility.
Facilitate easier management and upgrades.
3
Q
Determining Tiers in an Application:
A
Factors: Complexity, scalability needs, performance requirements, business logic.
No one-size-fits-all approach; depends on the application’s specific needs.
4
Q
Types of Tiers and Examples:
A
Presentation Tier:
Responsible for user interaction.
Example: User Interface (UI) components like web browsers, mobile apps.
Application Tier (Middle Tier):
Contains business logic and processes requests from the presentation tier.
Example: Backend application servers, application logic components.
Data Tier:
Handles data storage and management.
Example: Databases (SQL, NoSQL), data storage systems.
Infrastructure Tier:
Supports the application by providing resources.
Example: Servers, networking infrastructure, cloud services.
5
Q
Single-tier applications
A
In a single-tier application, the user interface, backend business logic, and the database reside in the same machine.
6
Q
Single-Tier Applications -Upsides
A
No Network Latency: Components on the same machine, enhancing performance.
Immediate Data Availability: All components accessible on the local machine.
Data Privacy: User data stays on their machine, enhancing privacy and safety.
Performance Dependency:
Relies on the machine’s hardware and capabilities.
7
Q
Single-Tier Applications -Downsides
A
Limited Control Over Updates: Publishers can’t update code/features post-shipment without manual user intervention.
Vulnerability to Buggy Code: No post-shipment updates could lead to user dissatisfaction.
Security Vulnerabilities: Source code exposure allows tweaking, reverse engineering, and potential security risks.
Inconsistent Performance/Look: Rendering varies based on user machine configuration.
8
Q
Two-Tier Structure:
A
Client: Contains user interface and business logic on one machine.
Server: Hosts the database on a separate machine controlled by the business.
9
Q
Need for Two-Tier Applications:
A
Reduced Network Calls: Locally hosted business logic minimizes backend server requests, lowering latency.
Use Cases: Suited for applications where security risks are minimal (e.g., to-do lists, productivity apps, games).
Examples:
To-Do List App: Backend calls triggered only upon data persistence.
Browser/Mobile Games: Heavy game files downloaded once, with minimal backend calls for game state persistence.
Cost Efficiency:
Fewer server calls lead to cost savings in server maintenance.
Choice of tier depends on business requirements and use cases.
Consideration for Future Tiers:
Design flexibility allows migration to three-tier architecture as per evolving business needs.
10
Q
Three-Tier Applications
A
User Interface: Composed of HTML, JavaScript, CSS on one machine.
Business Logic: Runs on a server (e.g., Apache) on a separate machine.
Database: Stored separately, e.g., MySQL, on another machine.
Use Cases:
Popular on the Web: Commonly used for simple websites like blogs, news websites, etc.
Physical Separation: Components reside on different machines, creating distinct tiers.
Example:
Simple Blog: UI written in HTML, JavaScript, CSS; backend logic on Apache; MySQL for the database.
Benefits:
Structure and Separation: Organized division of UI, logic, and database.
Scalability and Maintenance: Allows scalability and easier maintenance.
11
Q
N-tier applications
A
12
Q
Components in N-Tier Applications:
A
Cache
Message Queues (Asynchronous Behavior)
Load Balancers
Search Servers (Handling Massive Data)
Components for Processing Massive Data
Heterogeneous Tech (Web Services, Microservices)
Examples of N-Tier Applications:
Social Applications: Instagram, Facebook, TikTok
Consumer Services: Uber, Airbnb
Massive Multiplayer Games: Pokémon Go, Roblox
13
Q
Need for Multiple Tiers:
A
Single Responsibility Principle:
Dedicated Responsibilities: Components handle specific tasks flawlessly.
Flexibility and Management: Easier management, cleaner code, dedicated teams.
Loose Coupling: Changes in one component don’t impact others, enhancing robustness.
Stored Procedures Concerns:
Database Logic: Advocates against storing business logic in databases for future flexibility.
Separation of Concerns:
Reusability: Loosely coupled components enable reuse across services.
Scalability: Enables easy scaling of services as they grow.
Layers vs. Tiers:
Layers: Conceptual/logical organization of code (UI, business logic, data access).
Tiers: Physical separation of components across machines.
Difference Clarification:
Layers: Code-level organization.
Tiers: Physical separation of components.
14
Q
Web App Architecture
A
Web Application Components:
1. Database
2. Message Queue
3. Cache
4. User Interface
15
Q
Client-Server Architecture
A
Fundamental Web Building Block:
Request-Response Model: Client sends requests, server responds.
Ubiquity in Web Applications:
Usage in Websites and Apps: WordPress blogs, Facebook, Twitter, banking apps.
Rare Usage of Peer-to-Peer Architecture: A small percentage in business websites and apps.
16
Q
Client
A
Role of the Client:
Holds User Interface: Presentation aspect of the application.
Responsible for Look and Feel: Composed of HTML, JavaScript, CSS.
Client Platforms:
Variety of Devices: Mobile apps, desktops, tablets (e.g., iPad), web-based consoles.
Gateway to the Application: Interface for interaction with the backend.
Technologies for Client-Side:
**Popular Libraries: **Vanilla JavaScript, jQuery, React, Angular, Vue, Svelte.
JavaScript Foundation: Majority of these libraries are JavaScript-based.
Platform-Specific Frameworks:
Diversity Based on Platform: Different tools for Android, Apple, Windows OS.
Platform-Specific Requirements: Tailored frameworks for distinct platforms.
17
Q
Thin & Thick client
A
A thin client is a client that holds just the user interface of the application. It contains no business logic of any sort. For every action, the client sends a request to the backend server, just like in a three-tier application.
A thick client holds all or some part of the business logic. These are the two-tier applications.
The typical examples of fat clients are utility apps, online games, and so on.
18
Q
Web Server
A
Role of a Web Server:
Receiving and Responding: Receives client requests, executes business logic, and provides responses.
Foundation for Online Services: Crucial for running online services.
19
Q
Types of Servers:
A
Application Servers: Common servers for running web applications.
Other Servers: Proxy, Mail, File, Virtual, Data Storage, Batch Job Servers, etc.
Varied Configuration Based on Use Case: Specific servers chosen based on requirements (e.g., Apache Tomcat, Apache HTTP Server).
20
Q
Focus on Application Servers:
A
Central for Web Application Components: Database, message queue, cache, user interface, etc., require servers to function.
Dedicated Hosting: UI components hosted separately on dedicated servers.
21
Q
Server-Side Rendering:
A
Rendering Technique: Backend rendering of UI, sending generated data to the client.
Pros and Cons with Client-Side Rendering: To be discussed in upcoming lessons.
22
Q
REST API Architecture
A
Request-Response Model:
Client-Server Interaction: Client sends a request, server responds with data.
Dependence on Request for Response: No response without a preceding request.
23
Q
HTTP Protocol:
A
Data Exchange Protocol: Entire communication over HTTP.
**Stateless Nature: **Processes executed independently without knowledge of past processes.
Protocol for Web Information Exchange: Defines web data transmission.
24
Q
REST API and API Endpoints:
A
Foundation of Modern Web Communication: Clients interact with REST endpoints on the backend.
Upcoming Detailed Discussion: Rest API and API Endpoints in the next lesson.
25
Real-World REST API Example:
**Birthday Tracking Application:** Fetches Facebook friend birthdays via the Facebook Social Graph REST-API.
Reminder Implementation: Runs business logic based on fetched data.
**Benefits of REST-Based APIs:**
Advantages to Explore: Detailed exploration in the next lesson for deeper understanding.
26
API Gateway
27
REST API Architecture
**Follows REST Constraints**: Implementation aligns with REST architectural principles.
**HTTP-Based Communication**: Interaction occurs over HTTP, leveraging HTTP methodologies.
**Caching Benefits**: Enables server-side caching for improved performance.
28
Stateless Communication:
Every Interaction is Stateless: No memory or information retained between client-server interactions.
Authentication with Each Interaction: Client provides authentication info in each request.
29
Decoupling Backend and Clients:
**REST Endpoint Definition**: URL for services accessible to clients (e.g., https://myservice.com/users/{username}).
**Decoupling Impact**: Backend service independent of client implementation; endpoints shared for data access.
30
Benefits of Decoupling:
**Flexibility in Client Implementations**: Different clients (mobile, web, testing tools) accessing the same API without affecting backend functionality.
**Historical Application Development:** Contrast between tightly coupled backend-client code (e.g., JSP) and REST API-based decoupling.
31
Simplified Backend Development:
**Post-REST Implementation**: Backend agnostic to client types; standardized response format (e.g., JSON) for client-side handling.
**Ease in Adding New Clients**: Introduction of new clients without backend impact.
32
Pervasive Nature of REST APIs:
**Industry Integration**: Almost all online services utilize REST APIs for access to their data.
33
Role of REST-API as a Gateway:
Encapsulation of Logic: Handles client requests, manages authentication, authorization, input sanitization, etc.
Single Entry Point: Acts as the interface between clients and application resources.
34
HTTP Pull Mechanism
**Default HTTP Communication Mode**: Client requests data, server responds.
**Client-Pulled Data**: Fetches data from the server upon request.
**Bandwidth and Server Load**: Frequent requests consume bandwidth and stress server resources.
35
Issues with HTTP Pull:
Repetitive Requests: Continuously fetching data regardless of updates.
Resource Waste: Wastes bandwidth and may overload the server.
36
HTTP Push Mechanism:
Minimizing Client Requests: Initial request followed by server-initiated data updates.
Server-Pushed Updates: Server informs client of new data without additional requests.
Bandwidth and Server Load Reduction: Decreases network load and eases server burden.
37
Callback Concept:
Client's Initial Request: Server promises to notify when new data is available.
Server-Based Notifications: Notifies clients upon available updates
**Common Use Case:**
User Notifications: Backend events triggering notifications to users.
38
AJAX in Both Mechanisms:
Asynchronous JavaScript & XML (AJAX): Used for client-server communication in both Pull and Push methods.
**Technologies in HTTP Push:**
Ajax Long Polling
Web Sockets
HTML5 Event Source
Message Queues
Streaming over HTTP
39
AJAX – Asynchronous JavaScript and XML
Asynchronous Behavior Addition: Enables asynchronous behavior on web pages.
40
Data Fetching Methods:
Manual HTTP GET Request: Triggered by user interface events (e.g., button clicks).
Dynamic Interval-Based Fetching: AJAX used to pull data at regular intervals without user intervention.
41
AJAX Functionality:
Automatic Server Requests: Sends requests automatically at set intervals.
Dynamic Page Updates: Updates specific webpage sections dynamically upon receiving updates.
Common Examples: Dynamic event display on news/sports websites without page reloads.
42
AJAX Implementation:
XMLHttpRequest Object: Utilized by AJAX to send server requests.
DOM Update: JavaScript manages HTML DOM updates.
43
AJAX and jQuery:
Common Framework Usage: AJAX often employed with the jQuery framework.
44
Polling with AJAX:
Dynamic Information Fetching: Technique known as polling via AJAX.
Regular Data Requests: Fetches data periodically from the server.
45
AJAX Polling vs. Long Polling:
Difference: AJAX Polling - HTTP Pull mechanism. AJAX Long Polling - Hybrid (HTTP Push-Pull).
46
HTTP Push
47
TTL in HTTP Pull:
Connection Lifespan: Time to Live (TTL) for each request in HTTP PULL.
Browser Behavior: Closes connection if no response within TTL; requires re-sending the request.
48
Purpose of TTL:
Resource Management: Controls open connections to prevent server overload.
49
Persistent Connection:
Definition: Continuous open network link for multiple requests and responses.
Contrast to Regular Connection: Doesn't close after single communication.
50
Facilitating HTTP Push:
Role in Push-Based Communication: Enables sustained connection between client and server.
51
Heartbeat Interceptors:
Function: Blank request-responses to maintain open connections.
Preventing Connection Closure: Prevents browsers from terminating connections.
52
Resource Intensiveness:
Persistent Connection Resource Usage: Higher resource consumption compared to HTTP PULL.
Use Case Significance: Essential for certain applications requiring consistent connections (e.g., multiplayer games).
53
Techniques for Persistent Connections:
Examples: AJAX Long Polling, Web Sockets, Server-Sent Events.
Exploration in Upcoming Lesson: Detailed discussion on each method
54
Streaming over HTTP
55
Web Sockets:
Purpose: Bi-directional low latency data flow between client and server.
Use Cases: Messaging, chat applications, real-time social streams, browser-based multiplayer games.
Mechanism: Operates over TCP; requires mutual support by server and client.
56
AJAX Long Polling:
Functionality: Server holds response until update available; longer connection times than regular polling.
Benefits: Reduces client-server requests, conserves network bandwidth.
57
Server-Sent Events (SSE):
Approach: Server automatically pushes data to client as events.
Benefits: Reduces blank request-response cycles, conserves bandwidth.
Direction: One-way data flow: server to client.
Use Cases: Real-time feeds, notifications, stock quotes.
58
Streaming over HTTP:
Purpose: Breaks extensive data into smaller chunks for streaming over HTTP.
Use Cases: Multimedia content streaming (videos, images).
Functionality: Facilitates watching partially downloaded content in real-time.
59
Choosing Technology:
Considerations: Tech selection based on specific application requirements.
Tech Overview: AJAX for dynamic UI updates, Long Polling for asynchronous data, Web Sockets for bidirectional flow, SSE for server-to-client flow, HTTP streaming for large object transmission.
60
Server-Side Rendering (SSR)
Rendering technique where the UI is generated on the server and sent as HTML to the client.
Ideal for delivering static content like WordPress blogs.
Benefits SEO since search engine crawlers easily parse static content.
Reduces client-side rendering work, ensuring faster initial UI loading.
61
Client-Side Rendering (CSR)
UI rendering process occurs on the client browser using JavaScript after receiving raw data from the server.
Best suited for modern, dynamic, AJAX-based websites.
Allows on-the-fly content fetching and rendering, enhancing user experience.
Resource-intensive for server if used for entire page generation.
62
Use Cases for Rendering:
SSR: Static content-based websites, like blogs, where content rarely changes.
CSR: Modern web apps reliant on AJAX for dynamic content, requiring frequent updates without page reloads.
Hybrid Approach: Utilizes SSR for static sections and CSR for dynamic content, achieving a balance between speed and dynamic functionality.
63
Why does every browser request have a TTL Time To Live?
Because open connections between the client and the server are resource intensive and there is a limit to the number of open connections a server can manage.
64
When do we need to implement the HTTP PUSH mechanism in our application?
1. When the application is a real-time application like an online multiplayer game, a LIVE sports app.
2. When we need to reduce the number of client requests hitting the server every now and then, checking for new information.
65
Which HTTP PUSH based technology suits best for implementing a streaming media application?
Streaming Over HTTP
This technique is ideal for cases where we need to stream large data over HTTP by breaking it into smaller chunks. It is primarily used for streaming multimedia content, like large images, videos etc., over HTTP.
66
Which technique suits best for implementing a browser-based multiplayer game?
Web Sockets
Web Sockets facilitate bi-directional communication between the client and the server which is typically required in an online game.
67
When should we render the user interface on the server?
When the content to be rendered is static in nature.
Static content can be rendered on the server in comparatively less time as opposed to being rendered on the client. It can also be cached for future requests.
68
What is scalability?
Definition: Ability of an application to handle increased workload without compromising performance.
Ideal Scenario: Consistent response time for each user request, irrespective of the total number of concurrent requests.
Indicator: Maintaining system latency even under heavy traffic load signifies scalability.
Big-O Notation: Target complexity ideally O(1) for constant time; O(n^2) not scalable as data set increases.
69
Network latency
Definition: Time taken for the system to respond to a user request.
Key Components:
Network Latency: Time for data transmission between points A and B. CDN deployment minimizes network latency by leveraging edge locations.
Application Latency: Time for application to process user requests. Stress testing identifies bottlenecks.
70
Importance of Low Latency
User Experience: High latency leads to user frustration, impacting user retention and business revenue.
Industry Impact: Critical in sectors like gaming (MMOs demand real-time response), finance (low-latency trading), where even milliseconds matter.
Real-World Examples: Huawei spent millions to save traders six milliseconds by laying a fiber-optic link between London and New York.
71
Vertical scaling
Vertical scaling involves adding more power to a server, increasing its hardware capabilities like RAM, CPU, etc., to handle increased traffic.
72
Horizontal scaling
73
Latency
Latency is the time taken by a system to respond to a user request. It plays a crucial role in user experience, determining if users stay or leave a website.
74
Server-Side Rendering vs. Client-Side Rendering
Server-side rendering generates HTML on the server and sends it to the client, reducing initial load times. Client-side rendering renders pages in the browser, allowing for dynamic content.
75
Cloud Elasticity
Cloud elasticity refers to the ability to dynamically scale resources up or down based on demand. Cloud platforms provide scalability and cost efficiency for businesses.
76
Microservice Architecture
77
Vertical Scaling Pros
Simplicity: Adding resources to a single server is straightforward without complex configurations. Requires less administrative effort
78
Vertical Scaling Cons
Availability Risk: Few powerful servers might create a single point of failure. If one goes down, the entire system might face downtime.
79
Code Changes in Horizontal Scaling
Statelessness: In a distributed environment, code needs to be stateless to prevent data loss if a server goes down. Avoiding static instances and relying on distributed memory stores like Redis or Memcache.
80
Horizontal Scaling Pros
High Availability: Distributing workload among multiple servers ensures the system remains available even if one server fails. No single point of failure.
81
Horizontal Scaling Cons
Complexity: Setting up and managing a distributed environment requires more effort, monitoring, and system configurations compared to vertical scaling.
82
Choosing Scalability Approach
Predictable vs. Unpredictable Traffic: Vertical scaling works well for predictable traffic; single servers handle load. Horizontal scaling is suitable for unpredictable, high-traffic scenarios like social networks or online games.
83
Database Bottleneck
Single monolith database handling requests from multiple nodes can impact response time. Scaled database through partitioning, sharding needed.
84
Application Architecture
Poor design with sequential processing instead of asynchronous modules can bottleneck operations. Asynchronous tasks needed for uploads, notifications.
85
Caching Efficiency
Underutilizing caching layers affects response time. Caching helps intercept requests, reduces load, and lowers costs. Case study: PolyHaven's effective use of caching.
86
Load Balancer Setup
Inefficient load balancer configuration impacts latency. Correct setup crucial for optimal application performance.
87
Business Logic in Database
Adding business logic in databases creates tight coupling, complicates migrations, and testing. Prefer separating business logic from the database layer.
88
Database Technology
Choosing unsuitable database tech affects application latency. Relational for strong consistency, NoSQL for horizontal scalability.
89
Code Level Efficiency
Poorly written code—nested loops, tight coupling, neglecting complexity—impacts production. DENTTAL checklist ensures code robustness.
90
Dashboard Monitoring Architectural Flow
91
Profiling for Bottlenecks
Utilize application and code profilers to identify resource-consuming processes. Address bottlenecks to enhance performance.
92
Caching Efficiency
Optimize caching strategies. Cache static content, limit database hits, and utilize write-through caching for enhanced performance.
93
CDN Implementation
Implement Content Delivery Network (CDN) to reduce latency by bringing data closer to users.
94
Data Compression
Apply compression algorithms to minimize data size and enhance download speeds, reducing bandwidth consumption.
95
Reducing Round Trips
Minimize unnecessary client-server round trips. Consolidate multiple requests to optimize performance.
96
Scalability Testing
Test system parameters like CPU usage, network bandwidth, throughput, latency, and user experience under heavy load to ensure smooth scalability.
97
Capacity Planning
Provision appropriate hardware and computational power based on scalability testing results to handle expected traffic with a buffer.
98
Load and Stress Testing
Run load and stress tests using tools like JMeter to simulate traffic and prepare for traffic surges during events or peak seasons.
99
High Availability (HA)
HA ensures system uptime despite infrastructural failures, improving reliability and minimizing downtime. Expressed as a percentage in SLAs.
100
Importance of HA
Critical for mission-critical systems like aircraft, hospitals, and finance. Vital for continual connectivity and functionality.
101
Fault Tolerance & Redundancy
Fault tolerance: Systems withstand failures. Redundancy: Duplication of critical components for backup, reducing the risk of system failure.
102
Highly Available Cluster
A cluster designed for high availability. Multiple interconnected servers ensure continuous operation, even if one fails.
103
Cloud Platform HA
Cloud platforms ensure HA by distributing services across multiple servers/locations, using redundant hardware and automated failover mechanisms.
104
# common reasons for system failures:
Software Crashes:
Applications frequently crash due to various issues such as memory leaks, bugs, or corrupted files. Unresponsive processes or system crashes on devices are common occurrences.
105
Hardware Failures:
Hardware issues, including CPU overload, memory failures, hard disk crashes, or network outages, contribute significantly to system failures. A single node going down can disrupt the entire system.
106
Human Errors:
Mistakes in configurations, software deployment, or operational errors can cause system failures. Even minor errors can lead to significant outages, as seen in cases like network configuration issues causing widespread disruptions.
107
Planned Downtime:
Scheduled maintenance activities like software updates, hardware upgrades, or routine maintenance operations often require system downtime, impacting availability.
108
Fault Tolerance:
**System Resilience**: Fault-tolerant systems can endure faults without experiencing complete failure. They are designed to continue operating even if some parts of the system fail.
**Redundancy and Failover**: By employing redundant components or nodes, a fault-tolerant system ensures that if one fails, another can seamlessly take its place. This setup prevents system-wide disruptions.
Reduced Impact of Failures: Even if specific functionalities or services within a system encounter issues or go offline temporarily, the overall system remains functional.
**Application-Level Fault Tolerance**: Microservices architecture is a common approach to achieve fault tolerance at the application level. Breaking down a monolithic application into smaller, independent microservices enables resilience against failures in specific components.
109
Benefits of Microservices:
**Improved Management:** Easier management and maintenance of individual services rather than a monolithic application.
**Development Flexibility: **Enables easier feature addition and modification without affecting other services.
**Scalability and Availability:** Facilitates scalability and high availability by distributing functionalities across multiple services, ensuring that even if some services fail, others continue to function.
110
HA - Redundancy
Redundancy is duplicating the server instances and keeping them on standby to take over in case any of the active server instances go down. It is the fail-safe backup mechanism in the deployment infrastructure.
111
Redundancy and Active-Passive HA Mode:
**Backup Mechanism**: Redundancy involves having duplicate server instances on standby, ready to take over if active instances fail. This setup ensures fail-safe backup within the deployment infrastructure.
**Active-Passive Mode:** In this setup, an initial set of nodes are active and perform the primary functions, while a redundant set of nodes remain passive, only activated when active nodes encounter failure.
**Eliminating Single Points of Failure**: Redundancy eliminates single points of failure by distributing functionalities across multiple nodes. When one node fails, redundant nodes seamlessly take its place, ensuring the system continues functioning without disruption.
112
Advantages and Implementation:
**Zero Downtime Systems:** Critical systems like GPS, aircraft, and satellites rely on redundancy to ensure zero downtime, guaranteeing uninterrupted operation.
**Elimination of Bottlenecks: **Redundancy helps in overcoming bottlenecks by distributing workload across multiple instances, avoiding reliance on single components.
**Real-time Monitoring and Automation:** Continuous monitoring allows immediate detection of bottlenecks or failures, while automation enables self-recovery and dynamic scaling based on system requirements.
**Reducing Human Error Impact:** Automation significantly reduces the chances of system failures caused by human errors, improving system reliability and stability.
113
High Availability Clusters
Definition: A set of nodes working together to ensure continuous service availability.
Components: Nodes connected via a private network (heartbeat network), shared distributed memory, and coordination service (e.g., Zookeeper).
Techniques: Redundancy in disk mirroring (RAID), network connections, power sources to eliminate single points of failure.
Geographic Redundancy: Multiple clusters in a single zone for minimal downtime.
Importance: Crucial in maintaining uptime and ensuring service availability in software system design.
114
Load Balancing in Web Applications
Definition: Technique distributing incoming web traffic across multiple servers or resources.
Strategies: Round-robin, least connections, content-based routing, geographic-based routing.
Purpose: Optimizing resource utilization, ensuring responsiveness during high traffic.
Relation to Scalability/HA: Integral part of a robust system design for smooth user experience.
115
What is the need of a shared distributed memory in a high availability cluster?
To maintain a common state across several nodes running in a cluster.
116
Load balancing
Purpose: Enables scaling and ensures high availability during traffic spikes.
Function: Distributes heavy traffic across servers in a cluster using load balancers.
Risk Avoidance: Prevents overload on specific machines, averting latency spikes and performance degradation.
Failure Mitigation: Automatically redirects requests from failed servers to operational ones, ensuring overall service availability.
Single Point of Contact: Acts as the central entry point for client requests.
Granularity: Can manage traffic at various application component levels (backend servers, databases, message queues, etc.).
Health Checks: Regularly assesses server status to intelligently route requests only to operational machines.
In-Service and Out-of-Service Instances: Maintains a list of operational servers and redirects requests accordingly. Failed servers are temporarily removed and reinstated upon recovery.
117
Load Balancing At Different Components Of The Application
118
DNS
119
Understanding DNS:
IP Address vs. Domain Names: Every machine on the web has a unique IP address, but remembering and using IP addresses for every website is impractical.
IP and its Protocol: Internet Protocol (IP) is the system that enables data packet delivery between machines using their IP addresses.
DNS's Role: Maps easy-to-recall domain names (like amazon.com) to their corresponding IP addresses.
Simplified Access: Users can type domain names into browsers instead of complex IP addresses for website access.
120
How DNS Works:
DNS Querying: Occurs when a user enters a website's URL into a browser.
Key Components of DNS Infrastructure:
DNS Recursive Nameserver (Resolver): Interacts directly with the user's browser, performs queries, and caches responses for future use.
Root Nameserver: Manages the root zone of the DNS, providing information about Top-Level Domain (TLD) nameservers.
Top-Level Domain (TLD) Nameserver: Handles TLDs (.com, .org, etc.), providing details about authoritative nameservers for specific domains.
Authoritative Nameserver: Holds specific domain information, providing IP addresses and other records associated with a domain.
121
DNS Query Lookup Process:
Involves a series of requests from the recursive resolver to the root server, then to the TLD server, and finally to the authoritative server for the specific domain.
Each server in the hierarchy helps resolve the domain name to its corresponding IP address.
122
DNS query lookup process works
Step 1: User Action
User types domain name in the browser and triggers a request.
Step 2: DNS Resolver (Recursive Nameserver)
Receives the request, forwards it to the Root Nameserver.
Managed by ISPs in optimized data centers.
Step 3: Root Nameserver
Responds with the Top-Level Domain (TLD) Nameserver address (e.g., .com for amazon.com).
Step 4: TLD Nameserver
Holds data for specific top-level domains.
Provides the IP address of the domain's Authoritative Nameserver.
Step 5: Authoritative Nameserver
Manages the specific domain (e.g., amazon.com).
Returns the domain's IP address to the Resolver.
Step 6: Resolver's Action
Caches the retrieved data.
Step 7: Browser Interaction
Receives the IP address and requests website content from the domain's server.
123
DNS Optimization and Caching:
Server Setup: ISPs manage efficient Resolvers in data centers.
Caching: DNS data is cached at different levels for faster retrieval.
TTL: Time to Live (TTL) dictates cached data validity.
Browser Cache: Modern browsers cache DNS info for quicker revisits.
124
DNS Load Balancing
Purpose: Distributes user traffic across multiple data centers or server clusters.
Implementation: Occurs at the DNS level on the authoritative server.
Large-Scale Services: Big services like amazon.com operate across multiple data centers globally.
125
How DNS Load Balancing Works:
Authoritative Server Role: Returns different IP addresses of a domain to clients upon queries.
IP List Generation: Provides a list of IP addresses in response, changing the order in a round-robin fashion for each query.
Client Response: Client sends requests to the first IP in the list; can use other IPs if the first doesn’t respond promptly.
Dynamic IP Ordering: Authoritative server rearranges the list order for subsequent requests using round-robin.
126
Load Balancer Impact:
Load Balancer Interaction: Requests might hit a load balancer managing application server clusters, not directly the server.
Limitations: Doesn't consider server loads, content, processing time, or in-service status.
Caching Issues: Cached IPs might lead requests to out-of-service machines, impacting reliability.
127
DNS Laod Balancing Benefits and Drawbacks:
Advantages: Easy and cost-effective load balancing setup for companies.
Limitations: Doesn't account for various server parameters, relying on simple round-robin distribution.
DNS load balancing, though straightforward and cost-effective, has limitations in its inability to consider server status or load. Despite this, it remains a popular choice for distributing traffic across multiple data centers in large-scale services.
128
Hardware-Based Load Balancing:
**Physical Hardware:** These load balancers are physical devices that distribute traffic based on various parameters like active connections and compute utilization.
**Performance and Maintenance: **They're highly performant but require regular maintenance, updates, and specialized skills for upkeep.
**Expensive and Overprovisioning: **Costly to set up compared to software load balancers and need overprovisioning for handling peak traffic.
129
Software-Based Load Balancing:
**Installed on Commodity Hardware/VMs: **Runs on regular hardware or virtual machines, offering cost-effectiveness and greater flexibility for deployment.
**Ease of Upgrade and Provisioning:** Easier to upgrade and provision compared to hardware-based counterparts.
**Advanced Features:** Utilizes various parameters like server data, cookies, HTTP headers, and health checks for efficient traffic routing.
**Popular Examples:** HAProxy is a widely used software load balancer by major industry players due to its features and scalability.
130
Load Balancing Algorithms:
**Round Robin and Weighted Round Robin:** Sequentially sends IP addresses but with weighted distribution based on server capacity.
**Least Connections:** Routes traffic to servers with the fewest open connections, considering CPU utilization and request processing time.
**Random**: Randomly distributes traffic across similar servers.
**Hash**: Routes traffic based on hashed source IPs or URLs, ensuring consistency for client-server connections and efficient caching.
131
Why is DNS load balancing implemented on the authoritative nameserver?
Because the authoritative nameserver returns the IP addresses of a domain’s application servers. If we implement load balancing here, we can easily return different IP addresses of application servers on every client request.
132
Why does DNS load balancing make the authoritative nameserver return a list of IP addresses as opposed to just returning a single different IP every time?
This is done to enable the client to use other IP addresses in the list in case the first doesn’t return a response within a stipulated time.
133
Monolithic Architecture
**Single Codebase**: All components of the application reside within a single codebase, tightly coupled together.
**Tightly Coupled Modules**: Features like user posts, comments, marketplace, etc., are coded as part of one unit rather than separate services.
**Simplicity in Development**: Easier to build, test, and deploy compared to microservices architecture due to its unified structure.
**Early Stage Preference**: Often chosen during the early stages of a business due to simplicity and straightforwardness in development.
**Trade-offs in Transition**: Transitioning from a monolithic architecture to a distributed microservices architecture incurs significant costs in terms of time, effort, and resources.
134
Monolithic vs. Microservices:
**Monolithic Benefits:** Simplicity in development and deployment is the primary advantage, suitable for simpler applications or early-stage businesses.
**Microservices for Scalability:** Microservices offer scalability, flexibility, and better management of complex applications but come with increased operational complexity.
135
Pros of Monolithic Architecture:
Simplicity: Easier development, testing, deployment, and management due to a single repository and unified structure.
136
Cons of Monolithic Architecture:
Continuous Deployment Challenges: Minor changes require redeploying the entire application, complicating the deployment process.
Regression Testing Complexity: Changes in one feature may impact others, necessitating comprehensive regression testing.
Single Point of Failure: A bug in one feature can crash the entire application due to tight coupling.
Scalability Issues: Difficulties in scaling due to tight coupling and increased code size.
Limitation in Technology Usage: Hard to leverage multiple programming languages or technologies within one codebase.
Cloud Compatibility and Statefulness: Not inherently compatible with cloud architecture due to potential stateful components like static variables.
137
Ideal Use Cases for Monolithic Architecture:
Simple Applications: Suitable for straightforward applications with limited complexity, features, and anticipated growth.
138
Microservices Architecture
**Decomposition of Features**: Services are separated into smaller, independent modules, each responsible for a distinct feature or functionality.
**Single Responsibility Principle**: Each service has a singular responsibility, promoting a loosely coupled architecture.
**Simpler Maintenance**: Easier management, maintenance, and deployment due to smaller, focused modules compared to a monolithic architecture.
**Team Efficiency**: Facilitates the division of work among teams, allowing dedicated focus on specific modules or services.
**Scalability**: Scalability is inherent; services can be scaled independently without affecting others.
**Database Separation**: Each microservice ideally has its own database, reducing the risk of single points of failure and bottlenecks.
139
Pros of Microservices Architecture:
**No Single Points of Failure:** Due to loose coupling, the system remains operational even if individual services fail.
**Technology Flexibility:** Allows the use of diverse technologies via REST API interfaces between microservices.
**Independent Deployments:** Enables separate and continuous deployment of services, scalability, and dedicated teams.
140
Cons of Microservices Architecture:
**Management Complexity**: Handling multiple distributed services requires additional management components and skilled resources.
**Consistency Challenges:** Ensuring strong consistency across distributed nodes can be complex; eventual consistency may prevail.
141
Monolithic Architecture
**Definition**:
A monolithic architecture contains the entire application code in a single codebase, with tightly coupled modules handling various features.
**Pros:**
Simplicity in development, testing, and deployment due to a single repository.
Initially easy to build and deploy for simple applications.
No distributed environment complexity.
**Cons:**
Continuous deployment issues for minor changes, requiring full application redeployment.
Regression testing for the entire application after minor code changes.
Single points of failure; a bug in one module can impact the entire system.
**When to Choose:**
Fits for simple applications with limited features that won't expand much.
Cases where complexity and scalability aren't major concerns.
142
Microservices Architecture
**Definition**:
Microservices deploy distinct features as smaller, loosely coupled services working together to form a larger distributed system.
**Pros**:
No single points of failure; fault isolation within individual services.
Leverage heterogeneous technologies; different services can use different languages and databases.
Independent deployments and scalability for each service.
**Cons**:
Increased management complexity with distributed services.
Challenges in ensuring strong consistency across multiple services.
Additional setup needed for distributed logging, monitoring, communication, etc.
**When to Choose:**
Best suited for complex applications needing rapid expansion and feature launches.
Ideal for businesses aiming for quick development and scaling without impacting existing features.
143
Segment high-level architecture
144
Segment Microservices architecture
145
Why is generally a monolithic architecture not cloud ready?
Generally, a monolithic architecture holds a lot of application state in the static variables, and an application to scale inherently on the cloud needs to be stateless.
146
Monolithic Architecture
**Definition**:
A monolithic architecture contains the entire application code in a single codebase, with tightly coupled modules handling various features.
**Pros**:
Simplicity in development, testing, and deployment due to a single repository.
Initially easy to build and deploy for simple applications.
No distributed environment complexity.
**Cons**:
Continuous deployment issues for minor changes, requiring full application redeployment.
Regression testing for the entire application after minor code changes.
Single points of failure; a bug in one module can impact the entire system.
**When to Choose:**
Fits for simple applications with limited features that won't expand much.
Cases where complexity and scalability aren't major concerns.
147
Microservices Architecture
**Definition**:
Microservices deploy distinct features as smaller, loosely coupled services working together to form a larger distributed system.
**Pros**:
No single points of failure; fault isolation within individual services.
Leverage heterogeneous technologies; different services can use different languages and databases.
Independent deployments and scalability for each service.
**Cons**:
Increased management complexity with distributed services.
Challenges in ensuring strong consistency across multiple services.
Additional setup needed for distributed logging, monitoring, communication, etc.
**When to Choose:**
Best suited for complex applications needing rapid expansion and feature launches.
Ideal for businesses aiming for quick development and scaling without impacting existing features.
148
Micro Frontend Development Teams
**Backend Microservices:**
Each UI component has a dedicated microservice on the backend supporting its functionality.
**Full-Stack Micro Frontend Teams:**
Comprise frontend developers responsible for developing specific UI components.
Teams own end-to-end vertical slices from user interface to backend database.
Teams are self-sufficient, using the preferred technology stack for their component.
149
Application Integration
**Unified User Interface:**
Integration of all UI components forms the complete frontend of the e-commerce platform.
The integrated micro frontends deliver a seamless user experience without a centralized UI team.
150
Benefits of Micro Frontends
**Autonomous Teams:**
Teams have autonomy to choose technology stacks and develop their components.
Facilitates fault isolation and independent development.
**Vertical Slice Ownership:**
Teams own end-to-end slices, promoting ownership and accountability for their components.
**Seamless Integration:**
Despite diverse technologies, integrated micro frontends present a unified and cohesive user interface.
151
Micro Frontend
152
Full Stack Teams With Micro Frontends
153
154
Client-Side Integration:
**Approaches**:
Integrating micro frontends on the client.
Similar to client-side and server-side rendering.
**Basic Method**:
Rendering micro frontends via unique links.
Example:
Order Checkout: AWS - https://www.aws.amazon.com/onlinegamestore/checkout
Payment Service: Google Cloud - https://www.cloud.google.com/onlinegamestore/payment
**Issues with Basic Approach:**
Address changes visibly for the user.
Relies on outdated methods like Iframes.
**Modern Alternatives:**
Leveraging Web Components and frameworks like Single SPA.
Single SPA enables frontend development using different JavaScript frameworks.
155
Server-Side Integration:
**Process**:
Integration of UI components on the server.
Delivers pre-built complete website pages to the client on request.
Reduces website loading time significantly for users.
**Implementation**:
Requires additional logic on the server for integration.
Various technologies and frameworks facilitate this process.
**Frameworks for Server-Side Integration:**
Project Mosaic, Open Components, Podium.
Server Side Includes (SSI) for combining multiple web pages' content.
**Example**:
Zalando uses Project Mosaic for managing micro frontends.****
156
DB ARchitecture
157
Unstructured data
158
Structured Data
**Definition**:
Data conforming to a specific structure, often stored in a normalized format in databases.
**Ease of Use:**
Easier to work with, doesn't require data preparation before interaction.
**Example**:
Customer details stored in a row with predefined data types for each column.
**Management**:
Typically handled by SQL (Structured Query Language).
159
Unstructured Data
**Definition**:
Data lacking a definite structure, includes text, images, videos, PDFs, etc.
**Handling**:
Raw data collected from various sources like IoT devices, social networks, etc.
**Interaction**:
Requires data preparation before running analytics for meaningful insights.
160
Semi-Structured Data
**Definition**:
Mix of structured and unstructured data often stored in XML or JSON formats.
**Handling**:
Managed based on business requirements and handled as per specific needs.
161
User State Data:
**Definition**:
Information about user activity on a website.
**Examples**:
Browsing product categories, sorting, wishlists, etc.
**Benefits**:
Enhances user browsing experience, aids in retaining user progress.
162
Relational Databases:
**Definition**:
**Purpose**:
Persist data with relationships: one-to-one, one-to-many, many-to-many, many-to-one, etc.
**Data Model:**
Organized in tables with rows and columns.
**Primary Query Language:**
SQL (Structured Query Language) for data interaction.
163
DB Relationships
**One-to-Many Relationship:**
Example: A customer (C1) linked to multiple books (B1, B2, B3, B4, B5).
**Representation**:
Customer table linked to the product inventory table.
164
Data Consistency
**Normalization**:
Data stored in a simplified, atomic form in one place to maintain consistency.
**Purpose**:
Enables easy updates without data scattering across multiple tables.
165
ACID Transactions:
**Definition**:
Atomicity, Consistency, Isolation, Durability.
**Purpose**:
Ensures flawless execution of transactions without affecting other processes.
**State Management**:
Guarantees the system is in State A or State B, no intermediary states.
**Error Handling:**
Any failure leads to a rollback to the initial state (State A).
166
Selection Criteria for Relational Databases:
***Use Cases:***
**Strong Consistency:**
Needed in stock trading, personal banking, etc.
**Relationship-Centric Apps:**
Examples: Facebook, LinkedIn, etc.
**Transactions Involving Money or Numbers:**
ACID compliance crucial for such software.
167
Strengths of Relational Databases:
**Transactions and Data Consistency:**
Compliance with ACID rules.
**Experience and Reliability:**
Battle-tested, exist for a long time.
168
NoSQL Databases:
**Definition**:
**SQL Absence:**
Lack traditional SQL; structured more like JSON-based databases for Web 2.0.
Use Cases:
**High-Frequency Read-Writes:**
Suitable for social apps, micro-blogging, real-time sports, massive multiplayer games, etc.
169
Differences from Relational Databases:
**Scalability Issues in Relational DBs:**
Complex scaling processes, sharding, replication.
**NoSQL Scalability:**
Easily scalable by adding new server nodes, without human intervention.
170
NoSQL Database Capabilities:
**Cluster Intelligence:**
Designed to operate smartly on clusters with minimal human intervention.
**Self-Healing Infrastructure:**
Nodes with self-recovery capabilities for fault tolerance.
171
Trade-Offs and Considerations:
**Relational DBs vs. NoSQL:**
Both have specific use cases and trade-offs.
**NoSQL Sacrifices**:
Sacrifices strong consistency, ACID transactions for horizontal scaling.
**Eventual Consistency:**
NoSQL data tends to be more eventually consistent.
**NoSQL's Role:**
Suited for specific needs, not a universal solution.
172
Upsides of NoSQL Databases:
**Developer-Friendly Nature:**
**Learning Curve:**
Less steep compared to relational databases.
**Schema Flexibility:**
No strict schemas, offers flexibility in data handling.
**Relaxed Structure:**
No enforced relationships or joins, allowing more freedom.
**Benefits and Drawbacks of Flexibility:
Pros of Flexibility:**
Developer-friendly, absence of joins, relationships, etc.
**Cons of Flexibility:**
Potential for inconsistency due to data spread across.
173
Downsides of NoSQL Databases:
**Inconsistency Risks:**
**Data Spread Challenge:**
Updates across all locations required, prone to inconsistency.
**Normalized Data in Relational DBs:**
Relational databases maintain consistency through normalization.
**ACID Transaction Limitations:**
ACID Transaction Support:
Limited or absent in distributed NoSQL databases.
**Deployment-Level Constraints:**
ACID limited to specific entity hierarchies or regions.
174
Working with NoSQL Databases:
Ease of Development:
**Simplified Operations:**
No stress of managing complex queries or relationships.
**Efficiency**:
Key-based object retrieval leads to faster operations.
Popular NoSQL Databases:
**Industry Usage:**
Examples include MongoDB, Redis, Neo4J, Cassandra, Memcache, etc.
175
What are the two primary methods for data ingestion?
Real-time and batch processing are the two primary methods for data ingestion, chosen based on specific business requirements.
176
In what scenarios is real-time data ingestion preferred?
Real-time ingestion is favored in systems handling time-sensitive data such as medical data from IoT sensors (e.g., heartbeat monitoring) or financial data (e.g., stock market events).
177
When might batch processing be more suitable for data ingestion?
Batch processing is useful for systems analyzing trends over time, like data showcasing the popularity of a sport in a region over an extended period.
178
What challenges are associated with data ingestion?
Challenges include:
Heterogeneous data requiring standardization.
Slow processing due to data transformation and authentication.
Resource-intensive processes and the need for custom solutions.
Security risks when moving data through various stages.
179
Why does real-time data ingestion, despite its name, sometimes involve slow processing?
Real-time ingestion can involve slower processing due to data transformation, authentication, and the need for staged processing, even though it's designed for immediate data delivery.
180
What challenges did LinkedIn face with multiple data ingestion pipelines?
LinkedIn encountered data management challenges with fifteen data ingestion pipelines, leading them to develop Goblin, an in-house data ingestion tool, to address the issue.
181
Why is moving data around considered risky?
Data movement introduces vulnerabilities and requires additional resources to ensure continual compliance with security standards, as it traverses various staging areas.
182
What is the primary distributed data processing framework for ingesting Big Data?
Hadoop is the most popular distributed data processing framework used to analyze Big Data from sources like IoT devices and social apps.
183
How is Elasticsearch utilized in data streaming?
Elasticsearch, an open-source search framework, receives streamed data for indexing, enabling advanced search functionalities in web applications.
184
How was Elasticsearch used in a product search service?
Data was streamed from legacy storage to Elasticsearch for indexing, enabling product searches by replicating data and asynchronously delivering new data for real-time indexing.
185
Why is centralizing logs essential in large-scale services?
Centralizing logs allows tracking errors, studying system behavior, and moving back in time to analyze the system's holistic behavior, especially in microservices-based architectures.
186
What are examples of stream processing engines for real-time events?
Apache Kafka, Apache Storm, Apache Nifi, Apache Spark, Samza, and Kinesis are examples of stream processing engines used for real-time large-scale data processing in online applications.
187
What is the purpose of message queues like Kafka in real-time data processing?
Message queues like Kafka facilitate efficient ingestion, analysis, and rapid distribution of updated information in systems handling live data, such as sports applications.
188
What are data pipelines in a data processing infrastructure?
Data pipelines are fundamental components that enable the efficient movement of data from one point to another, allowing real-time filtering and processing.
189
What are some features of data pipelines?
Features include:
Ensuring smooth data flow
Enabling filters and business logic application
Avoiding bottlenecks and redundancy
Facilitating parallel processing
Safeguarding data integrity
190
How do data pipelines handle data movement?
Data pipelines automate the flow of data, encompassing extraction, transformation, combination, validation, and convergence of multiple data streams based on predefined rules without manual intervention.
191
What is the traditional approach to managing data movement?
ETL stands for Extract, Transform, Load. It involves extracting data from sources, transforming it into a standardized format based on business rules, and loading it into a storage location for further processing, usually done in batches.
192
What distinguishes ETL from real-time data movement via data pipelines?
ETL processes data in batches, while data pipelines handle real-time streaming of data, offering faster results but also support for both batch and real-time data processing techniques.
193
How do popular data processing tools like Apache Flink, Spark, and Kafka operate?
These tools facilitate processing data within a distributed cluster environment through data pipelines, managing data movement and processing in distributed systems.
194
What will upcoming lessons explore regarding distributed data processing?
Future lessons will delve into distributed data processing, covering concepts like Lambda and Kappa architectures, offering insights into how these systems work.
195
Distributed Data Processing
Distributed data processing involves dividing large data sets across multiple nodes in a cluster for parallel execution, enhancing scalability and availability while preventing data loss through redundancy.
196
What facilitates coordination among distributed nodes in a cluster?
Apache Zookeeper is a widely used node coordinator in distributed systems, enabling efficient task coordination among multiple nodes in a cluster.
197
How does distributed data processing differ from centralized processing in terms of task execution?
Distributed data processing shares tasks across multiple nodes for parallel execution, whereas centralized systems process tasks sequentially, one after another.
198
What is MapReduce, and which framework is a popular implementation of this programming model?
MapReduce is a programming model for distributed data processing. Apache Hadoop is a popular open-source implementation used for managing large-scale data processing tasks.
199
What capabilities does Apache Spark offer in a distributed computing environment?
Apache Spark is a cluster computing framework supporting high-performance batch and real-time processing. It integrates with various data sources and facilitates parallel execution in a cluster.
200
What is the primary use of Apache Storm in distributed systems?
Apache Storm is a distributed stream processing framework mainly used for processing massive amounts of streaming data, including real-time analytics and machine learning.
201
What role does Apache Kafka play in distributed stream processing?
Apache Kafka is a distributed stream processing and messaging platform used for managing streams of data, acting as a messaging system for real-time features, log aggregation, and more.
202
Which technologies are preferred for batch data processing and real-time streaming respectively?
Hadoop is preferred for batch data processing, while Spark, Kafka, and Storm are chosen for real-time streaming data processing.
203
What is the Lambda architecture in data processing?
The Lambda architecture is a distributed processing approach that combines batch and real-time streaming data processing to overcome latency issues inherent in batch processing.
204
How does the Lambda architecture address latency issues?
It utilizes both batch processing for accuracy and comprehensive results and real-time processing for quick insights, merging their outcomes to present a holistic view to end-users.
205
What are the primary layers in the Lambda architecture?
The Lambda architecture typically consists of three layers:
Batch layer
Speed layer
Serving layer
206
Describe the role of the batch layer in the Lambda architecture.
The batch layer handles results obtained from batch processing of data, ensuring comprehensive and accurate outcomes despite longer processing times.
207
What purpose does the speed layer serve in the Lambda architecture?
The speed layer collects data from real-time streaming processing, providing quick but less comprehensive insights due to analyzing a smaller data subset.
208
How does the serving layer function in the Lambda architecture?
The serving layer merges outcomes from both the batch and speed layers, presenting a unified view that combines the accuracy of batch processing with the speed of real-time processing.
209
What is Kappa architecture?
Kappa architecture employs a single data streaming pipeline, unlike Lambda architecture, simplifying data processing without distinct layers.
210
How does Kappa architecture handle the flow of data for both real-time and batch processing?
It channels data for both real-time and batch processing through a unified streaming pipeline, eliminating the need for managing separate layers in data processing.
211
How many layers comprise the Kappa architecture, and what are they called?
Kappa architecture comprises two layers: Speed (for streaming processing) and Serving (the final output layer).
212
Is Kappa architecture an alternative to Lambda architecture?
Kappa architecture is not an alternative to Lambda; both architectures have distinct use cases and advantages.
213
When is Kappa architecture preferred over Lambda architecture?
Kappa architecture is favored when batch and streaming analytics results align closely in a system, while Lambda is preferable if their outcomes differ significantly.
214
What is the significance of the two layers within Kappa architecture?
The Speed layer handles streaming processing, while the Serving layer represents the final output, simplifying the architecture compared to Lambda.
215
What is reactive programming in modern software development?
Reactive programming involves a programming paradigm that handles asynchronous data streams and the propagation of changes, emphasizing a responsive, resilient, and scalable approach to application development.
216
Define event-driven architecture in software development.
Event-driven architecture centers on the generation, detection, and reaction to events, emphasizing communication between loosely coupled components responding to various events.
217
What is meant by "blocking" in web applications?
Blocking in web applications refers to the pausing of code execution until a process completes. It delays further execution until the awaited process finishes.
218
Explain "non-blocking" behavior in software development.
Non-blocking behavior allows code execution to continue without waiting for a process to complete, enabling the execution of subsequent code while handling asynchronous tasks.
219
Why are tech and frameworks like NodeJS, Play, Tornado, and Akka gaining popularity over traditional technologies?
Modern tech and frameworks like NodeJS, Play, Tornado, and Akka are favored for their non-blocking, event-driven capabilities, which enhance responsiveness and scalability compared to traditional technologies.
220
How does non-blocking behavior benefit IO-intensive operations?
Non-blocking behavior in IO-intensive operations allows the execution of subsequent code while handling IO tasks, optimizing resource utilization and responsiveness in applications.
221
What distinguishes event-driven architecture from traditional CRUD-based applications in terms of code execution?
Event-driven architecture emphasizes responsiveness to events, enabling code execution without waiting for external processes to complete, contrasting with traditional blocking behavior in CRUD-based apps.
222
Event-driven architecture
223
What do events refer to in the context of web applications?
In web applications, events encompass various occurrences such as HTTP requests, user clicks, variable value changes, or messages ingested, contributing to a stream of events in real-time applications.
224
Define event-driven architecture in modern web application development.
Event-driven architecture (EDA) involves non-blocking, reactive frameworks like NodeJS, Play, and Akka, designed to handle a high volume of concurrent connections, reacting to events occurring regularly in applications.
225
What distinguishes event-driven architectures like NodeJS and Play in modern web development?
These architectures prioritize asynchronous models, handle a large number of concurrent connections efficiently, and manage IO-intensive operations with minimal resource consumption.
226
Explain the nature of reactive architecture within event-driven systems.
Reactive architecture involves coding to respond to frequent events, continuously monitoring streams of asynchronous data in applications inherently designed to be asynchronous.
227
What are some typical use cases for event streaming in an event-driven architecture?
Event streaming is vital for handling transactional events, tracking changing stock prices, user interactions in online shopping, and other real-time, IO-intensive scenarios.
228
Why might NodeJS not be suitable for CPU-intensive tasks?
NodeJS, being single-threaded and designed for handling IO-intensive tasks, is not well-suited for CPU-intensive operations that require substantial computational power like graphics rendering or complex data processing.
229
How does the emergence of non-blocking tech impact traditional technologies?
The emergence of non-blocking tech doesn't render traditional tech obsolete; rather, it emphasizes the importance of choosing the right technology based on specific use cases and trade-offs.
230
What are web hooks ?
Webhooks are a mechanism enabling communication between services without middleware, functioning like callbacks where a service notifies another when specific events occur.
231
How do Webhooks alleviate server load in API-based interactions?
Instead of continuous polling for updates, Webhooks allow services to register an HTTP endpoint. When new information is available, the server triggers an HTTP event to all registered endpoints, reducing unnecessary API requests.
232
Explain the concept of Webhooks using an analogy.
Webhooks work akin to providing a phone number (HTTP endpoint) to a service, instructing it to call when certain events occur, allowing services to carry on without continuous API polling.
233
What role do Webhooks play in event-based communication between services?
Webhooks establish an event-driven mechanism where services notify registered endpoints upon specific events, facilitating real-time updates without constant querying.
234
How do services utilize Webhooks to receive notifications?
Services register unique HTTP endpoints, acting as callback locations, enabling servers to push updates or notifications directly to these endpoints when relevant events occur.
235
Provide an example illustrating the use of Webhooks in everyday scenarios.
Browser notifications utilize Webhooks, notifying users of new content without requiring continuous visits to websites, similar to how Webhooks inform services of updates.
236
What is shared-nothing architecture in distributed systems?
Shared-nothing architecture refers to a system design where modules or services do not share memory or disk resources, aiming to eliminate single points of failure and ensure each component operates independently.
237
Explain the principle behind shared-nothing architecture.
In shared-nothing architecture, each module or service has its own dedicated resources (memory and disk), preventing dependencies and ensuring that if certain modules fail, others remain unaffected.
238
How does shared-nothing architecture contribute to system reliability?
Shared-nothing architecture reduces the impact of failures by isolating resources for each module, enabling the system to maintain functionality even if specific modules or services encounter issues.
239
What benefits does shared-nothing architecture offer for scalability?
Shared-nothing architecture aids scalability by allowing the addition of new modules or services without dependencies on existing resources, promoting independent growth and performance.
240
Define shared-nothing architecture in terms of resource sharing among modules.
Shared-nothing architecture involves modules sharing RAM and disk (database) but not sharing resources among themselves, ensuring independent operation and fault isolation.
241
What does shared-nothing architecture aim to eliminate in distributed systems?
Shared-nothing architecture strives to eliminate single points of failure by ensuring each module or service operates autonomously without shared resources, enhancing system resilience.
242
What are the three main components of the hexagonal architecture?
The three key components are Ports, Adapters, and Domain, aiming to create an independent, loosely coupled, and easily testable application structure.
243
Define the focus of the hexagonal architecture.
Hexagonal architecture aims to enable testing of applications with and without a UI, with mock databases and middleware, without modifying the code, by emphasizing independence and loose coupling among components.
244
Explain the role of ports and adapters in the hexagonal architecture.
Ports act as interfaces for external input to the application, while adapters implement these interfaces, converting external data to a format understandable by the core business logic (domain).
245
What lies at the core of the hexagonal architecture?
The core of the hexagonal architecture is the domain, which represents the business logic. External entities interact with the domain through ports and adapters.
246
Why is the hexagonal architecture represented visually as a hexagon?
The hexagonal shape is a visual representation; it doesn’t affect the pattern itself. It symbolizes the architecture's focus on isolating the business logic (domain) at the center, surrounded by ports and adapters at the edges.
247
How does hexagonal architecture differ from traditional layered architecture?
Hexagonal architecture is an evolved form of layered architecture, aiming to prevent scattering of business logic across multiple layers, unlike traditional layered approaches, which may result in complex, coupled codebases.
248
What is a peer-to-peer (P2P) network?
A P2P network is a decentralized network where computers (nodes) communicate directly with each other, eliminating the need for a central server.
249
What distinguishes P2P networks from centralized systems?
In a centralized system, communication occurs through a central server, whereas in a P2P network, nodes communicate directly, ensuring decentralization and resilience.
250
What are the downsides of centralized systems in comparison to P2P networks?
Centralized systems pose risks of data access, security breaches, single points of failure, and dependency on a central server, unlike P2P networks that offer resilience and data control.
251
What advantages do P2P networks offer?
P2P networks provide resilience during server downtimes, data control without central authority intervention, and efficient file sharing among peers.
252
What are some practical use cases of P2P networks?
P2P networks efficiently handle large-scale file sharing, overcoming memory limits and physical storage constraints. Protocols like BitTorrent enable efficient file sharing among millions of users worldwide.
253
What is the fundamental characteristic of a P2P network?
A P2P network allows nodes (computers) to communicate directly without relying on a central server, promoting decentralized connections among users.
254
Describe segmented P2P file transfer.
In segmented P2P file transfer, large files are divided into chunks, and nodes download these chunks while simultaneously re-hosting them, enhancing availability across the network.
255
How does an unstructured P2P network function?
In an unstructured P2P network, nodes connect randomly without a specific topology or indexing. Data search across the network is resource-intensive and time-consuming.
256
What is a key advantage of a structured P2P network over an unstructured one?
A structured P2P network utilizes indexing or topology, making data search more efficient by implementing distributed hash tables for node indexing.
257
What does a hybrid P2P network model entail?
A hybrid P2P network combines aspects of P2P and client-server models, often seen in blockchain applications for scalability, security, and availability purposes.
258
Can you name a few examples of P2P applications?
Examples include peer-to-peer digital cryptocurrencies like Bitcoin, decentralized microblogging services like Twister, and federated social networks such as Diaspora.
259
What defines a decentralized social network?
Decentralized social networks have servers distributed globally, eliminating a single authority's control, allowing users to host their data, promoting user privacy, data portability, and avoiding scalability issues.
260
Explain the 'Bring Your Own Data' concept in decentralized social networks.
Users own and control their data in decentralized networks, enabling data portability between different applications or networks without the risk of data loss.
261
What are the distinguishing features of decentralized social networks?
Features include data ownership, data privacy, economic compensation for network participation, robust infrastructure, and open-source protocols like ActivityPub.
262
Why is data safety highlighted in decentralized networks?
Decentralized networks prioritize user privacy by encrypting data, ensuring that no central authority can access or manipulate personal information.
263
Describe the infrastructure advantage in decentralized networks.
The decentralized structure distributes infrastructure costs, prevents network downtime, and encourages developers to innovate without concerns about server expenses.
264
What is ActivityPub, and what role does it play in decentralized networks?
ActivityPub is an open, decentralized social networking protocol providing APIs for content access and modification across the network, facilitating communication between network nodes or pods.
265
What does federated architecture entail in decentralized social networks?
Federated architecture extends the concept of decentralization, enabling interconnected servers or pods (like in Mastodon or Diaspora) to exchange information and maintain a network, akin to semi-autonomous entities sharing data.
266
How does federated architecture function in decentralized social networks?
In federated networks, numerous servers or pods are interlinked, with nodes subscribing to these pods. Even if some links break temporarily, the network stays functional, allowing nodes to communicate through their subscribed pods.
267
What role do pods play in a federated network?
Pods serve as discovery points for nodes. Unlike direct peer-to-peer links, pods streamline node discovery without the need for extensive network scans or centralized registries, enhancing network accessibility and usability.
268
What distinguishes pods from direct peer-to-peer connections?
Pods in a federated network simplify node discovery, providing a more efficient and structured approach compared to direct peer-to-peer connections, where discovering other nodes might be challenging without a central registry.
269
Why is federated architecture beneficial for decentralized social networks?
Federated architecture ensures network resilience by interlinking pods, allowing uninterrupted communication between nodes even if certain links between pods experience temporary disruptions.
270
What backend technologies are recommended for real-time data interaction apps like messaging, gaming, or audio-video streaming?
NodeJS, Python's Tornado, Spring Reactor, Play, and Akka are popular for handling real-time data interaction due to their non-blocking nature and ability to manage persistent connections.
271
Which technologies are suitable for building peer-to-peer (P2P) web applications such as distributed search engines or P2P live media services?
JavaScript-based protocols like DAT, IPFS, and frameworks like FreedomJS cater to building P2P web apps.
272
What technologies are typically used for CRUD-based applications like tax filing systems or online booking portals?
Technologies such as Spring MVC, Python Django, Ruby on Rails, PHP Laravel, and ASP .NET MVC are commonly utilized for CRUD-based applications with an MVC architecture.
273
What technology is recommended for small-scale applications like blogs, online forms, or simple integrations with social media platforms?
PHP is an ideal choice for small-scale applications due to its simplicity, lower hosting costs, and frameworks like Spring Boot and Ruby on Rails that facilitate rapid development.
274
Which technologies are suitable for CPU and memory-intensive applications like Big Data processing or parallel processing?
For CPU and memory-intensive tasks, languages like C++, Rust, Java, Scala, Erlang, Go, and Julia are recommended for their performance, scalability, and capabilities in handling large-scale computations.
275
What is the importance of being thorough with project requirements while picking a technology stack?
Being clear on project requirements ensures alignment between chosen technology and project needs. It includes considerations like data structure, scalability expectations, and the architectural setup.
276
Why is it advisable to consider using familiar technology before adopting new ones?
Familiar technology enables faster development due to reduced learning curves, increased productivity, and a better understanding of potential issues and solutions.
277
What role does an active community and good documentation play in choosing a technology stack?
An active community and comprehensive documentation ensure better support, continuous updates, and easy resolution of issues faced during development.
278
Why is it beneficial if the technology chosen is being used by major companies in production?
Adoption by major companies ensures battle-tested technology, fewer inherent issues, continuous updates, and patches, thus reducing the risk of scalability and security problems.
279
What are the advantages of opting for an open-source technology stack?
Open-source tech allows customization, community contributions, cost-effectiveness (no licensing fees), and fosters a larger community for support and feature enhancements.
280
How does the availability of skilled resources influence technology selection?
A technology with readily available skilled resources ensures smoother scaling, quicker feature releases, and ease in finding resources to work on the project.
281
What are the primary characteristics of a web-based mapping service like Google Maps in terms of user activity?
It's a read-heavy application with negligible write operations. It deals with spatial data that includes geometric and alphanumeric information.
282
Which databases are suitable for handling spatial data in a mapping application?
Spatial databases like MySQL, MongoDB, CouchDB, Neo4J, Redis, and Google Big Query GIS support the persistence of spatial data, each with dedicated plugins.
283
In the context of database scalability for a mapping service, which type of database (relational or NoSQL) is preferable, and why?
NoSQL databases are favored due to horizontal scalability and a lack of complex relationships in map data, enabling them to handle surge in traffic during peak hours efficiently.
284
For a mapping service with multiple features like route planning, real-time updates, and various APIs, which architecture—monolithic or microservice-oriented—would be better suited?
Microservice architecture is preferable to build separate services for various functionalities, ensuring scalability, maintainability, and high availability.
285
When rendering map tiles for a mapping service, what advantages does server-side rendering offer over client-side rendering?
Server-side rendering enables caching of static content, generation and caching of tiles, and improved performance by rendering smaller sections of the map (tiles) based on user interactions.
286
Which technologies and frameworks are typically used for developing the user interface of a mapping service?
UI technologies like JavaScript (Vanilla JS, React, Angular, Vue), HTML5, and open-source libraries like OpenLayers are commonly employed for map interaction and interface development.
287
How are real-time features managed in a web-based mapping service, and what considerations should be made?
Real-time features require persistent connections and should be implemented judiciously due to their resource-intensive nature, considering limitations in handling concurrent connections on servers.
288
Why is a relational database like MySQL preferred for setting up an online payment system?
For implementing a payment system, ACID compliance is crucial for transactional consistency, making a relational database like MySQL an ideal choice.
289
How can high concurrency scenarios, where multiple users try to purchase the same ticket, be managed effectively?
Implementing a message queue, applying the FIFO (First In, First Out) principle, or utilizing database locks with the correct transaction isolation levels are viable approaches to handle concurrency.
290
What purpose does caching serve in a ticket booking scenario, and which caching technologies can be utilized?
Caching helps reduce database load by holding consistent but possibly inaccurate ticket counts shown on the website. Technologies like Redis, Memcached, or Hazelcast are commonly used for caching.
291
Which backend technologies are suitable for managing the logic and messaging queue for user notifications in a ticket booking application?
Backend technologies like Java, Scala, Python, or Go can be used, while message queues such as RabbitMQ or Kafka are effective for sending notifications to users.
292
What are the recommended strategies for building a responsive user interface for a ticket booking portal?
Leveraging CSS3 for responsive design or using open-source frameworks like Bootstrap JS can ensure responsiveness. JavaScript frameworks such as React, Angular, or Vue are also viable options for UI implementation.
293
Why have mobile devices become the dominant platform for online services?
Mobile devices offer unparalleled accessibility and ease of use, making them the preferred choice for approximately 60-70% of global users accessing online services.
294
How has the transition to mobile affected the gaming industry?
The popularity of smartphones has led to a shift in gaming preferences, moving from web and desktop platforms to mobile. Gaming companies observed better engagement and retention rates with mobile versions, leading to a significant increase in Monthly Active Users (MAU) and Daily Active Users (DAU).
295
What impact has the mobile shift had on businesses?
Businesses are shifting focus from web-first to mobile-first strategies. Platforms like Facebook, once hosting numerous games, have now transitioned to mobile-only games, while businesses like Instagram, TikTok, and Pokemon Go thrive with billions of users on mobile.
296
How significant is the mobile segment for businesses today?
Mobile devices drive the majority of traffic for businesses, with the Google Play Store boasting over 2 billion monthly active users and hosting more than 3.5 million apps, totaling 82 billion app downloads.
297
What do terms like mobile-only, mobile-first, and mobile-friendly mean in the context of application development?
These terms represent different approaches in app development, where mobile-only refers to applications exclusive to mobile devices, mobile-first implies prioritizing mobile development, and mobile-friendly denotes applications optimized for seamless mobile user experiences.
298
What are the two approaches to designing responsive websites?
The two approaches are mobile-first and web-first. In the mobile-first approach, the website is designed for smaller screens, adapting later to larger desktop screens. Conversely, the web-first approach designs for bigger screens and adapts to smaller screens.
299
What is the analogy often used for content in responsive web design?
The analogy is that "content should be like water," meaning it should take the shape of the device or screen it is displayed on.
300
What is BootstrapJS, and how is it used in web development?
BootstrapJS is an open-source CSS framework designed for responsive user interface development. It includes pre-built CSS, JavaScript features, and commonly used website elements, simplifying web design. It offers a grid-based approach for creating web pages and provides ready-made templates and plugins.
301
Why is BootstrapJS a popular choice among developers, particularly those who are not designers?
BootstrapJS is favored due to its ease of use, intuitive nature, and availability of pre-built elements. It's especially useful for backend developers or non-designers transitioning to frontend development as it streamlines the design process and has a manageable learning curve.
302
What prompted the creation of BootstrapJS, and where was it originally developed?
BootstrapJS originated at Twitter to ensure consistency across their internal tools' user interfaces. Twitter later released it as an open-source project, aiming to provide a unified framework for frontend development.
303
Besides BootstrapJS, name other frameworks used for designing responsive websites.
Other frameworks include jQuery Mobile, Skeleton, HTML5 Boilerplate, Less Framework, among others, each offering different features and approaches to responsive web design.
304
305
What are native mobile apps?
Native apps are developed exclusively for a particular operating system (e.g., Android, iOS, Windows). They interact directly with the OS and device hardware, providing high performance and a consistent user interface matching the native OS.
306
What technologies are used for writing native apps on Android and iOS?
For Android, developers can use Java, Kotlin, or C++. For iOS, technologies include Swift, Objective C, and the Cocoa Touch framework.
307
What are hybrid mobile apps?
Hybrid apps combine elements of both native and web-based technologies. They can be installed from app stores, access device hardware, and communicate with the OS. Built using web technologies like HTML5, CSS, and JavaScript, they run within a native container but might experience slightly lower performance compared to native apps.
308
Name some popular frameworks used for developing hybrid mobile apps.
Frameworks like React Native, Apache Cordova, Ionic, and Flutter are popular choices for building hybrid apps. They leverage open web technologies and offer cross-platform development capabilities.
309
What are some examples of companies using React Native for their mobile apps?
Companies like Bloomberg, Walmart, Uber Eats, and Discord utilize React Native for their mobile applications, leveraging its cross-platform capabilities and JavaScript-based development.
310
How does the performance of hybrid apps typically compare to native apps?
Hybrid apps might have slightly lower performance and UI rendering speed due to the additional middle layer between web technologies and the native OS, which native apps do not have.
311
What are the challenges in native app development that led to the emergence of hybrid apps?
Developing dedicated apps for different platforms, requiring distinct technologies and teams, led to the need for a common codebase and a simpler approach, resulting in the emergence of hybrid apps using web-based technologies like HTML5 and JavaScript.
312
Why might businesses opt for hybrid apps over native apps?
Hybrid apps leverage web-based technologies, allowing developers with modern web development skills to build them without a steep learning curve. They offer a common codebase for multiple platforms, reducing the need for dedicated platform-specific teams and saving time and resources.
313
# 1.
What are the drawbacks or issues associated with hybrid apps?
Hybrid apps might lack the same level of performance and smoothness as native apps due to the additional layer between the web-based technologies and the native OS. They might face challenges in achieving the same user experience as native apps.
314
Can you provide examples of businesses that shifted from hybrid to native app development?
Airbnb shifted from React Native to native technology due to performance issues and challenges in achieving native functionality. Udacity abandoned React Native due to increased Android-specific features and maintenance costs. Facebook admitted their mistake in relying too much on HTML5 for their mobile app back in 2012.
315
What's an essential consideration when choosing between native and hybrid app development for a use case?
When choosing between native and hybrid app development, considering the specific use case, performance requirements, desired user experience, and long-term maintenance costs is crucial to make an informed decision.
316
What defines a native app?
A native app is built exclusively for a particular operating system (OS), accessing hardware directly, ensuring high performance and consistent user experience.
317
What technologies are used to develop native Android apps?
Java, Kotlin, or C++ are commonly used for writing native Android apps.
318
What defines a hybrid app?
A hybrid app uses web-based technologies (HTML5, CSS, JavaScript), runs within a native container, and communicates with the device's OS through a middle layer.
319
What are the main drawbacks of hybrid apps compared to native apps?
Hybrid apps might experience performance issues, lags, and difficulty in accessing device-specific hardware due to the middle layer between the app and the OS.
320
When should you opt for a native app development approach?
Native apps are preferred for heavy graphic or hardware requirements, demanding UI animations, complex functionalities requiring direct hardware access, and maintaining consistency with the native OS.
321
In what scenarios is choosing a hybrid app advisable?
Hybrid apps are recommended for simpler app requirements, resource constraints, initial MVP launches, testing the market, and when the team lacks proficiency in native technologies.
322
# [](http://)
What are Progressive Web Apps (PWAs)?
PWAs are web applications that offer a native app-like experience, accessible through web browsers, featuring functionalities like offline access, push notifications, and device hardware integration.
323
How can PWAs be installed on devices?
Users can install PWAs directly from the browser by clicking on an install prompt, adding a shortcut to the home screen without visiting app stores.
324
# [](http://)[](http://)
What technologies are used to build PWAs?
PWAs are developed using web technologies such as HTML, CSS, and JavaScript, along with frameworks like Angular, React, Ionic, and Google Polymer.
325
What benefits do PWAs offer to businesses?
PWAs help businesses provide an app-like experience directly in browsers, improving conversion rates, reducing bounce rates, and decreasing app sizes compared to native apps.
325
How do PWAs differ from native apps?
While PWAs offer native-like experiences, native apps maintain superiority in performance, especially for CPU-intensive tasks like gaming.
326
Can PWAs replace native apps entirely?
PWAs do not replace native apps. Instead, they compete more with responsive mobile websites, offering an app-like experience without the need for app store installations.
327
What is Mobile Backend as a Service (MBaaS)?
MBaaS is a cloud-based service model that manages backend infrastructure, offering features like user authentication, social network integration, push notifications, databases, and more for mobile apps.
328
What features does an MBaaS typically offer?
MBaaS services include user authentication, real-time databases, push notifications, caching, data storage, messaging, third-party tool integration, analytics, crash reporting, and more.
329
How does MBaaS benefit developers during the bootstrapping phase?
MBaaS offers a generous free tier, enabling solo developers or small teams to build and test their app ideas without worrying about initial hosting costs, allowing them to showcase their concepts to potential investors.
330
What role does an API play in MBaaS?
MBaaS provides APIs for various features, allowing app code to interact directly with these APIs to perform necessary operations like user authentication, database interactions, and messaging.
331
When should one use MBaaS for app development?
MBaaS is suitable for mobile-only services and apps where a custom backend isn't available or required. It's ideal for mobile games, messaging apps, and simple apps like to-do lists.
332
What are considerations while using MBaaS?
MBaaS involves keeping business logic on the client side and may require designing a custom backend if new features demand server-side logic. Vendor lock-in risk exists if the service provider discontinues or alters their service unfavorably.
333
