Interviews
etuoce:
conda create -n ecoute311 python=3.11 -y
conda activate ecoute311
pip install -r requirements.txt
pipdeptree | Select-String "mysql"
$env:OPENAI_API_KEY = "sk-your-key-here"
Solution design
Modern apps use both, depending on use case. For example, high flexibility. Where as if you need structure, like a banking app, you would go with SQL.
Slow
Redundancy is ok, tradeoff performance speed
Too slow for real time, for example onling multiplayer games.
For this, polling is inefficient, as it increases server load, and is inefficient. Instead, we use web sockets, providing consistent service between the two client and service
Core System Architecture Concepts
Client-Server Architecture
Function: A foundational model where a client (e.g., web browser, mobile app, or any other front-end application) sends requests to a server.
Process: The server, a continuously running machine, processes requests (such as storing or modifying data), performs necessary operations, and sends responses back to the client.
Locating Servers: IP Address & DNS
IP Address: A unique numerical address (like a phone number) that computers use to identify and communicate with servers on the internet. Every publicly deployed server has a unique IP address, and clients must send requests to the correct IP address.
DNS (Domain Name System): Translates easy-to-remember human-friendly domain names (e.g.,
algomaster.io) into their corresponding machine-readable IP addresses. When a user enters a domain name, their computer asks a DNS server for the corresponding IP address, which the browser then uses to establish a connection.
Intermediaries: Proxy and Reverse Proxy
Proxy Server: Acts as a middleman between a client's device and the internet. It forwards client requests to the target server, retrieves the response, and sends it back to the client, hiding the client's original IP address to keep their location and identity private.
Reverse Proxy: Sits in front of backend servers, accepting client requests and forwarding them to the appropriate server based on rules, acting as a single gateway.
Communication and APIs
Latency and Data Centers
Latency: The round-trip delay for data to travel from a client to a server and back. Physical distance is a major factor, and high latency can make applications feel slow and unresponsive.
Global Data Centers: Deploying services across multiple data centers worldwide reduces latency by allowing users to connect to the nearest server instead of waiting for data to travel across the globe.
Communication Protocols: HTTP & HTTPS
HTTP (Hypertext Transfer Protocol): Defines rules for browser-server communication. The client sends a request to the server, which includes a header (containing details like request type, browser type, and cookies) and sometimes a request body (carrying additional data like form inputs). The server processes the request and responds with an HTTP response, either returning data or an error message. HTTP has a major security flaw as it sends data in plain text.
HTTPS (HTTP Secure): The secure version of HTTP, encrypting all data using SSL or TLS protocols, ensuring that even if someone intercepts the request, they can't read it.
APIs: REST and GraphQL
API (Application Programming Interface): HTTP is just a protocol for transferring data, but it doesn't define how requests should be structured or what format responses should be in. APIs act as a middleman, allowing clients to communicate with the server. An API processes requests, interacts with databases or other services, and returns responses in a structured format, usually JSON or XML.
REST (Representational State Transfer):
- A widely used, stateless API style where every request is independent.
- Treats everything as a resource (e.g., users, orders, products).
- Uses standard HTTP methods: `GET` (retrieve data), `POST` (create new data), `PUT` (update existing data), and `DELETE` (remove data).
- REST APIs are simple, scalable, and easy to cache.
- **Limitation:** Often returns more data than needed (over-fetching), especially when dealing with complex data retrieval, leading to inefficient network phases.
GraphQL:
- Introduced in 2015, GraphQL is an API query language that lets clients ask for exactly what they need, nothing more, nothing less.
- It allows combining multiple data needs into a single query, reducing the number of requests compared to REST, where multiple endpoints might be needed.
- **Trade-offs:** Requires more server-side processing and is harder to cache than REST.
Data Storage and Management
Databases: SQL vs. NoSQL
Function: Dedicated servers for storing and managing massive volumes of data efficiently. A database is the backbone of any modern application, ensuring data is stored, retrieved, and managed efficiently while keeping it secure, consistent, and durable.
SQL (Relational Databases):
- Stores data in tables with a strict, predefined schema.
- Follows ACID properties (Atomicity, Consistency, Isolation, Durability).
- **Ideal for:** Applications that require strong consistency and structured relationships, such as banking systems.
NoSQL (Non-relational Databases):
- Designed for high scale and high performance with flexible schemas.
- Uses different data models, including key-value stores, document stores, graph databases, and wide-column stores, which are optimized for large-scale distributed data.
- **Ideal for:** Applications requiring high scalability and flexible data structures. Many modern applications use both SQL and NoSQL together.
Blob Storage
Function: Stores large, unstructured files (blobs) like images, videos, and documents, as traditional databases are not designed for this.
Example: Amazon S3.
Benefits: High scalability, pay-as-you-go model, automatic replication, and easy access. Blobs are stored inside logical containers or buckets, and each file gets a unique URL, making it easy to retrieve and serve over the web. Commonly used for streaming audio or video files.
Scaling Strategies
Application Server Scaling
Vertical Scaling (Scaling Up): Upgrades an existing single server by adding more CPU, RAM, or storage, making it more powerful.
- **Limitations:** Has physical limits and creates a single point of failure (if the server crashes, the entire system goes down). It's a quick fix but not a long-term solution for handling high traffic and ensuring system reliability.
Horizontal Scaling (Scaling Out): Adds more servers to distribute the load across multiple machines, making the system more scalable and fault-tolerant.
- **Benefits:** Increases capacity to handle increasing traffic more effectively and improves reliability, as other servers can take over if one fails.
Load Balancer
Function: Sits between clients and backend servers, acting as a traffic manager that distributes incoming requests across multiple servers.
Features: If one server crashes, the load balancer automatically redirects traffic to another healthy server.
Algorithms: Uses load balancing algorithms such as Round Robin, Least Connection, and IP Hash to decide which server should handle the next request.
Database Scaling Techniques
Indexing: A super-efficient lookup table that helps the database quickly locate required data without scanning the entire table. It stores column values along with pointers to actual data. Indexes are typically created on frequently queried columns (e.g., primary keys, foreign keys, or columns used in WHERE conditions), speeding up reads but potentially slowing down writes as the index needs to be updated.
Replication: Creates copies (read replicas) of a primary database across multiple servers. The primary database (primary replica) handles all write operations, while multiple read replicas handle read queries. Data is copied to read replicas to stay in sync. This improves read performance by spreading requests across replicas and enhances availability, as a read replica can take over if the primary fails. It's great for scaling read-heavy applications.
Sharding (Horizontal Partitioning): Splits a large database into smaller, more manageable parts called shards, distributing them across multiple servers. Each shard contains a subset of the total data, distributed based on a sharding key (e.g., User ID). This reduces database load and speeds up read and write performance by distributing queries across multiple shards instead of a single database. It's called horizontal partitioning because it splits data by rows.
Vertical Partitioning: Splits a database table by columns into smaller, focused tables. This is used when the issue is the number of columns rather than rows. For example, a user table might be split into smaller tables based on user patterns (e.g., profile details, login history, billing information). This optimizes queries by scanning only relevant columns and reduces unnecessary disk I/O.
Performance Optimization
Caching
Function: Stores frequently accessed data in high-speed memory (instead of repeatedly fetching it from the database) to optimize system performance, as retrieving data from memory is always faster than from disk.
Cache-Aside Pattern:
1. When a user requests data, the application first checks the cache.
2. If the data is in the cache (cache hit), it returns instantly, avoiding a database lookup.
3. If the data is not in the cache (cache miss), the application retrieves it from the database, stores it in the cache for future requests, and returns it to the user. The next time the same data is requested, it is served directly from the cache, making the request much faster. To prevent outdated data, cached items need a Time-To-Live (TTL) value.
Denormalization
Function: While normalized data reduces redundancy by breaking data into separate tables, it introduces slow join operations when retrieving data from multiple tables. Denormalization reduces the number of joins by combining related data into a single table, even if some data gets duplicated.
Benefit: Leads to faster query performance and better read performance, especially for read-heavy applications where speed is critical.
Downside: Increases storage needs and complicates data updates.
Content Delivery Network (CDN)
Function: A global network of distributed servers that work together to deliver web content (like HTML pages, JavaScript files, images, and videos) to users based on their geographic location. It caches content and delivers it from the server closest to the user.
Benefit: Significantly reduces latency and improves load times with minimal buffering worldwide, as content is served from the closest CDN server.
Advanced System Components and Concepts
Real-Time Communication: WebSockets & Webhooks
HTTP Polling (Inefficient Method): Most web applications use HTTP's request-response model, requiring the client to send another request for new data. For real-time updates, this leads to frequent polling (repeated requests every few seconds), which is inefficient as it increases server load and wastes bandwidth because most responses are empty when there is no new data.
WebSockets: Solves the polling problem by allowing continuous, two-way communication between the client and the server over a single, persistent connection. Once established, the connection remains open, allowing the server to push updates to the client at any time without waiting for a request. The client can also send messages to the server. This enables real-time interactions.
Webhooks: Allows a server to automatically notify another server about an event as soon as it occurs, eliminating the need for constant polling. The receiver (e.g., your app) registers a webhook URL with the provider. When an event occurs, the provider sends an HTTP POST request with event details to the registered webhook URL. This saves server resources and reduces unnecessary API calls.
Architectural Patterns: Monolith vs. Microservices
Monolithic Architecture: Traditionally, applications were built with all features inside one large codebase. This setup works fine for small applications, but for large-scale systems, monoliths become hard to manage, scale, and deploy.
Microservices Architecture: The solution is to break down an application into smaller, independent services called microservices that work together. Each microservice handles a single responsibility, has its own database and logic, can scale independently, and communicates with other microservices via APIs or message tools. This way, services can be scaled and deployed individually without affecting the entire system.
Asynchronous Communication: Message Tools
Function: Synchronous communication (waiting for immediate responses) doesn't scale well. Message tools enable services to communicate asynchronously, allowing requests to be processed without blocking other operations, thereby decoupling services and improving scalability.
Process: A producer places a message in a queue. The queue temporarily hosts the message. A consumer then retrieves and processes the message.
System Protection and Management
Rate Limiting: Restricts the number of requests a client can send within a specific time frame (e.g., 100 requests per minute) to prevent abuse, protect servers from crashing due to excessive requests, and prevent degradation of performance for legitimate users. Every user or IP address is assigned a request quota; if exceeded, the server temporarily blocks additional requests and returns an error. Common algorithms include fixed window, sliding window, and token bucket.
API Gateway: A centralized service that acts as a single entry point for all client requests in microservices-based applications. Instead of exposing each microservice directly, the API gateway handles request routing to the appropriate microservice, authentication, rate limiting, logging, monitoring, and much more. It simplifies the API and enhances scalability and security.
Idempotency: In distributed systems, network failures and service retries are common. Idempotency ensures that making the same request multiple times produces the same result as if the request was made only once, preventing duplicate operations (e.g., double payments). Each request is assigned a unique ID; before processing, the system checks if the request has already been handled, ignoring duplicates and processing new requests normally.
Distributed Systems: CAP Theorem
Principle: A fundamental principle of distributed systems, the CAP theorem states that a distributed system cannot simultaneously guarantee all three of the following properties:
1. **Consistency:** All nodes see the same data at the same time.
2. **Availability:** Every request receives a response, though it may not contain the most recent data.
3. **Partition Tolerance:** The system continues to operate despite network failures separating nodes.
Conclusion: Since network partitions (failures) are inevitable in distributed systems, a system must choose between Consistency (CP) and Availability (AP).
📅 Next Steps
Check out the blog at
blog.algomaster.iofor in-depth articles on complex system design topics with clear explanations and real-world examples.
Subscribe to the weekly newsletter at
gog.algomasters.iofor deeper dives into system design concepts with real-world examples, and articles on system design interview questions and tips.
IAM Experience
- I implemented secure user authentication using the OAuth2 Authorization Code Flow with OpenID Connect for browser-based login, and JWT-based service-to-service authentication for secure microservice communication.
- Built and deployed a production-ready SSO platform (OAuth2/OIDC), delivering secure, seamless authentication across multiple microservices and client apps.
- User Login with OAuth2/OIDC (Authorization Code Flow)
- Scenario: A human user (in a browser) logs in to your app.
- Flow:
- User visits your client app (API Gateway).
- They’re redirected to the identity-backend to log in.
- Upon success, a single-use auth code is issued and exchanged for a JWT (id_token).
- The JWT is stored in the session and used for user requests.
- Standard name: Authorization Code Flow (OAuth2/OIDC)
- Service-to-Service Authentication Using JWT
- Scenario: One of your backend microservices (e.g., Worker, Logging Service) needs to make authenticated requests to another service.
- How you do it:
- The service uses a JWT signed with a shared secret to prove its identity.
- The receiving service verifies the JWT (checks signature, issuer, audience, expiry, etc.).
- Standard name: This aligns with Client Credentials Flow in OAuth2, except in your code, there’s no explicit token endpoint where the service requests a new token by presenting its credentials—instead, you’re issuing and signing JWTs directly using a shared secret, often called “JWT Bearer Authentication” or “service-to-service JWT” in practice.
- You don’t implement the full OAuth2 “Client Credentials” handshake (POST to /token for service tokens), but you do implement direct JWT-based S2S auth, which is the modern standard in microservices.
- Led teams in adopting modern microservices best practices and paved road architecture.
- OAuth, used for token exchange and verification, an access token
- OIDC, used to provide identity, access token, and ID token, represented as a JWT, collection of JSON, ID, Name, when you logged in, id token's expiration, tampering. This data inside of a JTW is known as claims. For example, you can get email address from IDP, such as email address
Comments
Post a Comment