62 Advanced System Design Interview Questions and Concepts To Know

The CAP Theorem for distributed computing was published by Eric Brewer. This states that it is not possible for a distributed computer system to simultaneously provide all three of the following guarantees:

1. Consistency (all nodes see the same data even at the same time with concurrent updates )
2. Availability (a guarantee that every request receives a response about whether it was successful or failed)
3. Partition tolerance (the system continues to operate despite arbitrary message loss or failure of part of the system)

The CAP acronym corresponds to these three guarantees. This theorem has created the base for modern distributed computing approaches. Worlds most high volume traffic companies (e.g. Amazon, Google, Facebook) use this as basis for deciding their application architecture. It's important to understand that only two of these three conditions can be guaranteed to be met by a system.

When you run something asynchronously it means it is non-blocking, you execute it without waiting for it to complete and carry on with other things. Parallelism means to run multiple things at the same time, in parallel. Parallelism works well when you can separate tasks into independent pieces of work. Async and Callbacks are generally a way (tool or mechanism) to express concurrency i.e. a set of entities possibly talking to each other and sharing resources.

Take for example rendering frames of a 3D animation. To render the animation takes a long time so if you were to launch that render from within your animation editing software you would make sure it was running asynchronously so it didn't lock up your UI and you could continue doing other things. Now, each frame of that animation can also be considered as an individual task. If we have multiple CPUs/Cores or multiple machines available, we can render multiple frames in parallel to speed up the overall workload.

Round‑robin load balancing is one of the simplest methods for distributing client requests across a group of servers. Going down the list of servers in the group, the round‑robin load balancer forwards a client request to each server in turn. When it reaches the end of the list, the load balancer loops back and goes down the list again (sends the next request to the first listed server, the one after that to the second server, and so on).

Scalability is the ability of a system, network, or process to handle a growing amount of load by adding more resources. The adding of resource can be done in two ways

• Scaling Up
  This involves adding more resources to the existing nodes. For example, adding more RAM, Storage or processing power.
• Scaling Out
  This involves adding more nodes to support more users.

Any of the approaches can be used for scaling up/out a application, however the cost of adding resources (per user) may change as the volume increases. If we add resources to the system It should increase the ability of application to take more load in a proportional manner of added resources.

An ideal application should be able to serve high level of load in less resources. However, in practical, linearly scalable system may be the best option achievable. Poorly designed applications may have really high cost on scaling up/out since it will require more resources/user as the load increases.

Some of the advantages of REST web services are:

• Learning curve is easy since it works on HTTP protocol
• Supports multiple technologies for data transfer such as text, xml, json, image etc.
• No contract defined between server and client, so loosely coupled implementation.
• REST is a lightweight protocol
• REST methods can be tested easily over browser.

NoSQL is better than RDBMS because of the following reasons/properities of NoSQL:

• It supports semi-structured data and volatile data
• It does not have schema
• Read/Write throughput is very high
• Horizontal scalability can be achieved easily
• Will support Bigdata in volumes of Terra Bytes & Peta Bytes
• Provides good support for Analytic tools on top of Bigdata
• Can be hosted in cheaper hardware machines
• In-memory caching option is available to increase the performance of queries
• Faster development life cycles for developers

Still, RDBMS is better than NoSQL for the following reasons/properties of RDBMS:

• Transactions with ACID properties - Atomicity, Consistency, Isolation & Durability
• Adherence to Strong Schema of data being written/read
• Real time query management ( in case of data size < 10 Tera bytes )
• Execution of complex queries involving join & group by clauses

Low latency means that there is very little delay between the time you request something and the time you get a response. As it applies to webSockets, it just means that data can be sent quicker (particularly over slow links) because the connection has already been established so no extra packet roundtrips are required to establish the TCP connection.

Reliability is the probability that a system performs correctly during a specific time duration. During this correct operation, no repair is required or performed, and the system adequately follows the defined performance specifications.

Reliability follows an exponential failure law, which means that it reduces as the time duration considered for reliability calculations elapses. In other words, reliability of a system will be high at its initial state of operation and gradually reduce to its lowest magnitude over time.

KISS, a backronym for "keep it simple, stupid", is a design principle noted by the U.S. Navy in 1960. The KISS principle states that most systems work best if they are kept simple rather than made complicated; therefore simplicity should be a key goal in design, and that unnecessary complexity should be avoided.

The fundamental motivation is this:

You want to be able to rip an entire layer out and substitute a completely different (rewritten) one, and NOBODY SHOULD (BE ABLE TO) NOTICE THE DIFFERENCE.

The most obvious example is ripping the bottom layer out and substituting a different one. This is what you do when you develop the upper layer(s) against a simulation of the hardware, and then substitute in the real hardware.

Also layers, modules, indeed architecture itself, are means of making computer programs easier to understand by humans.

CDNs are very useful for a multiple reasons:

• It simply Decrease Server Load - On average, 80% of a website consist of static resources therefore when using a CDN, there is much less load on the origin server.
• Make Faster Content Delivery - For website owners who have visitors in multiple geographic locations, content will be delivered faster to these users as there is less distance to travel.
• 100 Percent Availability - CDN users also benefit from the ability to easily scale up and down much more easily due to traffic spikes.

• Atomicity: It ensures all-or-none rule for database modifications.
• Consistency: Data values are consistent across the database.
• Isolation: Two transactions are said to be independent of one another.
• Durability: Data is not lost even at the time of server failure.

Single responsibility is the concept of a Class doing one specific thing (responsibility) and not trying to do more than it should, which is also referred to as High Cohesion.

Classes don't often start out with Low Cohesion, but typically after several releases and different developers adding onto them, suddenly you'll notice that it became a monster or God class as some call it. So the class should be refactored.

• Purge - Removes content from cache immediately. When the client requests the data again, it is fetched from the application and stored in the cache. This method removes all variants of the cached content.

• Refresh - Fetches requested content from the application, even if cached content is available. The content previously stored in the cache is replaced with a new version from the application. This method affects only one variant of the cached content.

• Ban - A reference to the cached content is added to a blacklist (or ban list). Client requests are then checked against this blacklist, and if a request matches, new content is fetched from the application, returned to the client, and added to the cache. This method, unlike purge, does not immediately remove cached content from the cache. Instead, the cached content is updated after a client requests that specific information.

The most efficient operation for what you're describing would be to use a webSocket connection between client and server and have the server send updated price information directly to the client over the webSocket ONLY when the price changes by some meaningful amount or when some minimum amount of time has elapsed and the price has changed.

Here's a comparison of the networking operations involved in sending a price change over an already open webSocket vs. making a REST call.

webSocket

1. Server sees that a price has changed and immediately sends a message to each client.
2. Client receives the message about new price.

Rest/Ajax

1. Client sets up a polling interval
2. Upon next polling interval trigger, client creates socket connection to server
3. Server receives request to open new socket
4. When connection is made with the server, client sends request for new pricing info to server
5. Server receives request for new pricing info and sends reply with new data (if any).
6. Client receives new pricing data
7. Client closes socket
8. Server receives socket close

As you can see there's a lot more going on in the Rest/Ajax call from a networking point of view because a new connection has to be established for every new call whereas the webSocket uses an already open call. In addition, in the webSocket cases, the server just sends the client new data when new data is available - the client doens't have to regularly request it.

A webSocket can also be faster and easier on your networking infrastructure simply because fewer network operations are involved to simply send a packet over an already open webSocket connection versus creating a new connection for each REST/Ajax call, sending new data, then closing the connection. How much of a difference/improvement this makes in your particular application would be something you'd have to measure to really know.

Availability means the ability of the application user to access the system, If a user cannot access the application, it is assumed unavailable. High Availability means the application will be available, without interruption. Using redundant server nodes with clustering is a common way to achieve higher level of availability in web applications.

Availability is commonly expressed as a percentage of uptime in a given year.

ACID is a acronym which is commonly used to define the properties of a relational database system, it stand for following terms

• Atomicity - This property guarantees that if one part of the transaction fails, the entire transaction will fail, and the database state will be left unchanged.
• Consistency - This property ensures that any transaction will bring the database from one valid state to another.
• Isolation - This property ensures that the concurrent execution of transactions results in a system state that would be obtained if transactions were executed serially.
• Durable - means that once a transaction has been committed, it will remain so, even in the event of power loss.

Edge servers are the CDN servers used to cache content retrieved from your origin server or storage cluster. Another term often closely related to edge server is point of presence (POP). A POP refers to the physical location of where the edge servers are located.

That POP can have multiple edge servers caching content at that location.

Being able to deliver parts of a website from various locations helps decrease the distance between the visitor and the web server, thus reducing latency. This is exactly what CDN edge servers achieve.

Some of the disadvantages of REST are:

• Since there is no contract defined between service and client, it has to be communicated through other means such as documentation or emails.
• Since it works on HTTP, there can’t be asynchronous calls.
• Sessions can’t be maintained.

In software engineering, Don't Repeat Yourself (DRY) or Duplication is Evil (DIE) is a principle of software development.

Always keep static contents like images, css, JavaScript cacheable, with expiration date of 2 to 3 days. Never keep expiry date too high.

Dynamic contents should be cached for few hours only.

As we start to model more and more complex logic, we have to deal with the problem of managing business processes that stretch across the boundary of individual services.

• With orchestration, we rely on a central brain to guide and drive the process, much like the conductor in an orchestra. The orchestration style corresponds more to the SOA idea of orchestration/task services. For example we could wrap the business flow in its own service. Where the proxy orchestrates the interaction between the microservices like shown in the below picture.

• With choreography, we inform each part of the system of its job, and let it work out the details, like dancers all find‐ ing their way and reacting to others around them in a ballet. The choreography style corresponds to the dumb pipes and smart endpoints mentioned by Martin Fowler's. That approach is also called the domain approach and is using domain events, where each service publish events regarding what have happened and other services can subscribe to those events.

System is Resilient if it stays responsive in the face of failure. This applies not only to highly-available, mission critical systems — any system that is not resilient will be unresponsive after a failure.

Resilience is achieved by:

• replication,
• containment,
• isolation and
• delegation.

Failures are contained within each component, isolating components from each other and thereby ensuring that parts of the system can fail and recover without compromising the system as a whole. Recovery of each component is delegated to another (external) component and high-availability is ensured by replication where necessary. The client of a component is not burdened with handling its failures.

It is the process of improving the performance of the database by adding redundant data.

Elasticity means that the throughput of a system scales up or down automatically to meet varying demand as resource is proportionally added or removed. The system needs to be scalable to allow it to benefit from the dynamic addition, or removal, of resources at runtime. Elasticity therefore builds upon scalability and expands on it by adding the notion of automatic resource management.

Optimistic Locking is a strategy where you read a record, take note of a version number (other methods to do this involve dates, timestamps or checksums/hashes) and check that the version hasn't changed before you write the record back. When you write the record back you filter the update on the version to make sure it's atomic. (i.e. hasn't been updated between when you check the version and write the record to the disk) and update the version in one hit.

If the record is dirty (i.e. different version to yours) you abort the transaction and the user can re-start it.

This strategy is most applicable to high-volume systems and three-tier architectures where you do not necessarily maintain a connection to the database for your session. In this situation the client cannot actually maintain database locks as the connections are taken from a pool and you may not be using the same connection from one access to the next.

One such fallacy of distributed computing is that networks are reliable. They aren't. Networks and parts of networks go down frequently and unexpectedly.

A partition is when the network fails to deliver some messages to one or more nodes by losing them (not by delaying them - eventual delivery is not a partition).

The basic idea of CAP proof is that if a client writes to one side of a partition (namely, network fails), any reads that go to the other side of that partition can't possibly know about the most recent write. The proof of CAP relied on a total partition. In practice, these are arguably the most likely since all messages may flow through one component; if that fails then message loss is usually total between two nodes.

• The canonical definition of a DTO is the data shape of an object without any behavior. Generally DTOs are used to ship data from one layer to another layer across process boundries.

• ViewModels are the model of the view. ViewModels typically are full or partial data from one or more objects (or DTOs) plus any additional members specific to the view's behavior (methods that can be executed by the view, properties to indicate how toggle view elements etc...). In the MVVM pattern the ViewModel is used to isolate the Model from the View.

• Thread-safe code is code that will work even if many Threads are executing it simultaneously within the same process. This often means, that internal data-structures or operations that should run uninterrupted are protected against different modifications at the same time.
• Another definition may be like - a class is thread-safe if it behaves correctly when accessed from multiple threads, regardless of the scheduling or interleaving of the execution of those threads by the runtime environment, and with no additional synchronisation or other coordination on the part of the calling code.

It's proofed by construction. Basically we demonstrate a single situation where a system cannot be consistent and available in the same time:

If a client writes to one side of a partition, any reads that go to the other side of that partition can't possibly know about the most recent write. Now you're faced with a choice: do you respond to the reads with potentially stale information, or do you wait (potentially forever) to hear from the other side of the partition and compromise availability?