Question & Answer: Lessons In Building Scalable Systems Summary…..

Answer- Building Scalable Systems Summary:

• Scalability is the capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged to accommodate that growth.
• At the simplest level, scalability is about doing more of something. This could be responding to more user requests, executing more work or handling more data. While designing software has its complexities, making that software capable of doing lots of work presents its own set of problems.

• Some principles and guidelines for building scalable software systems:

1. Decrease processing time:

One way to increase the amount of work that an application does is to decrease the time taken for individual work units to complete. For example, decreasing the amount of time required to process a user request means that you are able to handle more user requests in the same amount of time. Here are some examples of where this principle is appropriate and some possible realisation strategies.

• Collocation : reduce any overheads associated with fetching data required for a piece of work, by collocating the data and the code.
• Caching : if the data and the code can’t be collocated, cache the data to reduce the overhead of fetching it over and over again.
• Pooling : reduce the overhead associated with using expensive resources by pooling them.
• Parallelization : decrease the time taken to complete a unit of work by decomposing the problem and parallelizing the individual steps.
• Partitioning : concentrate related processing as close together as possible, by partitioning the code and collocating related partitions.
• Remoting : reduce the amount of time spent accessing remote services by, for example, making the interfaces more coarse-grained. It’s also worth remembering that remote vs local is an explicit design decision not a switch and to consider the first law of distributed computing – do not distribute your objects.

2. Partition:

This involves breaking up that single piece of the architecture into smaller more manageable chunks. Partitioning that single element into smaller chunks allows you to scale them out and this is exactly the technique that large sites such as eBay use to ensure that their architectures scale.

3. Scalability is about concurrency:

Scalability is inherently about concurrency; after all, it’s about doing more work at the same time. If concurrency isn’t thought about, you have systems where data can easily become corrupted. On the other hand, too many guards around concurrency lead to systems that are essentially serial in nature and limited in the degree to which they can scale. Concurrent programming isn’t that hard to do, but there are some simple principles that can help when building scalable systems.

• If you do need to hold locks (e.g. local objects, database objects, etc), try to hold them for as little time as possible.

4. Requirements must be known:

In order to build a successful software system, you need to know what your goals are and what you’re aiming for. While the functional requirements are often well-known, it’s the non-functional requirements (or system qualities) that are usually absent. If you do genuinely need to build a piece of software that is highly scalable, then you need to understand the following types of things up-front for the critical components/workflows.

• Target average and peak performance (i.e. response time, latency, etc).
• Target average and peak load (i.e. concurrent users, message volumes, etc).
• Acceptable limits for performance and scalability.

It might be that performance isn’t critical, but you need to know this information as early as possible, because your approach to dealing with scalability will be driven by the performance requirements.

5. Test continuously:

Once you understand the requirements you can start designing and building the solution. The design that we come up with and the code that we write is static in nature, so you can never quite tell how it will work until it is executed. It’s for this reason, then, that all decisions on performance and scalability should be backed up by evidence, and this evidence should be gathered and reviewed from the start of the project and on a continuous basis thereafter. In other words; set measurable goals throughout the system, verify and measure the real performance and consider performance at all stages of the project.

6. Architect up front:

Probably the most important principle for building scalable systems is that, if you need your system to exhibit this characteristic, you have to design it in up front.One of the pitfalls that many people (including myself) have fallen into, particularly during the early days of J2EE, was that you could build an application and have it automatically scale up and scale out. Applications designed to scale out will almost always scale up, yet applications designed to scale up will almost never scale out. Most applications can be scaled up by running them on more powerful hardware, but scaling out is a more complex problem.

7. Look at the bigger picture:

Finally, remember to take the bigger picture view – look at the wood before looking at the trees. It’s really easy for us to get carried away tuning components at the fine-grained code level, but ultimately it’s the system as a whole that needs to be optimised. Focus on the end-to-end performance and scalability, sacrificing local optimisations if necessary.If you’re struggling to meet your performance and scalability goals, it’s worth questioning whether you have chosen the right architecture. Again, look at the bigger picture and ensure that somebody is taking on the architect role.