Design Software Systems Like a Senior Engineer – Complete Course

[00:00] Learn software system design and pass [00:02] interviews. This course provides a [00:04] practical five-step framework for [00:07] structuring your thinking and tackling [00:09] vague requirements under pressure. [00:12] Additionally, the course walks through [00:14] realworld examples designing systems [00:17] similar to Spotify, Booking.com, and [00:19] WhatsApp from scratch. Hike Simon [00:22] developed this course. [00:24] >> Most software engineering roles today [00:26] require system design knowledge. Whether [00:28] it's for the job itself or for [00:30] interviews, you are expected to know how [00:33] to design systems at scale. And [00:35] especially for senior and staff level [00:38] positions, it's one of the deciding [00:40] factors for the company. That's why I [00:42] put together this complete system design [00:45] interview guide with real production [00:47] system examples that show up the most [00:49] often in system design interviews. [00:52] First, we'll start with the framework to [00:54] know how to structure your thinking, [00:56] what clarifying questions to ask, and [00:58] how to make sure you cover everything [01:00] the interviewer is looking for without [01:03] going off track. And then we'll apply [01:06] this same framework across three [01:08] different system design examples so that [01:10] you can see how the same framework [01:12] applies to different production systems. [01:14] This is the most practical framework [01:16] that I know for passing system design [01:18] interviews. Early in my career, I failed [01:21] a lot of these interviews when applying [01:23] to senior roles. And looking back now, [01:27] there is one pattern that was the same [01:29] across all of them, which is system [01:31] design was almost always part of the [01:34] conversation. And you can argue that [01:36] nowadays, this is also a requirement for [01:39] mid-level positions, especially now when [01:42] all of the software engineers use AI to [01:45] write the code implementation. Companies [01:48] would like to know whether you [01:49] understand the bigger picture and how [01:52] components talk to each other, how to [01:54] design solutions from scratch and [01:56] discuss the trade-offs. Of course, the [01:59] depth of knowledge is different [02:01] depending on whether you apply for mid [02:03] to senior roles, but still you have to [02:06] be prepared for these type of [02:08] interviews, whether you're mid, senior [02:10] or above. So, this shows up in different [02:13] ways. Actually the most common one that [02:15] we know is the whiteboarding style where [02:19] there is a session where you design a [02:21] full system or a feature from scratch. [02:24] But there are actually three more types [02:25] to this. Another way can be discussion [02:28] about bottlenecks in the company's [02:30] existing infrastructure and see if you [02:33] can suggest solutions to this and you [02:36] have the basic knowledge of system [02:38] design. Another type is walking through [02:41] your past projects. So questions about [02:43] your past projects to see if you [02:45] actually understand the architecture of [02:48] what you built there. And it can also be [02:50] conceptual questions that test the depth [02:53] of your knowledge about system design [02:55] concepts. But even if you learn all the [02:58] concepts of system design, you still [03:00] need a framework to navigate a 45 or [03:04] 60inut discussion and make the most out [03:06] of that time. That's exactly why we have [03:09] this framework and guide. And another [03:11] important thing before we'll get to [03:13] discuss the framework is that most [03:15] engineers think they are only being [03:17] evaluated on the system they design [03:19] which is just one out of five dimensions [03:22] that you're being tested on. And just by [03:25] knowing what these dimensions are will [03:27] help you way more to prepare for these [03:30] type of interviews. Before getting to [03:32] the dimensions, let's first understand [03:34] why this feels hard compared to other [03:37] type of interviews like behavioral [03:39] rounds or live coding. For example, if [03:42] it's a live coding round, this usually [03:45] is either a lead code style problem or [03:47] some real world challenge. But either [03:50] way, there are clear inputs and outputs, [03:53] right? Let's say if it is a lead code [03:55] style challenge, then you have a [03:57] function which accepts these type of [03:59] arguments. you know exactly what the [04:01] type of the argument is. You know [04:03] exactly what you're expected to return [04:06] as an output from the function. So it is [04:08] clear whether your solution works or not [04:11] because you can copy paste this code and [04:13] try to run it and you can also double [04:15] check by running test cases on a [04:18] whiteboard without even running the [04:20] code. But system design is actually the [04:24] opposite of all of that. First of all, [04:26] the requirements are very vague and it [04:28] is your job to clarify those [04:30] requirements and lock in the scope and [04:32] also the output which is the solution [04:35] that you will propose. There is no [04:37] single right answer to it. And this type [04:39] of interview is testing how you think [04:42] not just what you know or not whether [04:44] you can come up with the right solution [04:47] because there is no single right answer [04:49] to the question that you will get during [04:51] system design interview. With that, [04:53] let's see what are you actually being [04:55] evaluated on. Most candidates prepare by [04:58] just memorizing couple of architectures [05:00] like Spotify, WhatsApp and so on, which [05:03] is of course a step forward if you know [05:06] couple of variants of systems and what [05:09] components go into this. But you need to [05:11] also understand the other four [05:13] dimensions that interviewers score you [05:16] on. So let's start with the first one [05:18] which is communication and [05:20] collaboration. This is whether you ask [05:22] clarifying questions, whether you [05:24] validate assumptions as you go or you [05:27] just make assumptions and start building [05:29] the system. And this kind of testing how [05:31] you would work in a real world setup as [05:33] well. Let's say your manager assigns a [05:36] task to you like a big feature or a big [05:39] system service you need to build from [05:41] scratch. What do you do in this kind of [05:44] situations? If some things are not [05:45] clear, do you just make assumptions, [05:48] build a solution and then figure out [05:50] that it was not what they were [05:52] expecting? So now you just wasted a lot [05:54] of time in between and you need to come [05:57] back and write another solution which is [05:59] actually what they were looking for or [06:01] you clarify this in the beginning. So [06:03] they are trying to also understand [06:06] whether it's easy to work with you or [06:08] not. Whether you are good at [06:10] communication skills apart from [06:13] technical knowledge. The next one is [06:15] whether you have structured way of [06:17] thinking and approaching problems or can [06:20] you break down the vague problem that [06:22] you just got into specific outlined [06:26] actions that you will take or you just [06:28] jump between random decisions. you go [06:30] back and forth between different designs [06:33] phases and so on. So strong candidates [06:35] here usually show that they can organize [06:37] their faults even when the problem is [06:40] not very clear. They at least have an [06:42] outline of what are the steps we're [06:44] going to take and that every step is [06:47] clear what we're working on right now [06:48] and what's up next. The next one is your [06:52] technical judgment. So when you pick a [06:54] solution let's say database a cache a [06:57] message queue can you explain your [06:59] decision why like why you chose [07:02] specifically SQL database why you chose [07:04] the caching mechanism or the specific [07:07] type of message queue that you chose. [07:10] This part shows that if you choose some [07:12] tools to be used in a system, you choose [07:14] them based on the requirements not just [07:17] out of habit or not just because you [07:19] know what Kafka is but maybe there was [07:22] another message cue that was a better [07:24] fit for this type of system. The fourth [07:26] dimension is your practical depth. This [07:30] part differentiates between two [07:32] candidates who might have same [07:34] theoretical knowledge but one of them [07:36] also has practice with building these [07:39] type of systems. So it will show up by [07:42] seeing whether you think about failure [07:44] modes about the scale about things that [07:47] can go wrong in the system because if [07:49] you just learned at the theory level you [07:51] wouldn't be able to catch these type of [07:54] failures in the system. So this part [07:56] specifically is for interviewers to spot [07:59] engineers who have actually shipped [08:01] production systems versus those who just [08:04] learned from theory. And then lastly, [08:06] it's the time management as well because [08:08] these type of interviews are usually [08:10] either 45 or 60 minutes and here you [08:14] need to not get stuck on one area and [08:17] run out of time by doing highle design [08:19] only. And it's worth asking your [08:21] interviewer early where they want to [08:24] focus on or proposing it yourself and [08:27] getting alignment to make sure that [08:29] you're managing your time effectively. [08:31] And along the process of an interview, [08:34] you need to always know what's the time [08:37] and how much is left, how much you spent [08:39] on this section. And if you're spending [08:42] a lot of time on it, it's time to move [08:43] on. So you need to also show time [08:45] management which again all of these five [08:48] pillars translate to how you will work [08:52] in a real environment. If you can't [08:54] manage your time during an interview [08:55] then you're also not good at managing [08:57] your time during work. If you don't have [09:00] practical depth then you also won't be [09:03] able to ship projects confidently if you [09:06] only know the theory. Right? And same [09:08] applies to all of these five pillars. [09:10] And now that we know these five pillars, [09:13] let's focus on the time management part [09:16] and what are the stages you need to take [09:18] to complete a system design solution. [09:21] There are five steps for any system [09:23] design problem. This is a repeatable [09:26] structure you can apply to any [09:28] whiteboarding system design interview. [09:30] And the goal here is to stay organized [09:32] and show the interviewer how you think [09:34] through the problems throughout these 45 [09:37] or 60 minutes. 60 minutes is actually [09:40] more common because there is a lot to [09:42] cover in this type of interviews. So [09:44] assuming it's 60 minutes, let's see what [09:46] are the steps we will take and how much [09:47] time we will dedicate to each step. The [09:50] very first one is clarifying [09:52] requirements. You don't want to start [09:54] designing anything before you understand [09:56] the scope and this is the most common [09:59] mistake candidates make. So let's see [10:01] what these five steps are at a high [10:03] level and then we'll get to discuss each [10:04] of them in more detail. The very first [10:07] one is you need to clarify requirements [10:09] at least for 5 to 10 minutes depending [10:12] on the length of the interview. You need [10:14] to make sure you have all the functional [10:16] and non-functional requirements before [10:18] moving on to designing anything. After [10:21] that you can either start by data and [10:24] API layer and then get into the highle [10:27] architecture or this part is kind of [10:30] variable. You can also start from [10:31] architecture and get into the data and [10:33] API layer afterwards. whichever feels [10:36] easier for you. For each of these you [10:39] would spend 15 to 20 minutes not more [10:42] and then you would move on to discussing [10:45] scale and edge cases towards the end [10:47] once you have the highle architecture [10:49] the data layer and APIs. So here you [10:52] might discuss either how you would scale [10:55] this system or if reliability is more [10:58] important for that type of system you [10:59] would discuss other edge cases and how [11:02] you would handle the failures. And the [11:04] last part is to wrap up and talk about [11:08] anything that you could improve on this [11:10] system but you didn't have time for it. [11:12] So just briefly mentioning this before [11:14] wrapping up the interview. Let's start [11:17] with the first step which usually takes [11:19] up to 10 minutes or less and this is [11:21] where you clarify the problem first [11:23] before getting into the solution. You do [11:26] not want to jump into designing anything [11:28] yet before clarifying the functional and [11:31] non-functional requirements. By [11:33] functional requirements, we mean what [11:36] are the main use cases of this system [11:38] that you're building, what is explicitly [11:41] out of scope. Because if you're given [11:43] something to design like Spotify, you [11:46] can never design it in 60 minutes. This [11:48] usually takes multiple teams, months of [11:52] efforts if not years to build and design [11:55] all of this. Right? So of course you [11:57] can't do it as a single engineer during [11:59] 60 minutes. The goal here is to see what [12:02] are the main use cases we are focusing [12:04] on. What we leave out of scope and it [12:07] also helps to clarify who are the users [12:09] and what type of platform we are [12:11] building. Then you also focus on [12:13] nonfunctional requirements which answers [12:16] what are the constraints for this [12:18] system. For example, how many daily or [12:20] monthly active users you will have? What [12:23] is the readto write ratio? You would [12:26] clarify the latency requirements and [12:28] also what is the acceptable downtime [12:30] which refers to availability of the [12:33] system before moving on to the next [12:34] stage. And it's a good idea to always [12:37] check at the end of each phase whether [12:40] you're good to move on because if you [12:42] missed some things during this stage, [12:45] interviewer can give you hints. For [12:47] example, you may failed to clarify some [12:50] of the important functional [12:51] requirements. This is a good place to [12:54] ask something like does this scope and [12:56] these requirements make sense before I [12:58] move forward to highle design. If they [13:01] say yes, then you clarified everything. [13:03] If they give you a hint then usually you [13:05] missed something in functional and [13:07] non-functional requirements. So you [13:09] would clarify that before moving on to [13:11] the next stage. And next is step two [13:14] which is either the data and API layer [13:16] or highle design depending on which one [13:19] is easier for you. If you think it helps [13:21] to start with data layers understand how [13:23] entities connect to each other and then [13:25] get to the high level design that's the [13:28] approach to take. If you think imagining [13:30] it high level first helps you understand [13:33] the user flow then you start with high [13:35] level design. I personally start with [13:37] highle design then get here but [13:40] sometimes I also start with data and API [13:42] layer. This part is to model what you're [13:44] storing and how clients will interact [13:47] with your system. When it comes to data [13:49] model we should focus on what are the [13:52] main entities for this type of system. [13:55] Let's say if it's something like [13:56] YouTube, then the main entities are [13:58] videos and users. And you would also map [14:01] out the relationship between them. What [14:04] are the primary keys, foreign keys, key [14:06] indexes that we will use in the database [14:09] and how big each record can get. This is [14:12] also a good place to clarify which data [14:14] needs strong consistency and where you [14:17] can afford eventual consistency. [14:20] Next part of this stage is to also [14:22] define the API contracts like what are [14:24] the main endpoints with request and [14:26] response formats. Here it will also be [14:29] good if you can talk about pagination, [14:31] error handling, rate limiting, [14:33] versioning and also follow the best [14:36] practices of rest API in case you're [14:38] using REST type of APIs which is in most [14:41] cases that's the case and this is also [14:44] where you will make the database choice [14:46] whether it's poss [14:48] MongoDB whatever you pick you should [14:51] also explain why given the requirements [14:54] you went with that specific option and [14:57] also or walk through the tradeoffs of [14:59] what are other things you considered but [15:01] you didn't choose. The other important [15:04] part is the highle architecture which [15:06] takes another 15 minutes. This is where [15:09] you spend a big portion of a time design [15:12] the components and show how data flows [15:14] through them. In case nothing comes to [15:17] your mind, you can start with these four [15:19] components which are almost always [15:21] present in majority types of systems. [15:24] These are API servers which will be [15:27] accepting the requests, a load balancer [15:29] that distributes traffic between them, [15:32] database where you persist the data and [15:35] optionally cache layers that caches some [15:39] common data that you're fetching from [15:41] the database. At this step, it usually [15:44] helps to map out the user journey from [15:47] front end to back end all the way to the [15:50] database. So what's the front end? It's [15:53] some sort of web app or a mobile app or [15:56] desktop app and then it connects to your [15:58] API gateway and load balancers which [16:00] will distribute among API servers. You [16:03] might have caching layer among these [16:06] servers. Optionally you may have message [16:08] cues that decouple the services and you [16:11] almost always have a database to store [16:13] the data in the back end. So walk the [16:16] interviewer through how a request moves [16:18] through your system from client through [16:20] load balancer API server into cache and [16:23] database and back all the read and write [16:26] operations. If you're using caching make [16:29] sure to also mention the TTL and what [16:32] invalidation mechanism you use. And for [16:34] different parts of your system, you [16:37] should also walk through your choice [16:39] between consistency, availability, and [16:41] partition tolerance, which is coming [16:44] from the CP theorem, which means some of [16:47] your components may need different [16:49] approaches. And explaining that shows [16:51] that you understand the theorem and also [16:54] the trade-offs that you're making for [16:56] this system. The fourth step is [16:59] discussing bottlenecks, scale and [17:01] reliability which can be anywhere from 5 [17:04] to 10 minutes depending on how much time [17:07] you spend on highle design data and API [17:09] layer. This is one of the important [17:12] parts where you show that you can think [17:14] about what breaks under heavy load not [17:17] just the happy paths and this is also [17:20] one of the clearest signals that [17:21] separates who's a mid-level candidate [17:24] here who's a senior level candidate. [17:26] This is where you discuss things like [17:28] single points of failure. You find them [17:31] and propose solutions on what we can do [17:33] to fix this single point of failure. [17:36] This is where you will talk about [17:37] fallbacks and failures. For example, [17:40] whether you have mechanisms to fall back [17:43] to a slower but functional path in case [17:46] something goes wrong. Here you can also [17:48] expect to talk about the redundancy like [17:51] multi availability zone replicas for [17:54] high availability application servers to [17:57] auto scale across availability zones [18:00] distributed caching with replication and [18:03] so on. Another point is whether you can [18:06] spot how you will be monitoring logging [18:08] and alerting issues for this system. [18:11] This part is often overlooked but it's a [18:13] strong signal that you think like a [18:15] senior engineer. This is where you would [18:17] cover API latency as well, error rates [18:20] and thresholds, what alerts you will set [18:23] up for this type of system and so on. [18:26] And again, it is always a good idea to [18:28] check in with the interviewer whether [18:30] there are specific failure scenarios. [18:32] For example, if you were discussing [18:33] failures that they would like you to [18:36] address and same goes for highle design, [18:38] whether you covered everything they were [18:40] looking for and same goes for database [18:43] and API design. The last part is where [18:46] you can expect to discuss trade-offs and [18:48] extensions. So this might be an [18:51] open-ended question like whether you can [18:53] spot some things that can be improved in [18:55] the system. You usually wrap up by [18:57] summarizing the key decisions you made, [19:00] what you prioritized for this system, [19:02] what you sacrificed because it's always [19:04] about trade-offs in system design. And [19:07] you should always be explicit about the [19:10] why behind the sacrifices. And this part [19:13] is also to see whether you will propose [19:15] some possible extensions for the future. [19:18] For example, if you had more time and [19:21] the load scaled to 10x or more, what [19:24] would you do to extend this service or [19:27] system? So this is for you to show that [19:30] you're thinking beyond the problem that [19:32] was discussed here. So what if we scale [19:34] this? What if something goes wrong? And [19:37] remember that you cannot propose a [19:39] perfect design in 60 minutes. So of [19:42] course there are always things that you [19:44] can improve. The goal here is not to [19:47] check whether your design was perfect or [19:49] not because it is not. You just need to [19:52] show that you could extend this. If you [19:54] had more time, you had more ideas on how [19:56] to improve this system. Now that's the [19:59] five-step system to manage your time. [20:01] But let's also come back to these five [20:04] dimensions that you're being evaluated [20:06] on and see what you should do and what [20:09] you should avoid doing during the [20:11] interview. So first thing I recommend [20:13] you do is ask clarifying questions [20:15] before designing. This is the first [20:17] step. So if you proceed jump into a [20:21] solution before understanding the scope [20:23] that is already a red flag for the [20:25] interviewer. So always this is a [20:27] mandatory step before you get into [20:29] designing anything. Another common issue [20:32] I see is engineers get stuck into one of [20:35] these extremes. They either stay silent [20:38] and draw diagrams without talking to the [20:41] interviewer or they talk all the time [20:44] and try to fill in all the silence [20:46] without inviting the interviewer for [20:49] their opinion. What you should do [20:51] instead is treat the interviewer as your [20:53] colleague. This is like two software [20:56] engineers in the same company who are [20:58] working on a solution together. So after [21:01] each major decision or each of the [21:04] phases, it's a good idea to first double [21:06] check whether this is what they were [21:08] looking to get out of this part. And if [21:12] you missed anything, for example, if it [21:14] was functional requirements that you [21:15] were clarifying, whether there are some [21:17] more things that you didn't ask for, but [21:20] those are still expected to be part of [21:22] the system. The other thing is always [21:25] suggest multiple approaches before [21:27] committing especially when there are [21:29] multiple ways to go about things. For [21:32] example, if you chose SQL and even if [21:36] it's a very obvious choice, for example, [21:39] it was let's say payment system where [21:42] consistency is the most important thing. [21:46] If you chose SQL without explaining why, [21:49] we just assume you didn't know or the [21:51] interviewer then has to follow up and [21:53] ask a question why you chose SQL, but [21:56] they may also not. So if you don't [21:58] explain, they would assume you don't [22:00] know if you chose SQL, still mention why [22:03] you didn't go with something like NoSQL [22:06] and what are the tradeoffs between those [22:08] solutions. And same goes for any other [22:10] decision that you will make where there [22:12] are more than one approaches to the [22:15] problem. So this also ties to the [22:18] tradeoffs which is you should always [22:20] explain trade-offs when picking any [22:22] technology. If you just pick them [22:24] without justifying the choice then it's [22:26] again another red flag during the [22:28] interview. And this last one overlaps [22:30] with the second point a bit which is you [22:32] need to check in at each major step. For [22:35] example, if you did the highle design [22:37] and you just move on without asking if [22:39] there are things we forgot to address, [22:41] then there might be things that you miss [22:43] during this step. [snorts] So to [22:45] summarize, use this framework to manage [22:48] your time and remember to always involve [22:50] the interviewer into the discussions [22:52] whenever you're working on any of the [22:55] stages of this interview. Remember that [22:57] this is a conversation between two [22:59] engineers, not a monologue where you [23:02] talk all the time and also not a [23:04] take-home assessment where you just work [23:07] on the solution without communicating [23:09] with them. And to quickly recap what are [23:12] these five steps, it's first to clarify [23:14] the requirements functional and [23:16] non-functional and understand what's in [23:18] scope and what's out of scope. Next, you [23:21] get to the models of data and API [23:24] layers. Then you come up with the high [23:26] level design with all the components [23:28] that are involved and how the data flows [23:31] for read and write operations. [23:34] Next you discuss the scale, failure [23:36] modes and reliability of the system. And [23:40] last part is usually open-ended which is [23:42] where you reflect on the system you [23:44] built. What are the tradeoffs you made [23:47] and how would you improve this if you [23:49] had more time? Building a music [23:51] streaming platform like Spotify is a [23:54] classic system design problem and that [23:56] is because it includes audio delivery, [23:58] metadata management, object storage and [24:01] everything in between that you need to [24:02] know to design things from scratch and [24:05] especially if you're a mid-level [24:07] developer who struggles to design [24:09] systems or features from scratch, then [24:11] this will be very helpful for you. So [24:14] let's see how we can design this in [24:16] seven simple steps. First we'll start by [24:18] understanding the requirements and [24:20] planning the capacity. Then we'll come [24:23] up with the highle architecture and the [24:26] system flow of read and write [24:28] operations. After which we'll need to [24:30] design the API and also the storage and [24:33] database. And then finally we'll move on [24:36] to scalability and understanding how we [24:38] can scale this system for let's say 50 [24:41] million users or above. And at the end, [24:44] I will also show you the pyramid that [24:45] separates senior engineers from [24:47] mid-level developers so that you can [24:49] understand what are the skills that you [24:51] need to learn to be able to design such [24:53] systems from scratch. No matter if it's [24:55] system design for Spotify or for Google [24:58] Doc or for any other system, let's start [25:00] from understanding requirements as a [25:02] first step because without this we [25:04] cannot move any further if we don't know [25:06] what we are building here. So we are [25:08] looking to build a music streaming [25:10] platform for about 500,000 users and it [25:14] will handle over like 30 million songs [25:17] in this platform. So the core [25:19] requirements here will be that artists [25:22] need to be able to upload their songs to [25:24] the platform. On the other end, users [25:27] need to be able to search and play the [25:29] songs that are uploaded by these [25:31] artists. They need to be able also to [25:34] create and manage playlists on their [25:36] accounts. For this we also need to have [25:39] a profile for users so that they can [25:42] maintain their own playlists and some [25:44] basic monitoring and observability like [25:47] health checks for the servers, error [25:49] tracking, performance metrics and so on. [25:52] For audio formats, we will use OG and [25:54] AAC type of files which are just audio [25:57] file formats that compress the music [25:59] without losing the quality and they also [26:02] take up less space. For example, we can [26:04] use 64 kilobits per second for mobile [26:07] data saving and 128 for standard [26:11] quality. And for premium users, we can [26:13] deliver 320 kilobits per second. And on [26:17] average, we also need to know the size [26:18] of these files like what is the average [26:21] size of a music file. And let's assume [26:23] that for normal audio quality, which is [26:25] the 128 kilobits per second, it is 3 [26:29] mgabytes on average. But this is also [26:31] something you need to clarify before [26:33] moving on to the next stage. Once we [26:35] have the requirements, the next step is [26:37] to plan the capacity of this system and [26:40] especially for song storage, strong [26:43] metadata and also the user metadata and [26:46] daily bandwidth that we'll be dealing [26:48] with. for song storage. Since we assumed [26:51] that 3 megabytes is the average size of [26:53] a song and we have 30 million songs that [26:56] we are working with, this translates to [26:59] 90 terabytes on average of raw audio [27:02] data. And this doesn't include the [27:05] replicas across different regions which [27:07] might be duplicates of the same song. [27:10] And it also doesn't include the [27:12] versioning overhead when artists [27:14] re-upload the songs for certain reasons. [27:16] Let's say they want to change something [27:18] and they just end up re-uploading the [27:20] same song. So that's why we will also [27:23] need to estimate this like at least two [27:25] or three times higher than the amount [27:27] that we have here which is 90 terabytes. [27:30] For song meta data let's say each song [27:33] needs a title, artist references, [27:35] duration, file urls and so on. So we [27:38] will need at least like 100 bytes per [27:41] song to store the metadata on average. [27:44] And if we assume that it's 100 bytes, [27:46] then we'll need to multiply that by the [27:48] 30 million, which will end up being 3 [27:51] GB. So this is practically like nothing [27:54] compared to the song storage. As you can [27:56] see, the main storage that we'll be [27:58] dealing with in this system is the song [28:01] storage, which will be an object [28:02] storage. For user metadata, again, this [28:05] won't be something huge like let's say [28:07] we need to obtain the user profiles, [28:09] preferences, and playlist data. If we [28:12] assume it's like 1 kilobytes per user [28:14] profile data that we'll be storing and [28:16] we have the 500k users which we [28:19] clarified in the first requirements [28:21] here. That means that we'll be dealing [28:24] with 500 * 1 kilobyte which will be just [28:27] half GB of storage for profiles. And the [28:31] fourth thing we need to estimate is the [28:33] daily bandwidth that we'll be using. The [28:35] average listen time let's say is 3.5 [28:38] minutes. This is also something that [28:40] needs to be clarified in the previous [28:42] step. And if you are using the 128 to [28:46] 160 kilobits per second, that is roughly [28:50] 3 to 4 mgabytes per streaming. And let's [28:53] assume each user streams like 10 to 15 [28:56] songs daily. This will lead to [28:58] significant egress costs. Igress costs [29:01] are the fees that cloud providers charge [29:03] when data leaves their network to reach [29:06] to the users. Overall the key insight [29:09] that we get from the capacity planning [29:11] is that audio will dominate both storage [29:13] costs and bandwidth. The metadata is [29:16] just a small part or portion of this [29:19] system. So with that we can move on to [29:21] the highle architecture like a very high [29:24] level architecture for now. We'll need [29:26] to have some sort of mobile app or let's [29:28] say the web interface of the [29:30] application. This will be the userfacing [29:32] app that handles the UI, the user [29:35] searches, the playback controls and [29:37] playlist management. It will make HTTP [29:41] calls to the rest APIs. But before that, [29:43] we will have a load balancer that will [29:45] distribute these incoming requests [29:47] across multiple API servers. And these [29:50] servers will be the restful APIs. As an [29:53] algorithm for the load balancer, we can [29:55] use either roundroin or we can use the [29:58] list connections algorithm. And the load [30:01] balancer will also make the health [30:03] checks of the API servers and make sure [30:05] that they are always available. Next, we [30:08] have the API layer where we have our [30:10] restful APIs. Here as a validation, we [30:13] can use JWT tokens and query the [30:16] database song for metadata and URLs. [30:20] We'll use an SQL database for storing [30:22] the metadata and these servers will be [30:25] querying it from the SQL database. But [30:28] for audio files, we'll need some sort of [30:30] blob storage to store these files since [30:32] they are the biggest portion of our [30:34] system. And these will be queried with [30:37] generated signed URLs, which is a [30:40] one-time URL that expires after some [30:42] time. So that we keep the URLs safe and [30:45] they are not always accessible through [30:47] the same URL. An example of this blob [30:49] storage can be like AWS's S3 storage [30:53] which is a way for storing large files [30:55] like the OG files that we need to store [30:58] as part of this Spotify design and these [31:01] type of blob stoages offer virtually [31:04] unlimited scalability and it also comes [31:06] with built-in durability and cost [31:09] effectiveness for the large files that's [31:11] why this is a good solution for storing [31:13] these type of audio files with this [31:16] highle design. Now we can think about [31:18] the read and write workflows of the [31:21] system and let's start from the read [31:24] workflow which is users playing a song. [31:27] So the very first step is user hits the [31:30] play button and they play a song in the [31:32] mobile app. This will make a request [31:35] like get to songs/ ID. This is also [31:38] something we need to design in the API [31:40] design very soon in the next step. And [31:43] then after that we'll hit the load [31:44] balancer which will route the request to [31:47] the API server. API server as a first [31:50] step it will validate whether the user [31:51] is authenticated with the JWT token. And [31:55] if they are authenticated then we'll get [31:57] into this block where we will be [31:59] starting to play the song. So it will [32:02] first query the song metadata from SQL [32:05] database where we store the metadata of [32:07] the song and user profiles. It will [32:10] return the song details. Once we have [32:12] the metadata, we will also generate the [32:14] signed URL here in the API server and [32:17] then we will return the metadata plus [32:19] the signed URL of the song to the mobile [32:22] app. Then mobile app will range requests [32:25] for audio chunks to the blob storage [32:28] using the signed URL and blob storage [32:31] will stream the audio back to the mobile [32:33] app. Let's say we are also tracking some [32:35] analytics for each played song. For that [32:38] reason, we will also make another post [32:40] request to songs/ ID of the song / [32:43] SLplay to track the analytics. And here [32:47] we can send the metadata that we want to [32:49] store for later. And we will return [32:52] authentication error in case the JWT is [32:55] invalid. So all of the steps above won't [32:58] be handled if the user is not [33:00] authenticated. Next we have the right [33:03] workflow of this same system which is [33:05] artists uploading a media file which is [33:08] the audio file. So the first point again [33:11] is the mobile app. It will make a post [33:14] request to store the song with the meta [33:17] data and this will again reach to the [33:19] load balancer first. Then it will [33:21] forward the upload request to the API [33:24] server. In the API we will likely have [33:27] some file size limits. So we will first [33:29] validate the file format and also the [33:32] size and if it is valid then we will [33:34] continue with the next steps. If it is [33:37] invalid then we will just return the [33:39] rejected payload with unsupported format [33:42] or unsupported size of the file. [33:45] Otherwise as a first step we need to [33:46] store the audio file in the blob [33:48] storage. So we'll send it to the blob [33:50] storage which we are using and then [33:53] after that we will extract the audio [33:55] metadata like the duration the bite rate [33:58] and also the other things which we will [34:00] get in the post body like the title of [34:03] the song and so on. And once we have all [34:05] the metadata like the song type, the [34:08] artist, the title, we can insert this [34:10] into the SQL database. [34:13] Also once it is uploaded we can also [34:16] move it by hierarchal structure if we [34:18] want to like if we want to store the [34:21] file inside of the artists then by [34:23] artist ID then the album than the album [34:26] ID we can do that also in double [34:28] storage. Technically the object stoages [34:31] don't have hierarchal structures. This [34:33] is just an illusion that you have it but [34:35] this is also useful for us to keep track [34:38] of things like they are in a folder [34:40] format. Then if you are going to use any [34:43] sort of CDN, we'll need also some CDN [34:46] invalidation. CDN here will be used for [34:49] caching copies of the content like [34:51] artist pages, album metadata, song [34:53] details which will be closer to the [34:56] users for fast delivery and it will also [34:59] clear the old cached data so users see [35:01] the latest songs and artist updates [35:04] right away. And that will also be used [35:06] in this request to make sure we are [35:08] keeping everything up to date in the [35:10] CDNs. And lastly, we will also send the [35:13] response to the mobile app notifying [35:16] that the upload has been successful. The [35:19] next step for us will be to design this [35:21] API and the database in a bit more [35:24] detail. So let's start from the API [35:26] design. For the search and discovery, [35:29] we'll need a way to search different [35:31] content types with pagination. Let's say [35:34] user searches for a specific song. We [35:37] will need to make a request to / search [35:39] endpoint and provide the query which [35:41] will be the exact thing they search for. [35:44] And here we can pass more details like [35:46] if we know that the user is searching [35:48] for a song, we can pass the type as [35:50] song. And we'll pass the limit which can [35:53] be like 20 by default and the offset [35:56] which means that from where we need to [35:58] start the counter in the songs. Next we [36:01] will also get the trending songs and [36:03] optionally this can also be filtered by [36:05] genre or by some other things. So it [36:08] will be a get request to slash [36:10] songs/trending [36:12] and then we can pass the additional [36:14] filters if we want to and the limit on [36:17] how many songs we want to query from the [36:19] API. [snorts] Users will also need to [36:22] get all songs by a specific artist. [36:24] Let's say they are browsing the songs [36:26] for a specific artist and they want to [36:28] see the songs only of this artist. For [36:31] that we'll need to have an endpoint for [36:33] SL artists/ ID of the artist and then / [36:37] songs which will return the songs that [36:39] only belong to this artist and the limit [36:42] as always to limit the response from the [36:45] API. Next, we need to also control the [36:48] content access. The first endpoint here [36:50] will be to get the song metadata and the [36:53] URL of the audio file. It will be a get [36:56] request to / songs/ ID of the song which [36:59] will return all the metadata and the [37:02] streaming URL. We also need to have a [37:04] direct streaming endpoint like / songs/ [37:07] ID of the song and then slash something [37:10] like stream which will be an alternative [37:12] to the signed URLs. And we need to also [37:15] design for the playlist since the users [37:17] can maintain playlists and they will be [37:20] querying it back optionally with also [37:22] the song list that is included in the [37:25] playlist. So it can be a get request [37:28] to/plays [37:30] id which will just return the playlist [37:32] details and then if you also include the [37:34] query like include songs then it will [37:37] also return the songs that are included [37:39] in this playlist. Now let's see what are [37:41] the user actions that we can expect. For [37:44] example, regular users will also be [37:46] creating playlists, right? So for that [37:48] we need some sort of post endpoint [37:50] to/plays [37:52] as a metadata. We can expect like [37:54] something simple like just the name of [37:56] the playlist and whether it is public or [37:59] only available to this user. Then once [38:02] they create it, they will be also adding [38:04] songs to it which will be a put request [38:06] to the same playlist with the ID and [38:09] then / songs and here we can include the [38:12] ids of the songs like some new songs [38:15] that they are adding to this playlist [38:17] and the position where we are adding [38:19] this. Let's say they already have four [38:21] songs in the playlist or five songs and [38:23] then we are adding from position of five [38:26] or six. So we will be sending the [38:28] position along with it as well. [38:30] Similarly, they will also remove a song [38:32] from the playlist, which will be the [38:34] same request URL, but at the end, we [38:37] will also pass the song ID that we want [38:39] to remove from this playlist. And they [38:42] can also like or unlike a song so that [38:45] it is added to their favorites and that [38:47] will be done with a post request to [38:49] slash songs/ ID of that song and slike [38:53] which will be maintaining their [38:54] favorites in the database. And lastly, [38:58] we'll need some way to manage the users. [39:00] For example, we will need to fetch the [39:03] current users playlists with some [39:05] request like get to/ users/me/plays. [39:09] And we can also limit this with some [39:11] limit if the playlists are too many for [39:13] the user. Similarly, we need the songs [39:17] liked by the user. So here they are [39:19] adding it to their favorites. Here we [39:21] can use like users/me/ [39:23] either likes or liked songs. But [39:26] whatever we are using here, it will be [39:28] also nice if you use the same plural [39:30] here as well. So that it's following the [39:33] restful API principles. And then the [39:36] limit which can be like 50 or something [39:38] else and the offset where we are [39:40] starting to retrieve this from. And they [39:43] need to also be able to follow artists [39:45] to be notified from their latest songs [39:48] which will be done with a post request [39:50] to the users/me/follow [39:53] and then we will pass the artist ID to [39:55] follow the specific artist. That is it [39:58] for the API basic design. Now we can [40:00] move on to the data storage. First let's [40:03] talk about the blob storage we will be [40:05] using. Let's say it's an S3 in AWS or it [40:08] can be like Azure's blob storage. We [40:11] will need to use this for storing the [40:12] audio files because such large files [40:15] cannot be stored in SQL or some simple [40:18] databases. So we need some blob storage [40:21] or object storage for this. The audio [40:24] files here are immutable meaning once [40:26] you upload them you cannot change them. [40:28] So if you need to make a change to the [40:31] uploaded file, you need to re-upload it [40:33] and remove the old one. And we can also [40:36] organize them in a hierarchical [40:38] structure as we saw before. So we can [40:40] store them like artist/ the artist ID/ [40:44] album/ album ID and then /d this id. [40:48] This will be easier for us for [40:50] retrieving the files later. As a [40:52] database, we will be using some sort of [40:54] relational database like either [40:56] PostcarSQL or MySQL. And this is the [40:59] basic structure we will have in the [41:01] database. Let's start with users where [41:04] we will be storing the user ID and some [41:07] metadata about them like the email, the [41:09] password, hash created at which will be [41:12] the creation or like the sign up date [41:15] for this user when they last logged in, [41:18] subscription type and country. And this [41:21] can be many more metadata but this is [41:23] just the bare minimum that we need to [41:25] have. And with the user ID we will have [41:28] a foreign key linked to the owner ID of [41:32] playlists. So since we will have [41:34] playlists we also need to have the owner [41:36] of which playlist belongs to which user [41:40] and we will also have the playlist ID to [41:42] for linking this to the playlist items. [41:46] We need the name and created at for this [41:49] playlist. Then each playlist will have [41:51] some songs in it. Right? So here we can [41:53] use a join table. This can be something [41:56] like playlist items which will contain [41:59] first of all the playlist ID and also [42:01] the song ID and the position like where [42:05] is this song positioned in the playlist [42:07] and when is it added to the playlist if [42:10] we need to also maintain that. This will [42:12] basically be the table that joins the [42:14] playlists and the songs together. For [42:17] the songs again we will have the ID [42:19] which is the most important thing here [42:21] and some metadata like the title, [42:23] duration and so on and another important [42:26] part here is the file URL which we'll be [42:29] using to query the audio file to stream [42:32] it from the blob storage. Then we will [42:35] also have artists which we can store in [42:38] another type of table other than the [42:40] users for keeping this separate. So we [42:42] can have the artist ID and then the [42:44] metadata about them like the name, [42:46] country and so on. And we'll need to [42:49] connect the artists to the songs. For [42:52] that we can use another join table like [42:54] artist songs where we can keep track of [42:57] most importantly the song ID that they [42:59] uploaded and the artist ID like which [43:02] song belongs to which artist. And [43:05] additionally we can store some more [43:07] metadata here. But this is the bare [43:09] minimum that we need to have in this [43:11] table. And why we choose SQL here? [43:14] Because we need a lot of joins as you [43:16] can see for playlists and songs for [43:19] songs and artists. So this is a natural [43:22] selection for us to go with SQL type of [43:24] database. We can do even more complex [43:27] queries here like find all songs by [43:30] artists that a user follows or some [43:33] other complex queries like that. And [43:35] that is possible with SQL type of [43:37] databases. [43:38] And once the database grows, we'll need [43:41] to also add indexes for the important [43:44] properties here, which will be the [43:46] frequently accessed columns. And in our [43:49] case, we can, for example, index the [43:51] title in the song table. We can index [43:54] the name in the artists table. And we [43:58] can also index the song ids in this join [44:01] table where we'll be joining the [44:02] playlists and songs. These indexes will [44:06] enable us to have fast search [44:08] capabilities which will also handle the [44:10] typos and partial matches that user [44:13] expects when they are searching for a [44:15] song or for a playlist. And the playlist [44:18] item index will speed up the queries [44:20] when fetching songs for a specific [44:22] playlist or finding which playlists [44:24] contain a specific song. That's it for [44:27] the planning of general object storage [44:30] and also the SQL database. Now let's see [44:33] how we can scale this. Let's say the [44:35] user base grows to 50 million users now [44:38] and the songs grow to 200 million songs. [44:41] This means our current architecture [44:43] cannot handle it more. Which means if we [44:46] calculate the requirements again we will [44:48] have 100 bytes for 200 million songs. [44:51] For song metadata that will be 20 GB. [44:54] For user metadata it will be 1 kilob [44:57] times the user base which will be 50 GB. [45:00] But for audio storage, you can see the [45:02] estimated 3 megabytes per audio file. [45:05] And if you multiply that by the amount [45:08] of song, which is now 200 million, let's [45:10] say, it will be equal to something like [45:13] 600 terabytes. And this is not even [45:15] including the replicas, the regional [45:18] copies or the multifile uploads by [45:20] artists. So in reality, this will be [45:23] like two or three times more. So let's [45:26] see how our scale phase might look like. [45:28] As a first step, we can add CDN edges [45:31] for different regions of the world. For [45:34] example, we can have one for North [45:36] America, we can have in Europe, we can [45:38] have in South America. This will speed [45:40] up the process of retrieving it from [45:42] CDN. And similar to that, we can have [45:45] load balancers in different regions. [45:47] Let's say we have at least two load [45:49] balancers in two different regions that [45:51] are still mapping the incoming requests [45:54] to the API servers that we have. Before [45:57] getting to API servers for the CDNs, we [46:00] need to use some sort of eviction policy [46:02] for cache management. And here it's a [46:05] good use case to use the LRU cache [46:08] method which is automatically removing [46:10] the list accessed content when storage [46:13] fills up and the cache would fetch from [46:15] the origin storage on a cashm for the [46:19] API servers. Since we are not [46:21] maintaining any type of user session or [46:24] any storage, these are completely [46:26] stateless meaning we can scale them as [46:28] many as needed to handle the incoming [46:31] traffic. So adding more capacity here is [46:34] straightforward. We are just replicating [46:36] the existing API servers. The main thing [46:39] here is that when designing the API [46:41] server, we need to design for truly [46:44] stateless servers so that it's possible [46:46] for us to scale this easily. Next, we [46:48] need to also scale the database because [46:50] the single instance cannot handle that [46:53] many metadata storage. Which means we [46:55] need to separate the reads and the [46:58] rights in the databases. For the rights, [47:01] we can use a leader replica where we [47:03] will be writing the new files of the [47:06] songs or new playlists that users are [47:09] creating. But when reading it, we can [47:11] have some slave or follower databases [47:15] which will be only used for reading. So [47:17] this will be in sync with the leader [47:20] database. But whenever we get any read [47:22] operations like getting the song [47:24] metadata, getting the playlist metadata, [47:27] we will be reading them from the [47:29] follower databases. As a next step, if [47:32] this is not enough, we can also consider [47:35] sharding the database. Sharding will [47:37] split the data across many database [47:39] instances. Let's say we will have one [47:41] shard for the US users and one shard for [47:45] the European users. We can split it by [47:48] geographically. And then we will [47:49] partition songs by the artist ID ranges. [47:53] This will give us horizontal scalability [47:55] and it will improve the performance by [47:57] keeping the related data together. And [48:00] also for the blob storage, we can [48:02] replicate this to be in different [48:04] regions. Like for Europe users, they [48:06] will be uploading to a blob storage that [48:09] is in Europe region. For North America, [48:12] they will be uploading songs to the blob [48:14] storage that is in their own region and [48:17] so on. For the other geographics in the [48:20] world and all of them will still be [48:22] using signed URLs for accessing the [48:24] audio files and streaming the music [48:26] chunks. Of course, there are many things [48:29] that can be improved on this system like [48:31] improving the reliability or monitoring. [48:34] But the key, as you can see, is starting [48:36] simple. We don't want to overengineer [48:38] for small amount of users and then [48:40] gradually increasing the scale as we go. [48:43] And lastly, let's see how we can also be [48:45] able to design such systems or features [48:48] from scratch and what's the real [48:50] difference that separates senior [48:51] engineers from mid-level developers. If [48:54] you think the designing part as this [48:57] pyramid on the bottom which is the main [49:00] like most important part is the planning [49:02] which is the blueprinting phase the [49:05] technical design understanding [49:06] requirements as you saw estimating [49:08] capacity choosing the architecture. This [49:11] is where everyone needs to start when [49:13] designing such large scale systems and [49:16] only then we move on to the text [49:18] selection which is like picking the [49:21] right database whether it's SQL or NoSQL [49:23] and if it's SQL then what exact type of [49:26] SQL database it will be the caching [49:29] layers eviction policies storage [49:31] solutions and all of that tech related [49:34] stock selection and only last stage [49:37] which is the most smallest part actually [49:40] at is designing is the coding which is [49:43] like starting to implement the things [49:46] but that is actually the thing that [49:47] requires the least brain power and [49:50] especially if coding is the only thing [49:52] that you can do then we have now these [49:55] cloud or GBT models which can do it [49:57] faster better and cheaper so what's the [50:00] point of any company to pay you if the [50:04] only thing that you can do is coding [50:06] that is where the most mid-level [50:07] developers are stuck because they have [50:10] this completely upside down. They start [50:12] developing things without even having [50:15] any plan, without having any text [50:18] selection. They just start coding and [50:19] see where it takes them. Whereas the [50:22] fundamental stickers who learn to think [50:24] through problems before even touching a [50:26] keyboard or before even starting to [50:29] build any code, these are the ones that [50:32] will never struggle to find roles [50:34] because all the companies are always in [50:36] need for such developers who can think [50:39] through problems from users perspective [50:41] and also bring rough ideas to reality by [50:46] understanding the requirements. By the [50:48] way, if you'd like to learn all these [50:50] concepts that I'm using for designing [50:52] these systems from scratch, I have a [50:54] whole system design guide covering all [50:57] these concepts that you need for system [50:59] design interviews on my YouTube channel. [51:01] You can check out the first link in the [51:03] description to find the link to my [51:05] channel and the first pinned video in my [51:08] channel is that system design guide [51:10] explaining all these concepts you need [51:12] for passing these system design [51:14] interviews. Last design a hotel [51:16] reservation system similar to [51:17] booking.com or Airbnb which is a typical [51:21] system design interview problem. The [51:23] mistake that most developers make here [51:25] is when they ask to design such a system [51:28] they treat it like a typical crowd [51:30] application with some caching and [51:32] replication for the availability. But [51:35] that's not the correct way to think [51:36] about it. For instance, one of the [51:39] problems that you might have is a moment [51:41] when two users are trying to book the [51:43] last room available at the same time and [51:45] you try your simple select then insert [51:48] approach which creates the double [51:50] booking problem. So you end up charging [51:52] two credit cards for one room and now [51:55] you're dealing with refunds with angry [51:57] customers and the system that cannot be [52:00] trusted for the next time. That's why [52:02] hotel reservation systems are hard [52:04] because they also test the exact skills [52:06] that separate mid-level developers from [52:08] seniors. Things like handling [52:10] concurrency without breaking it under [52:12] heavy load, keeping the data consistent [52:15] across multiple systems, and also [52:17] designing for failure in case something [52:19] goes wrong in parts of the system. And [52:22] it's not about memorizing these [52:23] patterns, but rather understanding the [52:25] actual problems that might occur. What [52:28] are the solutions to those? and what are [52:30] the trade-offs between those solutions. [52:32] So, we are going to design this system [52:34] in four simple steps. First, we will [52:36] gather the design requirements, estimate [52:38] the storage and queries per second, [52:41] transactions per second, and anything [52:42] else that we need for designing the [52:44] system. Then, we'll get into the basic [52:46] highle design including the API design, [52:49] database schema, reservation flows, and [52:51] so on. And in the first step, we will [52:53] scale the system to handle 1,00 queries [52:56] per second and 17 transactions per [52:58] second, which are the numbers that are [53:00] close to the realbook.com scale. And [53:03] then we will handle the failures under [53:05] heavy load and discuss the system [53:08] recovery whenever somethings or some [53:10] parts of the system fail. So if you're [53:12] tired of system design explanations that [53:15] skip the hard part and only give you the [53:17] surface level, then this one is for you. [53:19] Let's start with the first step which is [53:21] the design requirements. In such [53:23] challenges, we usually follow a [53:25] framework to come up with the solution. [53:27] And the very first step is always to ask [53:29] clarifying questions to lock in the [53:32] scope that we are going to design and [53:34] then also about functional and [53:36] non-functional requirements. So let's [53:38] start with the clarifying questions that [53:40] we might have. For example, one of the [53:42] questions can be about the payments [53:43] which is are we going to allow them to [53:46] pay at the platform or it is going to be [53:48] a payment in checking and this will let [53:51] us know whether we support payments in [53:53] the platform or not. For this one let's [53:55] say yes. So we are going to allow them [53:57] to pay at the platform which means we [53:59] need to have some sort of payment system [54:01] inside. Another question can be about [54:03] booking channels. So is this a website [54:06] only web app where they are going to [54:08] book the hotels or we also support the [54:10] mobile app which might have different [54:12] rates and different pricings. So let's [54:14] say we are going to design both like web [54:16] version and mobile version of this [54:18] application. Then about cancellations [54:21] and over booking. So do we support [54:23] cancellations of the reservations and [54:26] also over booking because hotels [54:28] typically allow some sort of over [54:31] booking taking into account that there [54:33] might be some cancellations. So if they [54:35] are getting let's say 10 bookings per [54:38] month then out of these like two or [54:41] three are going to be cancelled. So they [54:43] allow users to also overbook because of [54:46] the cancellations. Once they get some [54:47] cancellations the room is free now. So [54:50] they will allow over booking as well. [54:52] Then about the dynamic pricing. So are [54:54] the prices fixed or the price changes [54:57] based on the demand and the dates. And [55:00] for this let's say we are going to have [55:01] dynamic pricing. Whenever the demand is [55:04] higher so it's let's say Christmas time [55:07] which is around now. We are going to [55:10] increase the prices. For example this [55:12] room where I'm recording from it was [55:14] triple of the price as in usual because [55:17] that is the Christmas time. So they are [55:19] going to increase the numbers and [55:21] similar to that any other popular dates [55:24] that people are reserving we are going [55:26] to increase the prices based on the [55:28] demand that we are getting. Then we can [55:30] get to the functional requirements. So [55:33] one question might be about the hotel [55:35] management. Are we going to support the [55:38] admin panel like adding, updating, [55:40] removing hotels or we are only focused [55:42] on the user part. Admin panels might be [55:46] included in this but the core [55:47] functionality evolves usually around the [55:50] users which are booking the hotels [55:52] cancelelling them. Let's say we are [55:54] going to also support some sort of admin [55:56] panel in this system. Then we get to the [55:59] search and discovery questions like what [56:02] are the core functionalities we are [56:03] going to support. And a way you can [56:06] think about this is what are the [56:08] functionalities that if we remove from [56:10] booking.com [56:12] it doesn't become Booking.com anymore. [56:15] This might be a good approach for [56:17] identifying what are the core [56:18] functionalities. For example, if users [56:21] are not able to search hotels by [56:23] location, then it becomes some sort of [56:26] useless application. Right? So of course [56:28] one of the core functionalities we will [56:30] support is that users need to be able to [56:32] search by location, by dates, by guest [56:35] number, price range and some other [56:38] filters that we might set. Then about [56:40] the reservation flow, how usually it [56:43] happens is users book the rooms. Then [56:46] they view the reservations, they should [56:48] be able to cancel the bookings because [56:50] we also clarified it above. And also the [56:53] system should prevent double bookings. [56:55] This is one of the challenges we need to [56:57] identify here so that we also solve it [56:59] in the next steps. Then about the [57:02] payment processing since we clarified [57:04] that we are going to support payments in [57:06] the app then we need to integrate some [57:08] sort of payment processing like stripe, [57:11] PayPal or something else. And optionally [57:14] it might also include notifications. So [57:16] sending confirmations for bookings or [57:19] cancellations or payments to both user [57:22] and the admins brokers who are uploading [57:24] these hotels. Then we get into the [57:26] non-functional requirements which are [57:28] more about the consistency, load and [57:31] latency that we are going to support. [57:34] Let's start with the consistency. We are [57:36] going to have strong consistency because [57:39] two users shouldn't be able to book the [57:42] same room for the same dates. That is [57:44] non-negotiable. [57:46] But whenever they are searching or [57:48] discovering the rooms there, the [57:50] eventual consistency is fine and you [57:53] will see why also when we get to the [57:55] designing phase. Then about the [57:57] concurrency. So we need to handle [57:59] thousands of users trying to book the [58:02] same hotel simultaneously and without it [58:04] breaking the system. Also about the [58:07] latency, we need to provide low latency. [58:09] So search results should appear under [58:12] 500 milliseconds and booking [58:14] confirmations are fine if they are [58:17] handled between two or 3 seconds. You [58:20] should also clarify the availability and [58:23] scale. Let's say for availability we are [58:25] going to support the free 9 uptime. And [58:28] when it comes to scale we need to be [58:30] able to scale this horizontally without [58:32] redesigning the system from scratch in [58:34] the future. Once we gather the design [58:37] requirements, we can move on to the [58:39] estimation of data and capacity and [58:42] after that we can start designing the [58:44] system. Now when it comes to the data [58:46] estimation, the most important things we [58:49] are interested here is the queries per [58:51] second that are going to happen in the [58:53] system and also transactions per second [58:55] which are the rights to the database [58:57] when users are booking hotels. So in [59:00] this kind of system you can think about [59:03] it as this funnel here. The users that [59:06] come uh to the platform are at the top [59:08] here. So these are the users who are [59:10] viewing the hotel room. This is the 100% [59:13] of the traffic somewhere here. And then [59:16] next we have out of these users probably [59:19] like only 10% will go into browsing the [59:22] hotels and searching for some good [59:26] matches and booking the hotel. So that [59:29] will be the second layer here. And then [59:31] third we will have the reservations. So [59:34] only out of this only the 10% are [59:36] actually going to book anything because [59:38] most of the visitors are tire kickers. [59:41] They are not looking to book anything. [59:43] They are just browsing. But we still [59:45] need to design the systems for them [59:48] because they are also going to increase [59:49] the queries per second that are [59:51] happening in the system. So based on [59:54] this there are two ways for us to [59:56] calculate the load. One approach is [59:58] let's say to take a look at the [60:00] booking.com and you can use some sort of [60:03] extension like similar web to see the [60:05] traffic that it's getting per month. For [60:07] example this month it got 500 million [60:10] visitors in the website. That means that [60:14] 500 million users were browsing the [60:18] system this month. So at the top and [60:20] then you can start calculating it down. [60:22] based on this to see also the [60:24] transactions per second and what are the [60:26] queries per second based on this load [60:28] and the other way is the bottom up [60:30] approach which is to get the number of [60:33] reservations per year let's say and for [60:36] booking.com that is 1.1 billion [60:39] reservations per year and then we can [60:42] calculate this upwards so starting from [60:44] here how many visits we get per month or [60:48] per second to the platform and we will [60:51] also be able to calculate the [60:52] transactions per second because we have [60:54] the number for the year reservation. So [60:57] each of these numbers is fine. You just [60:59] need to get one of them from the [61:01] interviewer and then you can start the [61:03] calculations and also take into account [61:05] that these numbers are also rough [61:08] estimates. So you might get these [61:09] numbers exactly to calculate or you [61:11] might get just the visits per year and [61:14] the bookings per year and you should be [61:16] able to calculate it after that. The [61:19] other important metric that we need is [61:20] the average state. So what is the [61:23] average stay that users are booking the [61:25] hotels? Let's just say two or three [61:28] nights per booking. So let's go with the [61:30] two nights per booking and then we will [61:32] be able to calculate the rest. So let's [61:35] start with the daily reservations. We [61:37] have the number of bookings per year [61:39] which is 1.1 billion rooms per night. [61:42] Let's just start with this approach. [61:44] Then we can divide this by the nights [61:46] per booking and that comes down to 550 [61:50] million bookings per year because this [61:53] is the room nights in total. This is not [61:55] the bookings. So we need to divide this [61:58] by the nights per booking to get the [62:00] actual number that we are getting booked [62:02] per year. Then we can divide this by 365 [62:06] which is the days in the year and we'll [62:08] get the number of bookings per day which [62:11] is 1.5 million. And after that we can [62:14] divide the 1.5 million by the seconds [62:17] per day which is 86,400. [62:21] And that means that we are getting 17 [62:23] bookings per second which is the [62:26] transactions per second that we are [62:27] going to support in the database. Then [62:30] when it comes to the traffic funnel part [62:32] and the reads as you saw with similar [62:35] web we are getting something like 500 [62:37] million visits per month in the [62:39] booking.com with about eight pages per [62:42] visit which means that on each visit the [62:45] users are going to browse around eight [62:47] pages before they leave the website. [62:49] Based on this, we can first get the [62:51] visits per day and we can get that by [62:53] dividing the monthly traffic with 40 [62:56] which are the days in the month and [62:58] we'll get to 17 million visits per day. [63:01] After that, we can get the visits per [63:03] second. So, if we divide that by the [63:05] seconds in the day, we will get uh 200 [63:08] visits per second to our website. And if [63:11] we assume that they are going to browse [63:13] around eight pages per visit before they [63:15] leave, let's say that it comes down to [63:18] five backend reads per visit, which can [63:20] be things like search queries, [63:22] availability checks. So we need to just [63:25] increase this number by multiplying it [63:28] with the five and we'll get around 1,000 [63:31] read requests per second. And based on [63:34] these calculations, we can clearly see [63:36] that this is going to be a strong read [63:38] heavy system because we have 1,000 QPS [63:41] queries per second that we need to [63:43] support and only 17 TPS which are the [63:47] transactions per second that we will [63:48] get. So our challenge here will be to [63:51] optimize the architecture for reads [63:53] while still keeping the rights [63:55] consistent to avoid the double booking [63:57] problem. These are the core numbers we [64:00] need to start designing the system and [64:02] then the rest we can look at while [64:04] designing the API database and if there [64:06] are any more things that we need it's [64:08] fine to calculate those there but at [64:10] least we need to get the main things [64:12] that we need before starting with the [64:15] highle design. After the scale and data [64:18] size calculations we can now design an [64:20] architecture that fits these numbers. So [64:23] we can start by using a serviceoriented [64:26] architecture with clear boundaries but [64:29] also keep the tightly coupled data in [64:31] the same database. So as a starting [64:34] point let's say we have the mobile app [64:36] or the web app where users are browsing [64:39] these hotels and making the [64:41] reservations. We'll use CDN in front [64:44] here for the static assets like [64:47] rendering the images, media files, also [64:49] the JavaScript content and the hotel [64:52] images. Then we have an API gateway [64:55] which will be the single entry point to [64:58] our application. This is where we will [65:00] handle the authentication, the rate [65:02] limiting and also routing the requests [65:05] between all the services that we have. [65:08] It can also serve as a load balancer [65:10] across service instances. So if you have [65:13] multiple instances of hotel service, [65:15] let's say, this API gateway can serve as [65:19] a load balancer and route the traffic [65:21] between the instance one and instance [65:24] two. When it comes to services, we can [65:27] separate this with boundaries for hotel [65:30] service. This is where we will have [65:31] hotel and room information, mostly [65:34] static and heavy cachable service. An [65:37] important service here is a reservation [65:40] service. This will handle the booking [65:42] logic, inventory checks, reservation [65:44] creation and cancellation. And this is [65:47] also where concurrency will be handled [65:50] in the application. Next, we have a rate [65:53] service. This will be for room pricing [65:56] by date because depending on the date [65:59] and when you book the hotel, the price [66:01] will be different. So this will be [66:03] applying dynamic pricing based on the [66:05] demand of the rooms. To integrate the [66:08] payment gateways, we will also need a [66:10] payment service that will implement [66:12] let's say Stripe payments, PayPal [66:15] payments, Apple Pay, Google Pay and Adam [66:18] providers. And we also have a [66:20] notification service that talks with the [66:22] reservation service and with the payment [66:25] service. This is what sends the emails [66:27] and push notifications for the data [66:30] layer. Initially we can start by using [66:33] some sort of SQL database like possql [66:36] primarily because we will need the acid [66:39] guarantees and this will be also useful [66:42] for the transaction store and we will be [66:45] using many transactions. So we need the [66:48] consistency of SQL databases. That's why [66:50] we go with SQL in this case. We also [66:53] need some sort of caching like radius [66:56] for the data that doesn't change that [66:58] often. But we can also improve this a [67:01] bit further. So first of all if we think [67:04] about the scale of the reads that we [67:07] will have which is around thousand [67:09] queries per second postsql database [67:12] won't be able to handle that load. So we [67:15] need to give some sort of solution to [67:17] this. For such situations, tools like [67:20] elastic search are a good solution for [67:23] handling the massive scale for the [67:25] search. So we will introduce a search [67:28] service that will be responsible for [67:31] retrieving the hotels by the filters [67:33] that are set on the front end. And [67:36] elastic search will handle the full [67:38] textbased search on hotel names and [67:41] descriptions. Also, it will be able to [67:44] filter hotels, let's say within 5 [67:47] kilometers of location or within some [67:49] range to that location that is set and [67:52] it will be able to filter by the price, [67:55] range, ratings and any other filters [67:57] that we need. The thing with elastic [68:00] search is that it's way faster than the [68:02] postgrql database that we will use let's [68:05] say for listing the hotels and also it [68:08] is eventually consistent meaning that [68:11] eventually it gets updated with the [68:13] database and gets the latest data in the [68:16] elastic search and indexes everything [68:19] properly to be able to search quickly [68:22] based on locations or based on the price [68:24] range and any other filters. We can also [68:27] decouple the services a bit. Another [68:30] approach is to use different databases [68:33] for each microservices. Like for the [68:35] payment service, we can have payment [68:36] database. For the hotel service, we can [68:39] have hotel database. And we'll discuss [68:42] what's the pros and cons of this. But to [68:45] also decouple them, we can introduce a [68:48] message cube that will be responsible [68:51] for instance for payments. Whenever a [68:53] payment is processed in a payment [68:55] service, we will process that and [68:58] publish a message in a Kafka rabbit MQ [69:01] or some other messaging queue. And this [69:04] will first of all send email push [69:06] notification to the user or to the owner [69:10] as well as an email or as an inop [69:13] notification that the payment was [69:15] successful and the hotel is reserved. [69:18] And it will also update the availability [69:20] meaning the status of the hotel. So it [69:23] will push that to the search service [69:26] which will update it in the elastic [69:28] search and that is again getting [69:30] eventually consistent. So right after [69:33] the payment maybe if users search they [69:36] will still see that hotel available but [69:39] they won't be able to book it because we [69:41] are using the hotel database to double [69:44] check whether it's really available or [69:46] not. And eventually after some time this [69:49] will be persisted with the database and [69:51] it will have the same data as the [69:53] postgrql database. Next when it comes to [69:56] choosing whether all these services will [69:58] be like microservices or a single [70:01] monolith or they can also be pragmatic [70:04] microservices. For example, in our case, [70:07] a single monolith would simplify the [70:10] transactions, but it will become hard to [70:12] scale and deploy independently because [70:14] if many developers, let's say hundreds [70:17] of developers are working on this single [70:19] monolith, then it becomes very hard to [70:22] deploy independently and work on things [70:26] without stepping on each other. Another [70:28] approach is full microservices like this [70:31] where you have rate service, [70:32] notification service and so on with [70:35] separate databases which will introduce [70:38] distributed transactions and consistency [70:40] problems that don't exist at the [70:43] monolith level. That is because each [70:46] service as you can see like hotel [70:47] service owns its own database, [70:49] reservation service owns its own [70:52] database. So another approach is to keep [70:55] the tightly coupled data together in one [70:58] database and share it between the [71:01] services. For example, in our case, the [71:04] reservations and the hotel databases can [71:08] be kept together. This will let us to [71:10] use the local ACIT transactions instead [71:12] of the distributed transactions. And we [71:15] will have a service level separation at [71:18] the application layer but we will still [71:20] have the shared data for tightly coupled [71:23] entities. After you agree on the highle [71:26] design the next step is the API and [71:29] database design and you can start with [71:32] API design then database or vice versa [71:35] whichever way it's easier for you to [71:37] talk about the flow. So in our case [71:40] let's start with the API design. We will [71:44] use restful APIs for all the operations [71:47] because that is what makes sense in this [71:49] kind of system. Although as you can see [71:52] here the front end directly makes calls [71:55] to hotel API to other APIs as well that [71:59] we have. Maybe sometimes it will make [72:01] calls to reservation API. Another [72:04] approach here a very common one is when [72:07] you have multiple microservices usually [72:09] you would use a GraphQL layer in front [72:12] of the front end. So front end will only [72:14] make requests to the graphql and it also [72:16] will know the schema based on the [72:18] graphql schema and then the graphql [72:21] layer will be responsible for making [72:22] calls to all these services that you [72:25] have behind the scenes. So it kind of [72:27] extracts away the complexity and the [72:29] routing between these services. But to [72:32] keep it simple on the diagram let's just [72:34] assume that front end makes the calls [72:36] directly. So we have here admins who are [72:39] listing the properties. We have users [72:42] who are browsing the hotels and making [72:45] reservations. And we have the front end [72:46] app which is either a mobile app or the [72:49] web version. The very first step for [72:52] admins is they need to be able to loged [72:54] in to be able to use the features for [72:56] the admins while users can be browsing [72:59] the hotels without being logged in. But [73:01] whenever it comes the point for the [73:03] reservation, they need to also create an [73:05] account or login. For listing the [73:07] available hotels, we will make a get [73:10] request to / hotels with any necessary [73:13] filters that the user sets and we will [73:16] get the available hotels and then if [73:19] they want to browse a specific hotel [73:21] then we will make a request same get [73:23] request to / hotel/sp specific ID of the [73:27] hotel which will return us the hotel [73:30] that has that specific ID. Now how it [73:33] will happen behind the scenes is hotel [73:35] API will fetch the hotel details from [73:37] the database. So for specific hotel we [73:41] can also read this from the postcarql [73:44] database. But in the initial call where [73:47] we are making a request to / hotels that [73:50] is where we will be using the elastic [73:52] search to quickly filter based on the [73:54] filters that are set on the front end. [73:57] And at this step for a single hotel we [73:59] just basically forward the hotel details [74:01] to the front end and then front end is [74:04] responsible for parsing and showing the [74:06] details in the page. For admins who want [74:09] to add a new hotel, they will need to [74:12] make a post request to / hotels [74:15] endpoint. And this is where we will [74:17] insert it into the SQL first and then [74:21] once we get a successful insert, we will [74:23] just forward the 2011 status that the [74:27] hotel is created and later this will be [74:30] eventually persisted with the elastic [74:32] search database. But we will show the [74:34] confirmation at this step to the admin [74:36] that the hotel has been created. And [74:39] similarly, if they want to update the [74:41] hotel information and they make some [74:44] edits, we can make a put or patch [74:46] request to hotels/ ID of that hotel and [74:51] again update it in the database, get the [74:54] update result and send the confirmation [74:57] and show it for the admin. A similar [75:00] flow for removing hotels. Again a delete [75:03] request in this case to / hotel/ ID of [75:06] that hotel. We remove it from the [75:08] database and we show it to the user and [75:12] later this will be removed from the [75:14] elastic search. And within hotels we [75:17] will also have rooms because one hotel [75:19] has many available rooms. We will treat [75:22] them as a sub resource. So under hotels [75:26] and specific ID of the hotel we will [75:28] have / rooms first of all which will [75:31] give you all the rooms that are [75:33] available under this hotel and to fetch [75:36] the specific hotel you can make a [75:38] request to / id and get more details [75:41] from that hotel which will again fetch [75:43] from the database and then you will [75:45] forward it to the client and basically [75:48] here we are just repeating whatever we [75:50] have for the hotels but for the rooms. [75:52] So if you want to add a room you will [75:55] make a request to post/ [75:57] id of the hotel/ the rooms. This is [76:00] again following the general restful [76:03] principles. We will insert the room in [76:05] the database and provide with the [76:07] successful response. Of course you need [76:09] to also handle the failures whenever [76:12] something goes wrong showing the error [76:15] on why something went wrong in the API. [76:18] But when talking about the happy paths [76:20] or at least the available endpoints, [76:23] these are what we will have. We will [76:24] also have a put endpoint which is for [76:27] editing the room type again / rooms. In [76:30] this case, we need to provide an ID [76:32] because we are not creating a new one [76:34] but we are editing one. So we need to [76:36] specify which ID is that room that we [76:39] are editing. And same goes for deleting [76:42] the same URL as we have for the put. But [76:45] in this case the method will be delete [76:46] which will remove the listing from the [76:49] hotel. [76:51] Then for the reservations this is where [76:53] we will make requests to the reservation [76:56] service. Let's say a user wants to see [76:59] their reservation history. We need to [77:01] provide an endpoint here to fetch the [77:04] reservations. So a get request to [77:07] /reservations which will read it from [77:09] the SQL database and then forward the [77:13] response to the client. Then they want [77:16] to see let's say a reservation details [77:18] and the invoice that they paid. So that [77:21] is where we will have the get endpoint [77:23] to /reservations/ [77:26] ID of a single reservation which will [77:28] fetch the details from the database. we [77:30] will get the details back and we will [77:33] forward it to the front end and we will [77:35] parse and show it to the client. Next, [77:38] when creating new reservations, they [77:40] will fill the reservation form in the [77:43] front end. Then we will send this form [77:46] details in a post request to [77:49] reservations. One important thing here [77:51] is that at this step we can already [77:54] prevent double booking for single user. [77:56] There are two scenarios like one user [77:59] can try to doubleclick or click the same [78:03] reservation room twice. So that's one [78:06] scenario which we can handle here. The [78:08] other one is that two users can try to [78:10] book the same room twice. We will handle [78:14] that in the later step. But at this step [78:17] for example one solution is we can [78:20] include a key like a reservation ID and [78:24] this will be generated on the front end [78:26] which will be used to prevent double [78:28] bookings when user clicks submit [78:31] multiple times. So if we have the hotel [78:34] ID and the room ID that this user is [78:36] trying to book, basically we generate a [78:39] reservation ID once and then if they try [78:42] to double book it like book it again, we [78:45] will be able to catch it in the database [78:48] and see that the reservation ID is the [78:51] same. So we will block them from making [78:53] a request or we can even do it on the [78:55] front end side. Another solution is you [78:58] can also do it on the back end. For [79:00] example, when they make a request to [79:02] book something, you can combine the [79:05] hotel ID in combination with room ID, [79:09] the user ID who tries to book this and [79:13] also the date or some sort of timestamp. [79:17] Why? Because in the future this user [79:19] might try to book it again. And if you [79:22] just use these in combination, then [79:25] basically you are blocking them to make [79:27] a future reservation on the same hotel. [79:30] So you can also use these in combination [79:33] to generate some sort of hash and [79:35] whenever the hash match is from the [79:38] database then you block them from making [79:40] the reservation. But this is way simpler [79:43] solution on the front end side to just [79:45] generate some sort of UU ID uh for the [79:48] reservation ID. If you are talking only [79:50] about a single user and preventing them [79:53] from booking again, then this is a very [79:56] lightweight solution and way simpler [79:59] than inserting this hash into the [80:01] database and checking it every time in [80:03] the back end. So here we will first [80:06] check for existing reservation ID. If [80:09] it's not found, then we will insert the [80:11] new reservation. So assuming it's not a [80:14] double booking and we get the [80:15] reservation confirmation, we will just [80:17] show them the confirmation and behind [80:19] the scenes the Kafka message will be [80:22] published and it will send the [80:24] notifications to the user and to the [80:26] owner of the hotel. But if it's a [80:29] duplicate reservation attempt, then they [80:32] will make a request and this will [80:34] contain that reservation ID. So we will [80:37] check for the existing reservation ID [80:39] and if a transaction like that already [80:41] happened and we have the reservation and [80:44] it already exists then we will return [80:46] the existing reservation and show it to [80:49] the user that this is already reserved. [80:52] Similarly they might want to either [80:54] cancel the reservation or change the [80:57] dates of the reservation. So we need to [81:00] provide endpoints for that as well. In [81:02] case of cancelling, it will be a delete [81:04] request to reservations/ ID of that [81:07] reservation. And in the database, we [81:10] just need to check whether it's allowed [81:12] to cancel because if they try to cancel [81:15] a week before the appointment, then it's [81:17] allowed or if they try to cancel it last [81:20] minute, then it's probably not allowed. [81:23] So, in case it is allowed, we will just [81:25] send the confirmation. Otherwise, we [81:27] will send a response that we cannot [81:29] delete the reservation, meaning we [81:31] cannot cancel it and show the options to [81:33] the user. Similar to this, you might [81:36] also have edit reservation with put [81:38] request which will update the dates. For [81:41] example, if plans have changed and you [81:43] want to update the dates, then we will [81:45] make a put request to that ID [81:47] reservation. Let's also see how the [81:50] search will happen. So another service [81:53] we will have for this is the search [81:55] service which will be connected with the [81:57] elastic search. So let's say user sets [82:00] the filters like the location where they [82:02] want this hotel to be located at the [82:05] dates start and end date and so on and [82:08] we make a get request to / search. These [82:12] will include the location, the check-in, [82:14] checkout time, how many guests you have, [82:16] how many beds you want, the price range, [82:18] and any other necessary filters. This is [82:22] where we will query the available hotels [82:24] and room types. But an important thing [82:27] is we will query this from the elastic [82:30] search. In this case, we won't query it [82:32] from the database because it's way [82:34] faster and that's the point of us [82:35] setting up elastic search so that we can [82:38] quickly return the search results. And [82:41] this will return the available hotels [82:44] within the range of that location and [82:47] within that price range and the check-in [82:50] checkout time and front end will be [82:53] parsing and showing it to the users. [82:56] Next, let's move on to the data models [82:59] and the database design. One important [83:03] thing here is that you book the room [83:05] type and not the specific room. So you [83:08] book a specific type of room like is it [83:11] a double queen bed or something else and [83:14] not a specific room because the room [83:16] numbers are assigned at the check-in and [83:19] not here. But let's start from the [83:21] hotels because it will make more sense [83:24] this way. So we will have of course the [83:27] ID of the hotel which is the indicator [83:30] of the unique ID for this hotel. It can [83:33] have a name, address, location, [83:35] description and so on. Now there might [83:38] be also hotels that are under the same [83:41] brand name. In this case you can either [83:44] have the same hotel name but different [83:47] hotel ids. So it will be a different [83:51] hotel LLC in US. Let's say it will be a [83:53] different hotel LLC in Europe. The name [83:56] will be the same but the ID will be [83:58] different. Or you can use a helper table [84:01] like hotel groups and then group them [84:04] under a brand name if there are brands [84:08] that have multiple hotels at different [84:11] locations. Then the important part is [84:14] the room type that we need to have. So [84:16] the ID of the room type, its name, [84:19] description and then how many people it [84:22] can serve, the ID of the hotel which [84:25] connects back to the hotel. So one hotel [84:28] can have multiple room types and this is [84:31] a one to many relation. Then each room [84:35] type is also connected with room type [84:38] rate because each of these rooms will [84:40] have a different rate. Let's say we have [84:42] a room which is the basic package. And [84:45] as a basic example let's say it costs [84:48] 100 bucks per day. So this is the basic [84:51] package. Then you can have a room with a [84:54] better view. In this case it will be [84:57] something like 150 per day. You can have [85:01] a hotel room with breakfast included and [85:05] that will be something like 200 and so [85:07] on. So that is the reason why we have [85:09] this room type rates because for [85:12] different room type we will have a [85:15] different rate that will be multiplied [85:17] by the base uh number that we have here. [85:20] And this is again one to many relation [85:23] because one room type can have multiple [85:26] room type rates. Then this is where we [85:29] have the actual rooms. So what is the [85:32] room ID? This is the exact room that we [85:34] are going to assign at the check-in. [85:37] Then it will have some more metadata [85:40] like floor number whether it's [85:42] available. This is the important part [85:43] that will be tracking the status. And [85:46] this is also what we are reading when [85:48] checking the availability [85:51] and the room type ID which again refers [85:53] back to the room type. So one room type [85:55] can have multiple rooms in it which [85:58] means you can have a hotel under that [86:01] hotel you have multiple room types like [86:03] you have let's say the basic type and in [86:06] the basic type you might have three [86:08] rooms available that are all basic [86:11] rooms. This also connects to the hotel [86:14] ID so that you can determine which exact [86:17] hotel this belongs to. You could also [86:20] join it with the room types and then get [86:22] the hotel ID like that. But it will be [86:24] helpful for us to have the hotel ID [86:27] here. So this will be like which hotel [86:30] does it belong to the room that we are [86:32] assigning at the check-in. Next we will [86:34] have the room type inventory. Inventory [86:37] is tracked per hotel, per room type and [86:41] day. So not the physical room. This will [86:44] enable us to have bulk availability [86:47] updates also scalable pricing and search [86:50] and it will be simple over booking logic [86:53] because we will also support over [86:55] booking considering the cancellations [86:57] that we will have in the rooms. So here [87:00] we will track the room ID type, the [87:03] hotel ID and this is again one to many [87:06] relationships between the room type and [87:08] between the hotel and some more things [87:11] like the date, the total inventory, [87:13] total reserved and so on. Another [87:16] important table here is the reservations [87:19] because once user selects the hotel room [87:22] specific room we will need to reserve [87:25] it. So we will track the reservation ID [87:27] which we will link later to the [87:29] transactions. We will have the room type [87:32] ID. This refers to the room types table. [87:35] We will have the exact hotel ID where [87:37] they booked this which refers to the [87:40] hotels table. The guest ID which is the [87:43] user who booked it. So for that we will [87:45] have a guests table. And we can [87:47] additionally have let's say admins or [87:50] brokers table which will be the users [87:53] who are publishing these rooms. So here [87:56] the most important thing for us is the [87:58] guest ID the first name last name email [88:01] and so on. So we will connect it with [88:04] one to many relationship here because [88:07] one user one guest can reserve multiple [88:11] hotels. So that's why it is one to many. [88:14] Then some more metadata like start date, [88:17] end date, status, whether it's reserved, [88:19] whether it's canceled and so on. And [88:22] another important piece here is that we [88:24] will separate transactions into a [88:26] separate table. The reason here is that [88:29] let's say you reserve the hotel first. [88:32] So first the status is reserved. [88:35] Then the user can either change their [88:38] mind and here they can either cancel the [88:41] reservation or they might request a [88:44] change in the date. So they might [88:46] request to change the start and end [88:48] date. In the second case, there will be [88:51] a price difference that they will either [88:53] have to pay or we have to refund them. [88:56] So that will be another transaction, [88:58] right? And that's why one reservation [89:01] can have multiple transactions. So the [89:03] first transaction is when they reserve [89:06] and then the second one is when they [89:08] decide to change the dates let's say and [89:11] update the reservation they make another [89:13] payment here which is a second [89:15] transaction or if they decide to cancel [89:18] that is another transaction where the [89:20] amount in this case is minus so we are [89:23] refunding the payment but that is still [89:25] a separate transaction and that's why we [89:27] will have separate transactions table [89:29] where we will track the ID of [89:31] transaction which reservation Does this [89:33] belong to the type of the transaction [89:36] like refund, change or the new [89:39] transaction and the amount which can be [89:41] both positive and a negative number. [89:45] With this schema, the room type [89:47] inventory table has one row per hotel [89:51] room type and date combination. We will [89:54] prepopulate this for the next two years [89:57] and then a daily chrome job can add new [90:00] dates as the time moves forward. So we [90:03] can also take a look at some example [90:05] data here. We'll have the hotel ID where [90:09] these room types belong to. Then the [90:12] room type ID. Let's say these three [90:14] rooms are from the same type like the [90:17] basic type. uh dates where we are [90:19] checking these for like December 15, 16, [90:23] 17 and so on and then we will see the [90:26] total inventory and also total reserve [90:29] which will give us an idea on how many [90:31] rooms we have available. When checking [90:34] the availability, if you're using the [90:36] SQL database, let's say for December 23 [90:39] to 27, we will select the date, total [90:43] inventory, and total reserved, which is [90:46] this column here. And then the total [90:48] inventory, which is the available. We [90:51] will select it from the room type [90:53] inventory where the hotel ID and room [90:56] type ID match. And we will set the date [90:58] to be between the range that we want to. [91:01] This will give us the total reserved and [91:03] total inventory. And by using this we [91:07] can check that if the total reserve plus [91:10] rooms to book is still less than equal [91:13] to the total inventory then we allow the [91:15] booking. Which means if let's say here [91:18] we have 100 and here we have 92. So we [91:21] still have eight rooms available. And if [91:23] user is trying to book just one room [91:25] that will be 92 + 1 and we are checking [91:29] this against 100. So the condition will [91:32] be true and we will allow the booking [91:35] and coming back to the initial [91:37] requirements since we allow let's say [91:40] 10% over booking or whatever that number [91:43] is we can include that over booking [91:46] factor here. Let's say if we allow 10% [91:48] over booking then we will allow 10% like [91:52] set the 10% here. Meaning if we have 100 [91:56] total inventory available, then we will [91:59] allow them to book up to 110 rooms [92:02] because we know that the other 10 will [92:05] probably be cancelled until the date of [92:07] the reservation. And lastly, let's also [92:10] discuss why we move with relational [92:12] database. In this case, we kind of [92:14] touched on this, but let's get a bit [92:17] deeper in why we choose this. So first [92:20] the most important thing we need here is [92:22] the acid guarantees and specifically the [92:25] consistency of SQL databases. This is [92:28] important for us for preventing double [92:30] pooking issues which we will handle [92:32] soon. From the calculations we can also [92:35] see that this is a very read heavy [92:37] system. So relational databases handle [92:40] these kind of reads well and we can [92:43] always combine it with elastic search [92:45] for scaling it. But with 1,000 reads [92:48] queries per second versus 17 writes per [92:52] second, this clearly shows that we need [92:54] to go with an SQL database here in [92:56] combination with the consistency factor [92:59] that we need from the ACIT. And then the [93:02] relationships are more clear this way [93:05] like between rooms, hotels, the room [93:08] types, reservations, transactions and so [93:11] on. And we'll need to make complex [93:13] queries to check the availability of the [93:16] rooms or make multiple joins for [93:19] inventory with pricing, filter by dates, [93:22] aggregate totals and so on. So SQL will [93:25] handle this well as well. We also need [93:27] to handle the concurrency. We already [93:30] discussed the first scenario where the [93:32] same user is trying to book the same [93:34] room twice and the solution here is just [93:37] to have that reservation key. So any [93:39] second request will get duplicate key [93:42] error and we will return the existing [93:44] reservation. So that's the simple one [93:47] that we can solve. But then the second [93:49] scenario is we have two different users [93:51] who are trying to book the same room [93:54] type which is only one spot left for [93:57] that hotel. And if you know about the [93:59] SQL isolation layers, by default this [94:02] will be read committed and concurrency [94:05] bugs still happen at read committed [94:07] isolation layer which means that you [94:10] only see data after other transactions [94:13] commit. But there is a gap between when [94:15] you read like check the availability and [94:18] when you write which is you book the [94:20] room. And in that gap you might have two [94:23] different transaction happening like [94:25] user A reads the total reserved. He sees [94:27] that one room is available because the [94:29] total is 100. And user B also reads and [94:33] sees that there are 99 rooms reserved [94:35] and 100 available which means that there [94:38] is one available. So for both of them [94:41] the check shows successful like the [94:43] condition is true which means that we [94:46] allow them to proceed with the booking. [94:49] So read committed by itself is unsafe [94:52] for bookings because two transactions [94:54] can both read valid data pass checks and [94:57] then override each other and that's [94:59] exactly how double bookings happen. As a [95:02] solution to this we have two main [95:04] approaches in this case. One approach [95:06] here is that we can use serializable [95:08] isolation layer which means that when [95:11] beginning the transaction at the code [95:12] level we can set the isolation layer to [95:15] be serializable. By default, it is read [95:18] committed. This is simple to implement [95:21] and it will work fine for low TPS [95:24] meaning transactions per second because [95:26] these won't be blocking each other if [95:28] you don't have that much high volume. So [95:30] it's fine to use serializable in that [95:33] case. The downsides to this is that [95:36] transactions will block each other. So [95:38] if you have high TPS then this won't [95:41] scale well and also you introduce [95:43] potential deadlocks if locking multiple [95:46] resources in different orders. However, [95:49] you can use let's say log timeouts and [95:52] set it to some number to prevent [95:54] indefinite weights and also keep the [95:57] transactions short. But still there is a [96:00] better solution in this case which is to [96:02] use read committed isolation layer which [96:05] is still the default mode in combination [96:07] with row locking. This is when you're [96:10] using select for update and it is used [96:13] to lock the inventory rows while we are [96:16] checking and booking them. So that other [96:19] users have to wait their turn. If you [96:21] have two transactions who try to log the [96:24] same row, the second one needs to wait [96:27] for this transaction to be processed. [96:29] But if there are other transactions [96:31] which are not trying to write to this [96:33] same row, then they are free to move [96:36] forward and basically they can go in [96:39] parallel instead of the serializable [96:41] where the transactions need to go one by [96:44] one. So the for update here prevents the [96:47] race condition by serializing access to [96:50] specific rows in this case. The read [96:53] committed is still the postgrql's [96:56] default and works fine here. And [96:59] serializable is fine to be used if you [97:01] have low TPS and not that much traffic [97:04] coming to write to the database every [97:08] second. The next phase is talking about [97:11] scaling the system. things like database [97:13] sharding, whether it's needed, caching [97:15] strategy, how we are going to optimize [97:17] the elastic search queries. So, let's [97:19] start from the database sharding. Since [97:21] we're going to handle 1,000 queries per [97:24] second and 17 transactions per second, [97:27] we need to eventually shard this and [97:30] that will be obvious by the time it [97:33] takes for typical queries to run. And [97:36] there are tables here that we can split. [97:39] When it comes to sharding, the important [97:41] decision is what will be the short key [97:44] and by which field you are going to [97:46] shart. One example here is to shard it [97:49] by hotel ID. Why? Because most queries [97:53] will filter by hotel first to see their [97:56] location whether it matches with the [97:58] search query filter and then they are [98:01] going to fetch the room types available [98:04] in this hotel. All data for one hotel [98:07] will still stay on the same shard. So we [98:10] will not be crosssharding the joints and [98:13] this will also enable local transactions [98:16] within the shard. For initial deployment [98:19] we can use modulo sharding which is the [98:22] formula for sharding. So the shard [98:24] number will be equal to this won't be [98:27] consistent hash initially. So it will be [98:30] just the hash of the hotel ID in [98:32] combination with the number of shards [98:35] that we will have. This will work well [98:37] when you know your shard count won't [98:39] change often. So if we decide to let's [98:42] say split this,000 read queries per [98:44] second between 16 shards then this will [98:48] range about 62 reads meaning queries per [98:52] second and one writes transactions per [98:55] second per shard. So in that case the [98:57] modular sharding will work fine and then [99:00] later that is where we can integrate [99:02] consistent sharding if needed to [99:05] minimize the data movement. So the thing [99:07] with modular sharding is that if we [99:10] decide to add another database let's say [99:12] number 17. This will require us to [99:15] reshuffle almost all the data from 1 to [99:19] 16. Whereas if we use consistent [99:21] hashing, this only moves about one of [99:25] the 17th of the data, which is the data [99:29] that is going to be stored in the 17th [99:33] chart. And you can check out the video [99:36] in the system design course where I talk [99:38] about consistent hashing. Basically, it [99:40] uses a hash ring where both servers and [99:43] keys map to points on a circle and each [99:47] key goes to the next server clockwise to [99:50] the ring. So, if you have a ring like [99:52] this, then you calculated the key which [99:55] is ending up here and the closest server [99:58] to that is here. Then this will be [100:00] stored in that server. So with [100:03] consistent hashing adding a new server [100:05] only affects the keys between it and the [100:08] previous server which is the number 16. [100:11] Whereas with modulo sharding you need to [100:14] basically reshuffle all the data. So [100:16] with small number of shards modulo [100:18] sharding is the way to go and if you [100:21] know that you'll be typically adding new [100:23] shards to the database then consistent [100:26] hashing is a better solution here. Now [100:28] let's see the caching strategy that we [100:31] can use for the hotels. We will need to [100:34] cache the hot inventory data in radies [100:37] or memcache for smart invalidation. So [100:41] this is the cache structure that we can [100:43] have which is for the key we will store [100:46] the inventory which will be the [100:48] combination of hotel ID, room type ID [100:51] and the date and for the value we will [100:55] store the inventory available. Let's say [100:58] in this hotel again 100 are available [101:01] and the total reserved which can be [101:03] something like 87 and we will also have [101:07] in combination with this the over [101:09] booking rate which will be 10% but of [101:12] course we don't store this in the cache [101:13] we have this as a fixed number somewhere [101:16] and when calculating we will use this in [101:18] combination with total inventory and [101:20] total reserved and we will assign the [101:23] cash a time to leave which can be [101:25] something like 5 minutes per inventory. [101:29] Now the cash flow will look something [101:31] like this. User will first search for [101:34] hotels and this is where we will check [101:36] the radius cache first for the hotel [101:38] inventory. If the cache is hit meaning [101:41] we stored this data in the cache. If [101:44] that is true then we will show the hotel [101:46] inventory to the user immediately from [101:49] reading from the cache. But in case it [101:52] is not in the cache we have cache miss. [101:55] This means that we need to read it from [101:57] the database then we will also populate [102:00] it after reading in the radis cache so [102:02] that next time this will be a cache hit [102:04] and also if we read it from the cache so [102:08] it was cash hit but the TTL is expired [102:11] then we basically repeat the same flow [102:14] so we read it from the database we store [102:16] it in the cache and then we show it to [102:18] the user after we show this to the user [102:21] let's say the hotel is available user [102:23] can click on booking. So to book the [102:26] hotel, this is where we will need to [102:28] validate the booking against database [102:32] and update the database with the booking [102:34] transaction. Again, this is where we [102:37] will invalidate the affected cache keys. [102:40] If they decide to book it and the [102:42] booking goes successful then we need to [102:45] basically invalidate the cache update [102:47] the fields and on the next user search [102:50] we will check the radius again and this [102:52] will be up to date with what just [102:55] happens when this user let's say books [102:57] the hotel. So next time we don't show it [102:59] as available but we show that this room [103:02] type is no longer available if it was [103:04] booked with the previous search. Let's [103:06] also discuss the search optimization [103:09] with elastic search and also a bit more [103:12] in detail on what's the use case for [103:14] elastic search. So elastic search is a [103:17] search and analytics engine that is a [103:20] specialized database for specifically [103:22] search operations and not transactional [103:25] workloads like with PostgrSQL or MySQL. [103:28] So, PostcarSQL or MySQL are still great [103:31] with give me the exact row where ID [103:34] equals to 1 2 3 or insert this into the [103:37] database where the hotel ID equals to [103:41] something. But on the other hand, [103:43] Elastic Search is great at finding [103:46] complex filters. For example, find all [103:49] documents that kind of match with this [103:52] text and rank them by relevance. filter [103:55] by multiple criteria and do it fast [103:58] across millions of documents. For [104:01] example, in our case, it will be find [104:03] all hotel rooms where hotel is located [104:07] within Georgia. Let's say the price [104:11] range is between 80 to 180 bucks per [104:16] day. The stars are above 8 plus and some [104:21] more stuff. And again, do it quickly. [104:23] This is why we use elastic search for [104:26] bookings as well. If you were to just [104:28] use postgrql for search, this is what we [104:31] would have to do like select all from [104:33] hotels where location is like Paris, [104:37] let's say, where amenities is like Wi-Fi [104:40] included, where base price is within [104:43] 200, where the description contains [104:45] something like luxury because we have [104:48] this in the search. Basically every time [104:50] you use the like keyword, this kills the [104:53] performance of SQL because that is very [104:55] slow. The problem here is that like and [104:59] then some text this does full table scan [105:02] in case of SQL and even indexes won't [105:05] help us here because in case of like [105:07] this is basically a full table scan and [105:09] you can imagine having so many hotels or [105:13] room types and we basically have to scan [105:16] the full table just for getting the [105:19] hotels available in Paris which is [105:21] probably less than 10% of the full [105:24] table. Also, it cannot rank by [105:26] relevance. So, it cannot get the which [105:29] hotel is most relevant. It cannot handle [105:32] typos. If a user misses the last S, [105:36] let's say from Paris, then again, you [105:38] need to use the like keyword or [105:41] otherwise if you set it to exactly [105:43] party, then you won't find any results. [105:46] It is very bad at geographical like [105:49] distance calculations and also [105:51] aggregations are expensive in SQL where [105:54] we are aggregating data together. [105:56] Instead, if we use elastic search, we [105:59] have the full text search with relevance [106:02] ranking. So when user let's say searches [106:05] luxury hotels in Paris, we first [106:08] tokenize the luxury hotel and Paris all [106:10] the keywords. We find all documents that [106:13] contain these terms. Then we also rank [106:16] them by relevance by using an algorithm. [106:19] One of these algorithms that elastic [106:21] search uses. It can also detect the [106:24] typos. So if in the Paris you missed the [106:27] I here, it can detect that user was [106:30] searching for Paris and edit the [106:33] distance of one. And it's also great at [106:36] hotels with ranking. So it can give you [106:38] the hotels within 5 kilometers of [106:42] coordinates close to Paris. So maybe you [106:44] are fine with booking a hotel near Paris [106:47] in a smaller town with a cheaper price. [106:50] Elastic search basically has that [106:52] indexes optimized for this. But how do [106:55] we basically store this data in elastic [106:57] search? Do we duplicate or what do we [106:59] do? And yes, we kind of duplicate the [107:02] data. So let's say you have the hotels [107:05] table in SQL. You have the hotel ID, the [107:08] name, address, and then you have the [107:10] room types within this hotel that belong [107:12] to this hotel specifically. You have the [107:15] amenities for this hotel and anything [107:18] else. Basically, elastic search stores [107:21] this in its own way, which means it will [107:24] convert this into this JSON like format [107:27] where it has the hotel ID, the name, [107:29] address, and then it will nest the [107:31] location and the room types in it. So [107:34] here as you can see there are no joints. [107:36] It basically condenses everything into [107:39] one document which will also be able to [107:42] use to search quickly. And then how do [107:46] we also keep elastic search in sync with [107:48] postgrql because our single source of [107:52] truth is still postgrql. This is where [107:54] we check whether the room is available [107:57] or not or whether we can do over booking [108:00] or anything else basically that you [108:02] check postsql is the single source of [108:05] truth. That's why when we are doing [108:07] transactions we are doing transactions [108:09] by checking the postgrql and not elastic [108:12] search. But when we are doing searches [108:14] we use elastic search in that case. So [108:17] we can use change data capture CDC to [108:21] keep elastic search eventually [108:23] consistent with postrSQL. This is how it [108:26] works. We have postSQL which is again [108:28] single source of truth. Then it uses [108:32] write ahead log to send it to the bases [108:36] connector which reads this write ahead [108:39] log. It publishes this to Kafka and [108:43] starts the event stream. Then elastic [108:46] search consumer will process the events. [108:48] So we will have some consumer that [108:50] processes these events and it updates [108:52] the index in elastic search and makes it [108:55] eventually consistent with the postgrql [108:58] database. This is also for crash [109:01] recovery. So if even the postgrql is [109:04] crashing then you can read it from the [109:07] transaction log. You have dbium which is [109:10] an open-source CDC tool. It connects to [109:14] PostgrSQL and reads the logs in real [109:17] time. Then it publishes the changes. [109:19] Whenever changes are happening, it [109:20] publishes it to a Kafka topic and each [109:24] table gets its own topic. So then the [109:27] consumer reads it from the Kafka events [109:29] from each topic for each table meaning [109:32] and it updates it with the elastic [109:34] search. That is overall the use case of [109:36] elastic search why we use it and how we [109:39] keep it up to date with our SQL [109:41] database. Next, it will also need [109:44] database replication this system because [109:46] we are handling thousand queries per [109:49] second. So, one option is the sharding [109:53] which is scaling the scans in the tables [109:56] and it's splitting them based on the [109:58] geographical locations. But you can also [110:01] scale it for the reads by replicating [110:03] it. So you can have the primary write [110:07] database and basically replicate it to a [110:10] read replica one in different [110:12] availability zone and a different [110:15] replica in another availability zone. So [110:18] then the user will connect to the [110:20] closest available database and the [110:23] search or the query will be faster. In [110:26] this case, if we imagine the [110:28] reservations, let's say we are quering [110:30] the reservations for this user with the [110:33] specific user ID. We can have a variable [110:36] which detects whether we should use the [110:38] primary or not. If it's a recent, write, [110:41] then we should use the primary because [110:43] the read replica might not be up to date [110:46] yet. If that's the case, then we will [110:49] get the primary connection and read it [110:52] from this database here. But if it's not [110:55] the case, if it's not a recent write, [110:57] then we can basically read from the [111:00] closest replica, we will get the replica [111:02] connection and run the query against [111:05] this replica instead of against the [111:07] primary database. Next, let's see also [111:10] the load balancing that we will use [111:12] here. We can have users from very [111:15] different various locations, right? We [111:18] can have users from US, from Europe, [111:20] from Asia, from other continents of the [111:23] world. So the algorithm for this of [111:27] course it makes sense to use geol [111:28] latency algorithm which is based on [111:31] their region. Of course there might be [111:33] exceptions. Maybe a US person is looking [111:35] to travel to Europe. So that is a [111:38] different story. But in general the [111:41] users in Europe are looking to book [111:43] hotels in Europe. The users in US are [111:45] looking to book hotels in US. So that's [111:48] the algorithm that makes the most sense [111:50] here. And the thing that we can [111:52] replicate here is based on the locations [111:56] we can have different database replicas [111:58] which are read only. We can have the [112:01] cache for radies again in each location [112:04] because we will probably cach the US [112:07] region users hotels in these regions [112:10] radius and it doesn't make sense to also [112:13] cach the Europe hotels here. So we will [112:15] have it in a different replica in a [112:18] different region and also the elastic [112:20] search because users again are more [112:23] likely from US to search for hotels [112:25] within US. So we have all that data in [112:29] one region which is quickly accessible [112:31] and similarly we have it for Europe we [112:33] have it for Asia and any other [112:36] continents that we support but the [112:38] primary database stays one here at least [112:42] even if this is one of the shards let's [112:44] say shard 7 this stays as a single [112:47] database to avoid having conflicts [112:50] between different regions rewriting to [112:52] the database and then we have to manage [112:54] the conflict so we can have one primary [112:56] database. Since the transactions per [112:59] second are not a lot as we saw in the [113:02] calculations, we can have one primary [113:04] chart and multiple replicas and regions [113:08] for each of these replicas. This is the [113:10] most in-depth breakdown of designing a [113:13] messaging app like WhatsApp from scratch [113:15] that you will be able to find on the [113:18] internet. Messaging apps like this show [113:20] up all the time in senior and staff [113:22] level interviews and that is because [113:24] they force you to handle stateful [113:26] connections. Think about real-time [113:28] delivery, consistency and scale of the [113:31] system, failures of pieces of the system [113:34] or overall disaster recoveries and [113:37] multi-device sync in case of users using [113:40] multiple devices like mobile phones and [113:42] desktop apps. So in a short interview, [113:45] you need to show that you understand not [113:47] just the happy path but also all the [113:49] things that can go wrong in messages [113:51] when they are flying around between [113:53] disconnected users and mobile networks. [113:56] And we are not going to ignore the hard [113:58] parts and pretend that everything just [114:01] works without us preventing the [114:03] failures. So we will start by first [114:06] building this from scratch. Starting [114:08] with a simple version that works for a [114:10] few thousand users. Then we'll scale it [114:12] up to handle billions of users sending [114:15] messages across unreliable mobile [114:17] networks. And after that, we'll identify [114:20] where it breaks and how to fix it [114:22] properly. At the end, you will also find [114:24] the full set of footnotes explaining [114:27] every technical concept that I mentioned [114:29] while designing this. So you can revisit [114:31] anything and you can go deeper on each [114:33] topic. Before getting to the high level [114:36] design and sketching diagrams, we need [114:38] to start with the requirements to [114:40] understand what we are building here. So [114:42] the core requirements that we have is [114:45] that users need to be able to send and [114:47] receive messages in oneonone setup. So [114:51] on top of that, we also have group chats [114:53] with up to 100 participants that we need [114:56] to support and messages are delivered [114:58] not only when users are online but also [115:01] when they are offline. We need to handle [115:02] that scenario as well. They can attach [115:06] media files like images, videos or audio [115:09] clips like voice notes. We need to track [115:11] the statuses of messages like when they [115:14] are sent, when they are delivered and [115:16] when the message is read from the second [115:19] user. And of course, we need to track [115:21] the online offline status to show them [115:24] the last scene date of when the user was [115:27] online. These are the core requirements. [115:29] When it comes to non-functional [115:31] requirements, we need to provide low [115:33] latency delivery. So under 500 [115:36] milliseconds when users are online. We [115:40] also need guaranteed message delivery. [115:42] So of course messages can't just [115:43] disappear and not reach to the second [115:46] user. In the scaling phase, we need to [115:48] handle billions of users with high [115:51] throughput. And to keep the storage [115:53] manageable, we don't store messages on [115:55] servers longer than they are necessary. [115:58] So it needs to also stay resilient when [116:01] individual components fail. Let's say [116:03] one of the components fails like the [116:05] server that needs to push the message to [116:08] the second user. We need to handle these [116:10] scenarios as well to make sure that the [116:12] user eventually receives the message [116:14] when the server is back online. With [116:16] that we can do some back of the envelope [116:19] calculations to understand the scale of [116:21] the system and then we'll start with the [116:23] highle design. Let's start with user [116:25] metrics. We have at the scale phase 1 [116:28] billion registered users and 500 million [116:31] daily active users. During peak hours it [116:35] will be handling 50 million concurrent [116:37] connections and on average we will [116:40] expect to have 10 to 20 messages per [116:43] user daily. With this we can start the [116:45] message estimations. For example, for [116:47] the daily messages, we need to multiply [116:50] the active users with the average daily [116:53] messages that will be sent, which is 20 [116:56] at max. And that will be 10 billion [116:58] messages sent every day in the system, [117:01] which means 10 billion writes to the [117:04] database every day. That comes down to [117:07] 115,000 messages per second on average. [117:10] So 115,000 rightes to the database. And [117:14] during peak hours, we can expect to get [117:17] three to five times more of the traffic [117:20] than on the average, which will be 350 [117:23] to 500,000 users rights on average per [117:27] second. Now let's also estimate the [117:30] storage. So each message with metadata, [117:33] let's just consider it to be 1 [117:35] kilobytes. It should include the text [117:38] content, the server and receiver ids, [117:40] the timestamped when the message was [117:43] sent and the status flux. Since we have [117:46] 10 billion messages per day, it will be [117:48] 10 billion times the 1 kilobyt. And we [117:51] will be storing 10 terabytes of storage [117:53] per day for the messages. And on annual [117:56] basis, this comes down to 3.6 pabytes [117:59] per year, which is 1,000 terabytes for [118:02] each pabyte. And the bandwidth will be [118:05] 50 million concurrent connections at the [118:08] peak traffic time. And each connection [118:10] will average out to 10 kilobytes per [118:13] second for active messages which comes [118:15] down to 500 GB per second of bandwidth. [118:19] One important thing here is that most [118:21] messages get delivered and cleared from [118:24] the server within seconds. So we are not [118:27] going to store this everything forever. [118:30] users download the messages to their [118:32] devices and also back up if they want to [118:35] for example to Google Drive and then we [118:37] can clean up the server side to not [118:40] store any unnecessary data. Here in the [118:43] beginning of requirements we have to set [118:46] it to be stored for 30-day retention for [118:49] undelivered messages and maybe let's say [118:51] 10% of users keeping history on servers. [118:55] So this allows WhatsApp to run on [118:57] minimal infrastructure because they are [118:59] very efficient about not sending the [119:01] unnecessary data. Now with this row of [119:04] calculations we can start with the very [119:06] high level design of what the system [119:09] will look like. Let's start simple and [119:11] add complexity only when we will scale [119:14] to more users. But for now this is the [119:17] initial setup. We will have the mobile [119:19] app which is the client like the user's [119:21] phone. It is responsible to maintain a [119:24] consistent connection with the chat [119:26] server, handle the UI and manage the [119:29] local message storage and also retries [119:32] when failed operations happen. The next [119:35] piece is the load balancer which sits [119:37] between the mobile app and the chat [119:40] servers. This will be responsible for [119:42] routing the incoming traffic between [119:45] multiple chart servers to handle the [119:47] peak load and traffic. These chat [119:50] servers will maintain a websocket [119:53] connection to the client, persistent [119:55] two-way communication channel that will [119:57] stay open and they will route messages [120:00] between users and track who's online and [120:03] handle messages persistence. Then we'll [120:05] introduce a message queue which will be [120:08] for decoupling message writing from [120:10] delivery. And this is also one of the [120:12] things that will allow us to handle the [120:15] failures. [120:16] So when one of the chat servers receives [120:19] the message, it will immediately [120:21] acknowledge and tell the mobile app that [120:24] the message was sent and then it [120:26] asynchronously pushes the message to the [120:29] message queue for storage and delivery. [120:32] And as an example, it is can be either a [120:34] Kafka message Q or rabbit MQ that will [120:37] work here. Now this message storage will [120:40] consume that message that is coming from [120:42] the server from the chat server and [120:45] persist it to the message storage [120:47] database as a database solution. NoSQL [120:50] will work better here something like [120:52] Cassandra or Dynamo DB because we need [120:55] high write throughput as you saw in the [120:57] estimations and our query patterns are [121:00] mostly straightforward. We will be [121:02] fetching messages by user by [121:04] conversation or by time stamp. And since [121:07] we're writing billions of messages per [121:09] day, we need something that can handle [121:11] the massive write volume without slowing [121:14] down. And that's why like something like [121:16] Cassandra or Dynamo DB is the best fit [121:19] here. Next, we'll need also some user [121:21] connection cache, something like radius, [121:24] which will be tracking which users are [121:26] online, which chat server they are [121:28] connected to, and their last activity [121:31] time stamp. This will help these chart [121:34] servers to make the routing decisions [121:36] faster because checking this ready [121:39] storage takes something like microsconds [121:41] versus quering it from the database [121:44] which will take milliseconds and at [121:46] scale that difference really starts to [121:48] show up. In the requirements we also [121:50] have the requirement to store media [121:52] files like messages with voice notes or [121:56] with images. For that we will use some [121:58] sort of globe storage. This can be like [122:01] the AWS S3 or Google cloud storage and [122:05] this will be storing the media files and [122:08] hold the actual files. But then we will [122:10] also distribute this to a CDN network [122:14] which will catch the popular files at [122:16] age locations worldwide. So that [122:18] downloads are fast regardless of where [122:20] the users are because one user might be [122:23] in US and the second user might be in [122:26] Europe. And in this case you will have [122:28] the same copy of the file in Europe [122:31] region as well. And for the Europe user [122:33] he will download it from the CDN which [122:36] is closer to his location. Next we need [122:39] also a notification service. This is [122:41] mainly for the offline users which need [122:44] to receive the notifications either [122:46] through Apple push notification service [122:48] or Firebase cloud messaging for Android. [122:52] And when someone messages you while you [122:54] are offline like let's say user A [122:57] messages to user B and then user B is [123:00] offline at the moment they will receive [123:02] instead the push notification and then [123:05] later they can read the message when [123:07] they open the app. And lastly, we need [123:09] also a dedicated presence service which [123:12] will be used for managing online and [123:15] offline statuses of the users. Let's say [123:17] user A again is online and user B is [123:21] offline. This presence server will [123:24] receive the heartbeats from users [123:26] updates from the radius and also publish [123:29] the presence changes to the interested [123:31] subscribers like the user A which is [123:34] interested whether the user B is online [123:36] or offline. Now let's choose the [123:39] communication method between the mobile [123:41] app and the services the chat servers. [123:44] We have three options here. Polling, [123:46] long polling, and websocket [123:48] communication. Of course, most messaging [123:50] apps use websocket for communication. [123:53] But let's understand why is that the [123:55] case by looking at their pros and cons. [123:58] Let's start with the polling option. [124:00] Let's say you have the client on this [124:02] side and then chat server on that side. [124:05] with the regular polling option. This is [124:07] how it works. We will do periodic [124:09] polling from the front end to back end [124:12] asking whether they have any new [124:13] messages or not with some API endpoint. [124:17] And then let's say we don't have any new [124:18] messages. So this will be an empty [124:20] response saying that there are no new [124:22] messages in the server. And then from [124:26] sometime again we will also need to [124:28] query the same thing and ask whether [124:30] there are any new messages. And then [124:32] let's say for the second time there are [124:34] new messages. So they will receive the [124:36] messages that they received. This as you [124:38] can see wastes a lot of bandwidth and [124:41] adds latency because if you're checking [124:43] every 2 seconds and the message arrives [124:45] right after you check, you wait 2 [124:48] seconds to see it while you could get it [124:50] instantly. And if you also multiply that [124:53] by the millions of users constantly [124:55] asking for updates even when there is [124:58] nothing new in the chat server, you're [125:00] burning a lot of resources for no [125:02] reason. That's why one better approach [125:05] than this is the long polling option. In [125:07] this case, we wait for the new messages [125:10] or timeout. So let's say we request the [125:12] new messages from the server and then [125:14] the chat server will wait for some time [125:18] until the timeout. If there are any new [125:20] messages, it will push them back to the [125:22] client. And if there are no new [125:24] messages, then after some time out, it [125:27] will push the response, which will be [125:29] again an empty response saying that [125:31] there are no new messages in the chat. [125:33] And if there are no new messages, then [125:35] this needs to again query to the chat [125:37] server and then basically this process [125:39] occurs again. This is still problematic [125:42] because you're constantly opening and [125:44] closing connections and each message [125:46] requires a full HTTP handshake. At [125:49] scale, this is again an overhead because [125:51] your chat servers spend more time [125:54] managing connection life cycles than [125:56] actually delivering messages. Plus there [125:59] will be some idle HTTP connections like [126:01] in this one where they are waiting for [126:04] the response and they will consume the [126:06] server resources like memory for [126:09] connection state and thread pool slots [126:11] without the efficiency benefits of [126:14] websockets. That's why most of the chat [126:17] services work with websocket [126:19] communication. This is one persistent [126:21] birectional connection between the chat [126:24] server and between the mobile app. So [126:27] client and server can both push data at [126:29] any time. First client will initiate the [126:32] HTTP connection with the server and then [126:34] they will perform the web socket [126:36] handshake and after this point the [126:39] connection is established and it is [126:41] persistent throughout the session. This [126:44] way let's say user A sends a message to [126:47] user B which is this user server is able [126:50] to independently push the new messages [126:52] to the user B which received the [126:56] message. So push the new messages as [126:58] soon as they are available and also the [127:01] client which is the user B is able to [127:03] send message or acknowledge any time [127:05] without making a new connection and this [127:08] connection is persistent and allows real [127:11] time birectional updates both from the [127:14] chat server side and from the client [127:16] side. The trade-offs of websockets is [127:19] that it requires special load balancers [127:21] that support them. So, we'll need to use [127:24] layer 4 load balancing instead of layer [127:26] 7. They also use more memory per [127:29] connection on the server side because [127:30] you're keeping connections open, but [127:33] it's still worth it because of the [127:35] real-time messaging and the latency of [127:37] the message requirements. Next, let's [127:40] see how we are going to manage the chat [127:42] servers which are routed through the [127:44] load balancer. [127:46] In production and in reality, chat [127:49] servers come and go. they will crash, [127:51] get deployed again, scale up and down. [127:54] And how does the system keep track of [127:56] what's available? So the problem here is [127:59] that the load balancer needs some way to [128:01] know which servers are available like [128:03] chat server one and two are available at [128:06] this moment and chat server 3 is [128:09] unhealthy. Let's say it either throws [128:11] timeouts or it's throwing errors on [128:13] requests. So one solution that we can [128:15] use here is service discovery cluster [128:18] like a service registry similar to [128:21] console or AWS cloud map and here is how [128:24] it will work. Whenever these chat [128:26] servers start, let's say this server one [128:29] boots up on some IP address, it will [128:32] register itself in this service [128:34] discovery cluster and after that we will [128:37] do health checks every 10 seconds with [128:40] this server one to see if the server is [128:43] up and running. And then similarly we'll [128:46] do that for server two and server three. [128:49] And now let's say the load balancer [128:51] needs the list of available servers. It [128:54] will first query the healthy servers [128:56] from the service discovery cluster and [128:58] then the cluster will send the server [129:01] list to the load balancer. With this new [129:04] list, let's say we forwarded only the IP [129:06] address of the chat server one and two. [129:09] So the load balancer will be able to [129:11] update the routing table and then route [129:14] all the traffic only to server one and [129:17] to server two and it will know that [129:19] server 3 is removed from the routing. So [129:21] we are not making any requests to the [129:24] server free. That's how we will manage [129:26] the load balancing and traffic routing. [129:28] Now let's move on to the data model and [129:31] designing the data tables that we will [129:34] have the primary ones. We will first [129:36] need a users table and this one can be [129:39] actually stored in a SQL database like [129:42] PostgresSQL or MySQL. So the user will [129:45] have an ID which will be the primary [129:48] key. Then a phone number which needs to [129:50] be unique for all users. Then a username [129:54] created at which is a time stamp and [129:56] last scene which we will use for [129:58] controlling and showing to the other [130:00] users when this user was last online. [130:03] And of course more fields will be here [130:06] but these are let's say the primary ones [130:08] that we will need. The next table will [130:10] be messages which is the table in [130:12] Cassandra actually where we will be [130:14] storing messages since this is a high [130:17] volume database. We will be writing and [130:20] reading a lot of messages from. We will [130:23] need the sender ID here which is the [130:26] connection with the user ID from the [130:28] users table. We will need the receiver [130:31] ID which is again a connection to the [130:33] same field in the users and a message ID [130:36] which will be the primary key for this [130:38] table. Apart from that any other things [130:41] that relate with the message like the [130:43] content which is the text media URL if [130:46] we are storing any voice notes or any [130:49] image video files. The status of the [130:51] message which will be either sent, [130:53] delivered or read. the time stamp of [130:57] when this message was delivered and [130:59] whether it's deleted or not because they [131:01] are also able to delete messages on [131:03] WhatsApp. For managing the groups on [131:06] WhatsApp, we will also need groups and [131:08] group members tables and this can be [131:11] again in the postgrql database. For the [131:14] groups, we will need ID for each group, [131:16] which will be the primary key, which we [131:18] will use for identifying the group, the [131:21] name of the group, when it was created, [131:23] and who's the user that created this [131:25] group. And then this is a one to many [131:28] connection with the groups and group [131:30] members where we will be adding new [131:33] members. And this is the ID of the group [131:36] where they are added, the role of each [131:39] member when they joined and the user ID [131:42] which links to the users table user ID [131:45] and that's the primary key of users. [131:47] Apart from this, one more important [131:49] thing is to have user connections [131:51] registry where let's say this can be [131:54] stored in a cache like radius. This will [131:57] store the user ids and the server ID [132:00] where they are connected to also the [132:03] connection ID. if it is open, last [132:05] active time and device type and we can [132:08] add this to the radius cache and then [132:11] update every let's say 300 seconds to [132:14] refresh it with each heartbeat. Once we [132:17] have the database ro schema, we can move [132:19] on to the API design specifically for [132:22] webocket and rest API flows for the [132:25] messaging app. The first step from the [132:28] client will be to authenticate with the [132:30] server by sending the user ID, the token [132:32] and device information. If it is [132:35] successful, the server will respond with [132:38] off success, sending the session ID, the [132:40] server time and the onlet count. After [132:43] that point, the client can start sending [132:46] and receiving messages. Let's say it [132:48] sends a message to the receiver. So it [132:50] will send the information about who's [132:52] the receiver, the user ID, the content, [132:55] the media if there are any and the time [132:58] stamp of the message. And at this point [133:00] the websocket connection is already [133:02] established. So there is a birectional [133:05] communication which means that server is [133:07] also able to push messages or [133:10] acknowledgements to the client. So it [133:12] will acknowledge for example when the [133:14] message is received. It will also [133:16] deliver any new messages that there are [133:19] from other users like the message ID, [133:21] the sender, the content and the media [133:23] files. And then client will be able to [133:26] also acknowledge when they receive the [133:28] messages from the server. The receiving [133:30] is one part. Maybe they received but [133:32] haven't opened the message. Once they [133:34] open the message, they will also send [133:36] the read received with message ID and [133:39] the time stamp when that was read. [133:42] Whenever let's say the message was read, [133:44] the server needs to also update the read [133:47] received with the message ID and when it [133:50] was read so that the sender who's on the [133:52] other side also knows when the message [133:55] was read. Also the client and server [133:58] need to maintain the presence heartbeat [134:00] between each other every let's say five [134:03] or 10 seconds. So client will check [134:05] whether the server is still available [134:08] and similarly the server needs to check [134:10] whether the client is available because [134:12] maybe you close the app on your phone or [134:14] you close the browser tab of WhatsApp [134:17] and in this case the connection needs to [134:19] be closed to not waste the bandwidth. [134:22] For uploading the media files we will [134:24] handle this through a rest API. For [134:27] example, for uploading a file, we will [134:29] use a post endpoint something like / [134:31] API/ media where we will send the file [134:35] type, the size of the file and also of [134:38] course the file and conversation where [134:40] this file is uploaded to. So this will [134:44] be sent to rest API which will be [134:47] responsible for handling the media [134:49] uploads. And then on the other side we [134:51] have also the media storage where this [134:54] will be stored. So server will respond [134:56] with the upload URL the media ID and [134:59] when this expires and after that the [135:02] client will be able to send the upload [135:04] file to the pre-signed URL for the media [135:08] storage and once upload is successful [135:10] the media storage will just respond with [135:13] the upload success and the client will [135:15] acknowledge it and as a last step let's [135:18] say it included the media file to be [135:21] uploaded the server will just [135:23] acknowledge that the message was [135:25] delivered or if there was any errors [135:27] while uploading the file or sending the [135:30] message, it will send the message [135:31] delivery failed response with the error [135:34] message in the body. Now let's cover the [135:36] multi-device sync. Let's say the user A [135:39] can have three devices and user B can [135:42] have two devices. How are we going to [135:44] sync messages between devices as well? [135:47] We have only this side user A's phone, [135:50] user A's tablet and user A's desktop. [135:53] These are the three devices for user A. [135:56] And on the other side we have user B's [135:58] phone and user B's laptop. So first [136:01] thing let's say first interaction that [136:03] happens is user A sends a message to the [136:06] chat server A. And then chat server will [136:10] acknowledge and display the status as [136:13] sent for this first message. Since we [136:15] agreed that we are going to store the [136:18] statuses of users availability in a [136:21] radius cache. So the chat server A will [136:24] then ask where is the user B because [136:27] let's say this message was sent to the [136:29] user B and then radius responds that [136:32] user B has two devices and it is [136:35] connected on server B and server E each [136:38] of these devices the other two devices [136:41] of first user will be connected let's [136:44] say one of them is connected to chart [136:46] server C and the second one is to chart [136:49] server D. So the once the chat server [136:52] stores the send message, it will also [136:55] display through the chat server C which [136:58] is connected to user A's tablet. It will [137:01] display the sent message and the chat [137:04] server D will display the sent message [137:06] for user A's desktop. This is how we [137:09] will sync the same message between three [137:11] devices of first user. Now let's see how [137:15] these messages are sent to the user B [137:17] and synced with the two devices that the [137:20] second user has. So user B's phone is [137:23] connected to chat server B and user B's [137:26] laptop is connected to chat server E. [137:29] The chat server B will read the message [137:32] from database and deliver it to the [137:34] phone. And this chat server will deliver [137:37] the same message to the laptop. And [137:40] let's say user B checks the message from [137:42] the phone. So it will deliver the [137:44] acknowledgement to chat server B which [137:47] sent the message to the phone after [137:49] which the chat server B will mark this [137:52] as delivered for the chat server A which [137:55] then it will be able to send to the [137:58] first user's mobile device and update [138:01] the status to delivered and next action [138:04] that will happen is user B will read the [138:06] message. So once it reads the message, [138:08] we will send the read receipt to the [138:11] chat server B and then this chat server [138:13] B will again forward this to the chat [138:16] server A with the message red status in [138:19] this case and then the chat server A [138:22] will update the status and now it will [138:24] display as red and since this is red now [138:27] we need to also sync that between the [138:29] user B's devices. So the chat server B [138:32] will mark it as red also on user B's [138:35] laptop which is here. So this way the [138:39] user A's tablet, mobile phone and [138:42] desktop and also user B's phone and [138:44] laptop always stay synced with the [138:47] latest data whether the message was sent [138:49] whether it was delivered or it was read. [138:52] And in case there are conflicts between [138:54] multiple devices, we will use the last [138:57] right wins technique with version [138:59] numbers to prevent the conflicts. And of [139:02] course this is not the ideal solution. [139:04] First of all, we need to apply [139:06] optimization here. For example, this can [139:09] be incremental sync because sending the [139:12] entire message history to a new device [139:14] is expensive. Let's say we are syncing [139:17] with this chat server B and user's phone [139:20] and also user's laptop with chat server [139:22] E. We don't need to send the whole [139:25] history in the device. So instead the [139:28] device can request messages for last 30 [139:31] days like most users only need the [139:33] recent history and older messages are [139:36] fetched on demand when needed. When they [139:39] scroll up the user scrolls up and then [139:42] the old messages will start to load and [139:45] very old messages will be archived in [139:47] cold storage and they will be loaded [139:50] lazily. Adding this multi-device sync [139:52] comes also with a trade-off because we [139:55] need to support roughly double of the [139:58] delivery workload. Each message goes to [140:01] multiple devices per users. But that is [140:04] an important feature of chat messaging [140:06] app like WhatsApp because users demand [140:08] the same messages the same chats to be [140:12] synced between their mobile phones and [140:14] between their desktop apps. Now let's [140:16] see the core flows on how messages [140:19] actually move in case of users both [140:21] being online and sometimes one of the [140:24] users might be offline. So let's start [140:27] with both online version. In this case [140:30] user A let's say sends the message to [140:32] user B and it's connected with the chat [140:35] server A. Message goes through web [140:38] socket and reaches to this chat server. [140:41] Now this chat server will generate the [140:44] message ID which is a unique ID [140:47] timestamp based for this message and [140:50] then it will send the acknowledgement [140:52] with status of the message being sent. [140:55] Then this server will push the message [140:58] to the message Q for persistence. After [141:02] that we need to check where user B is [141:04] and which server it's connected to so [141:07] that we can send forward this message to [141:09] user B and we can check that from the [141:12] radius cache where we have the user [141:14] connection cache. We will read where the [141:16] user B is from this cache and then let's [141:20] say it responds that user B is on server [141:23] B and once we have that information the [141:25] server A can route message to server B [141:28] via Rady's popsup model and on this side [141:32] we have the server B and user B [141:34] connected with web soocket. So server B [141:37] will now deliver message in real time [141:40] and user B's device will acknowledge the [141:43] delivery. Next, we need to let the first [141:46] server know that the message was [141:48] forwarded and delivered. So, server B [141:51] will make a request to the server [141:53] Acknowledging [141:55] the forward delivery. And then we can [141:57] update the status with delivered instead [142:00] of send. And then whenever user B opens [142:03] the message, the same process will [142:06] repeat and we will now mark the message [142:08] as read instead. And optionally we can [142:11] also incue the message for analytics or [142:14] archival by using async processing [142:17] between server A and between the message [142:19] Q. And meanwhile the message Q will be [142:23] writing the message to the message [142:26] database like Cassandra for storing the [142:28] history. Now let's see this same flow [142:31] but in case when the user B is offline [142:34] which makes it a bit harder to manage [142:37] but still we need to make sure that the [142:40] message is received and we also notify [142:42] the user B when they received even when [142:45] they were offline. So the initial steps [142:48] are the same for this situation. User A [142:51] sends the message to user B through chat [142:54] server A and then we check whether the [142:57] user B is online from the radius cache [143:00] and in case the user B is offline which [143:03] is our situation here we will add the [143:07] message to user B's inbox with radius [143:10] and then we will store this in inbox [143:13] table in the database inbox and after [143:16] that this chat server will connect with [143:19] the notification server. service that we [143:21] have here and send push notifications [143:24] for user B. Once the notification [143:26] service receives the message, it also [143:28] triggers push notification through [143:30] Apple's or Google's notification system [143:34] and then we incue the message for [143:36] further processing to the message queue [143:39] and after that message Q will persist [143:42] this into the database and once the user [143:46] B comes back online let's say they open [143:48] the app and establish the websocket [143:50] connection with chat server C we will [143:54] forward the message for delivery to the [143:57] chat server C which has the websocket [143:59] connection with the user B and after [144:02] that we will deliver the message in real [144:04] time to the user B's device [144:07] in case user B was online during this [144:09] time we will just forward the message [144:12] for delivery to the chat server where [144:14] the user B is connected let's say it's [144:16] the chat server C but if user B was [144:19] offline first and now they become [144:22] available this is the flow that we will [144:25] have. So let's say user B comes online [144:28] after being offline for a while. We [144:30] already sent the push notification and [144:32] they connect to one of the chat servers [144:35] like chat server C. First thing we will [144:37] do from this server is we will retrieve [144:40] the pending messages from the inbox from [144:42] radius and if there are any messages [144:45] that are pending we will retrieve them [144:48] any missed messages from the database [144:51] inbox and we will deliver the pending [144:54] messages to user B and similar to the [144:56] previous flow we will also notify the [144:59] user A that the message was read. Next, [145:02] we need to design this for group [145:04] messages as well and see how we can [145:07] handle 50 to 100 person groups in [145:10] WhatsApp. This is how it will look like [145:13] at a high level. Let's say we have [145:15] multiple users in the same group chat. [145:17] We will first of all need a group [145:20] service which will be managing these [145:21] groups. And as an example, let's say [145:24] user A is part of this group and we have [145:28] another 50 to 100 persons in this group. [145:32] First flow is when user A sends a [145:35] message to the group. Let's say for each [145:37] group we have ID as we described in the [145:40] data model. This gets sent to the chart [145:43] server where they are connected. And [145:46] then the chat server first request it [145:48] will make to the group service is to [145:50] check who's in this group which is ID of [145:53] 789 as an example and it will retrieve [145:57] the list of users like user ids who are [146:00] in this group part of this group and [146:03] after that we can check the status of [146:05] these 50 users we can use radius to cach [146:09] the status of users to not waste the [146:12] database queries for reading this and [146:15] let's Say we check the status of the 50 [146:17] users and find out that 30 of them are [146:20] online and the rest the 20 other users [146:23] are offline. For each online user, we [146:26] need to look up which chat server they [146:29] are on. So we will publish to channel [146:32] user one, user two to user 30 to the [146:36] radius popsup model and then this will [146:39] parallelly deliver to online users. [146:42] Whoever is online, they will receive [146:44] their messages in their own chat server [146:47] where they are connected. Let's say 15 [146:49] of the users are connected to server B [146:52] and the other 15 are connected to server [146:55] C. We will in parallel distribute the [146:57] message to both servers and then these [147:00] servers will distribute them via [147:02] websockets. So it will deliver via [147:05] websocket to the 15 users from server B [147:09] and to the other 15 users from server C. [147:12] For each of these deliveries, they will [147:14] acknowledge from user's device that the [147:17] message was delivered. And same for the [147:20] other 15 who are connected to the other [147:22] server. And then we will forward these [147:25] acknowledgements to the server A which [147:29] originally sent the message so that it [147:32] can update the status of the message to [147:35] send or delivered in the user A's [147:38] device. For the other 20 users who are [147:41] offline, we need to use the same thing [147:44] generally that we did for the offline [147:46] users in oneonone connection. We'll need [147:49] a radius inbox where we will write the [147:52] messages that they need to receive which [147:54] is let's say these other 20 users who [147:56] were offline. We will write it to user [147:59] inboxes and we will write to 20 inbox [148:03] records in the database inbox. And for [148:07] all of them we need to send the push [148:09] notification either through Apple's or [148:11] Google's push notification system to [148:14] notify them through notification service [148:16] that the message was sent in the group [148:20] and then the notification service will [148:22] push this through either Apple's or [148:24] Google's notification system. So this [148:27] way we are just storing message once. So [148:29] we are doing one write operation from [148:32] chat server A which received the message [148:35] from the user to the message Q and then [148:38] message Q writes this to the database. [148:40] So eventually it is just one write [148:42] operation per message and lastly we will [148:45] also update the status that it was [148:47] delivered to 30 and it is pending to 20 [148:51] users for the user a device. Now, so far [148:54] we haven't covered how we are going to [148:56] manage the media files, the image [148:59] assets, the videos. So, let's cover that [149:01] next. Of course, the wrong way here will [149:04] be to clients upload the image to the [149:07] chat server and then chat server store [149:09] it in database and then recipient [149:12] downloads from chat server. This is of [149:14] course not scalable. Your chat servers [149:16] are now bandwidth limited by file [149:19] transfers and they are not doing what [149:21] they are good at which is routing [149:23] messages quickly and instead they are [149:25] doing what they are terrible at which is [149:27] moving large blobs around. So the right [149:30] approach is using pre-signed URLs and [149:33] for all these technical terms like [149:34] pre-signed URLs notification services [149:37] Apple's notification service these kind [149:39] of things will be included in the [149:41] footnotes. If you're not sure what they [149:43] are, feel free to check it there. So, [149:46] let's look at the correct approach for [149:47] uploading the media files. Let's start [149:50] with the upload flow. Let's say we have [149:52] user A who's the sender of the file on [149:55] this site and we have user A's device [149:57] which is the app and we have user B who [149:59] will be receiving the file. So user A [150:03] taps to send a message and this is where [150:06] the application will request pre-signed [150:08] upload URL from the chat server which [150:11] they are connected to. Let's say this is [150:13] the server A. At this step the chat [150:16] server will generate pre-signed upload [150:18] URL to upload this into some blob [150:21] storage like AWS S3. The blob storage [150:24] will return the upload URL to the chat [150:27] server and then we will forward this URL [150:29] to user ace application. Once they have [150:32] this upload URL, the user ace [150:35] application can do a direct upload. It [150:37] can upload the media using the [150:39] pre-signed URL we just got from the [150:41] server and this will upload it directly [150:44] to the S3 bucket and then S3 will [150:47] confirm that upload and return the media [150:50] URL for accessing it later. And in the [150:52] next step, we can already send that chat [150:55] message including the media file URL to [150:58] the chat server A which we are connected [151:00] to. And this is where we need to notify [151:03] the user B that they received the [151:05] message and also include this media file [151:08] URL. So we will deliver the message with [151:11] media URL to user B's application. So [151:15] first user B will receive the [151:17] notification. [151:18] Next they will connect to the chat [151:20] server. Let's say this is the chat [151:22] server B that they are connected to with [151:25] via web soockets. So they will receive [151:27] the messages along with the media file [151:30] URL. And now then they tap to view the [151:33] media. This is where we will request the [151:35] pre-signed download URL. And in this [151:38] case we will generate the pre-signed [151:40] download URL from the chat server B with [151:43] the blob storage that we have on S3. And [151:46] then S3 will return the download URL to [151:49] this chat server B which we can now [151:51] forward to the user P application for it [151:55] to open and review the media file. And [151:58] then here we have two approaches for [152:00] downloading and reading the media file. [152:02] The more scalable approach is via CDN. [152:05] So we can replicate the media files [152:07] across CDNs to be closer to each user. [152:11] And in this case we will download the [152:13] media file through CDN. So this is the [152:16] user application and this is the CDN and [152:19] they will read it from there and stream [152:21] the media back to the user application [152:24] and the simplest approach is to just [152:26] read it directly from the storage. So [152:28] the user will request to download the [152:30] media file directly from the S3 bucket. [152:33] It will stream back the media. And of [152:35] course here we have a trade-off between [152:37] directly reading from the storage or [152:39] from CDN. We have slightly more complex [152:42] flow with CDNs because we need to do [152:44] extra requests to get the pre-signed [152:47] URL. But the performance and cost [152:49] benefits here are massive for this [152:51] scale. So CDN is most likely the best [152:54] solution and option to go with because [152:57] without this a single chat server [152:59] handling 100,000 let's say connections [153:02] can get crushed by just 100 users [153:05] simultaneously uploading some small [153:07] megabyte files into the blob storage. So [153:11] with this setup, the chat servers like [153:14] the server A only handles the small text [153:17] and messages and whenever we need to [153:19] upload some heavy files like videos, [153:22] images or voice notes, we will offload [153:24] that heavy bandwidth to some blob [153:27] storage or to some CDN and we will use [153:30] pre-signed URLs which will expire after [153:32] the TTL that we will set and then the [153:35] media TTL will align also with the [153:38] retention policy that we have in the [153:40] chat servers. [153:41] So with this setup, chat servers only [153:43] route small messages with URLs and blob [153:46] storage and CDNs will handle the heavy [153:49] bandwidth and our chat servers stay fast [153:52] and focused on their job which is [153:54] delivering the messages and media files [153:57] get the TTL which stands for time to [154:00] live with automatic expiration of let's [154:03] say 30 days which will match the message [154:05] retention and then they will get [154:07] automatically deleted. Now let's get [154:10] into how we would scale this because our [154:13] chat services now can handle thousands [154:15] of users concurrently. But if you want [154:19] to scale it even further, we need to [154:21] think of a way to decouple our servers. [154:24] And the most common approach here is [154:27] using some sort of message Q like a [154:29] Kafka. For our setup, we can have [154:32] message Q on this side and then we can [154:34] assign partitions to each instance. Over [154:37] here we have the message storage service [154:40] instances. Let's say we scale it up to [154:43] two, three or even more instances. Since [154:45] this is like a stateless instance, we [154:48] can scale it to as much as many as [154:50] needed. So we can assign partitions to [154:53] each of these instances. And as an [154:56] example like this can be from 0 to 9, [154:58] this can be 10 to 19. This can be 20 to [155:01] 29 partitions that we will assign to [155:04] each of these instances here. And the [155:06] way it will work is we will have [155:08] messages that arrive to this message [155:10] queue and then each instance will [155:13] consume these messages from Kafka. We [155:16] can even do batch rights for efficiency [155:19] and then we will handle retries and [155:21] failures inside of an instance. So first [155:24] we have the Kafka consumer in each of [155:26] these instances. We will try to use [155:29] batch writing to write this to the [155:31] database cluster. If for any reason this [155:35] fails, we will have a retry logic inside [155:37] of each of these clusters and then we [155:40] will retry to budge right after some [155:42] time out and write this to the database [155:45] cluster. And if for any reason this [155:48] still fails, we make sure that we don't [155:51] delete the message from the message Q so [155:53] that we don't end up losing the messages [155:56] or we can use something which is called [155:58] dead letter Q. These are cues that you [156:01] can use separate from the main queue. So [156:03] whenever some of the messages here fails [156:06] to be processed from the instance, it [156:09] will instead be stored in this dead [156:10] letter Q and then we can have some other [156:13] mechanisms here for processing it later [156:16] inside of the same instance and writing [156:18] it to the database. And then once a [156:21] database write is successful, we need to [156:23] also update the indexes. So inside of [156:26] each instance we will have the index [156:28] updater which will update the indexing [156:31] in the database for searching later for [156:34] querying them effectively. But let's [156:36] look at a more detailed flow of how the [156:39] processing will work from message cues. [156:41] So again on this left side we have the [156:43] message queue which receives all the [156:45] messages that arrive from user devices [156:48] and then we can set some batch size like [156:51] 500 messages for each batch and we will [156:54] consume this batch inside of one [156:56] instance. We will have the Kafka [156:58] consumer. This consumer can poll for new [157:01] messages every let's say 100 [157:03] milliseconds so that it has time to [157:06] process the previous ones and then it [157:09] will process the batch in parallel. [157:11] Let's say we can have thread pool of 20 [157:14] workers and we can process this in [157:16] parallel. For each of these single [157:19] message, we will validate the schema [157:22] first of all and then do meta data [157:24] checks to check the size of the file. If [157:27] they uploaded some media file that [157:29] exceeds the limit, we will need to [157:32] respond with an error message. And [157:34] otherwise, if it passes the criterias, [157:37] we will validate and enrich the schema. [157:40] Here is where we can calculate the [157:42] message hash for dduplication. Extract [157:45] the mentions like if they mentioned some [157:48] username right in the WhatsApp you can [157:50] mention someone with at symbol for [157:52] example. So if they mention that we need [157:54] to extract those mentions from the [157:57] message content. We need to identify the [157:59] links for preview generation because on [158:01] WhatsApp you also generate the links if [158:04] they attached anything to the message. [158:07] And this is also where we need to [158:09] generate the indexes for querying them [158:12] later. And next, instead of writing all [158:15] these messages one by one, we can cue [158:17] the validated messages into a batch [158:20] writer, we can accumulate some batch of [158:23] messages in the memory like 1,000 [158:26] messages every 5 seconds. And whenever [158:29] this buffer hits the 1,000 limit, we [158:32] will start sorting messages by the [158:34] target chart. We will group messages by [158:37] conversation and then we will prepare [158:39] the botch for inserting into the [158:41] database. And the reason we are doing [158:44] this is to save write operations in the [158:47] database. Of course, if we can budge [158:49] thousand messages and then try to budge [158:51] write them into the database. So we will [158:54] still prepare insert statements for each [158:57] of those that cannot be inserted in one [158:59] go. But at least that saves the database [159:03] sometimes to process the current batch [159:05] and then after the 5 seconds we will [159:07] send it the next batch. So the first [159:10] happy path is that it will be inserted [159:12] with atomic batch insert into the [159:14] database cluster. We will also update [159:17] the search and analytics and indexes via [159:20] index updater and then we will mark them [159:23] as processed if everything goes well. [159:26] But in case this fails, we will have a [159:28] failure and then we will have the retry [159:31] and dead letter Q handler. So if these [159:35] messages or one of these messages, let's [159:37] say out of this thousand, we have 10 of [159:40] them which failed to be processed, we [159:42] will write them into the dead letter Q. [159:45] And before writing it to dead letter Q, [159:47] you can have some simple logic that [159:49] tries to retry. Maybe that was a rate [159:53] limit we hit or something else simple [159:55] that we can just retry. If after that [159:57] point it fails, we write it to the dead [159:59] letter Q. And then here we will have [160:01] more mechanisms like step one, step two, [160:05] two, step something like 10, which are [160:08] the ways we will try retry to insert [160:10] this into the database before sending [160:13] the user a notification that this [160:16] message was not delivered because of [160:18] some reason. So the main benefit that we [160:21] get with dispatching is a single [160:23] database insert takes 5 milliseconds. If [160:27] we are inserting something here, it [160:29] usually takes 5 milliseconds. And all of [160:32] these thousand individual inserts will [160:35] be equal to 5 seconds instead. 1 [160:38] millisecond. 1 second is 1,000 [160:41] milliseconds. But if we do one batched [160:43] insert of thousand messages, it will [160:46] instead end up costing us 50 [160:48] milliseconds which is actually 100x [160:52] throughput improvement and it will start [160:54] to show up when we are starting to write [160:57] millions of messages concurrently to the [161:00] database. And if you'd like to see this [161:02] at the code level as an example, this is [161:04] how we will write to Cassandra database [161:07] with the message chart. So this will be [161:10] a transaction. We will begin the batch. [161:12] We will insert the messages into the [161:15] messages table and we will pass the [161:18] values like the timestamp. the ID of the [161:21] sender and receiver which are the user [161:24] ids then the content then anything else [161:26] like media files and then similarly we [161:29] will do all the other inserts of other [161:32] messages into one batch and then this [161:35] will be like one batch of writing 1,000 [161:38] messages into the messages table in [161:41] Cassandra database the next important [161:44] feature of WhatsApp that you might be [161:46] asked to design is to add end to end [161:49] encryption and key management. Typically [161:52] on WhatsApp, messages are encrypted on [161:54] the sender's device and only decrypted [161:57] on the recipient's device. Even WhatsApp [162:00] servers can't read the content. So if [162:02] you have user A and user B and you're [162:05] sending some messages between each [162:07] other, even the user that is in the [162:09] middle or the user that is connected [162:12] with the chat server A and then the user [162:14] B which is connected to chat server B. [162:17] Even this A and B servers cannot read [162:19] the message. So how we are going to [162:21] design something like that in messaging [162:23] app? The thing we can use here is a [162:25] signal protocol. So we can add end to [162:28] end encryption. And this is how it will [162:31] work in detail. Whenever user A sends a [162:34] message to user B, we upload the public [162:38] identity token into the key server. So [162:41] each user generates an identity key pair [162:44] on registration on WhatsApp and then [162:47] they have the public key uploaded to the [162:49] server and private key which never [162:51] leaves the device. So you have private [162:54] key which is on user A's device and you [162:57] will have the public key which server [163:00] has access to and then on the other side [163:03] we have the user B. So user B will again [163:06] upload the public identity key to the [163:09] key server plus they will upload [163:11] prekeys. These are generated let's say [163:14] 100 signed prekeys and it uploads them [163:17] to the server. Server stores them for [163:20] future initiators. And whenever user A [163:23] sends a message, we will also fetch the [163:25] user B's keys which includes their [163:27] identity key plus one of these prekeys [163:30] that they stored. Let's say they stored [163:32] something like 100 keys here. We will [163:34] fetch one of these for user A. And then [163:37] once the key server returns the user B's [163:40] keys to user A, this is where the [163:42] session will be established. So user A [163:44] fetches the user B's public identity key [163:47] and prekey from the server. It generates [163:50] the key pair which will match for both [163:53] user A and B and then it will perform [163:56] the key exchange. After that both user A [163:59] and B have access to the same keys. So [164:01] whenever they are sending messages [164:03] between each other, all of the messages [164:05] will be encrypted with user A's public [164:08] key and then once user B receives the [164:11] message will decrypt the message by [164:14] using the user A's public key. So each [164:17] of these messages will be encrypted with [164:20] some of the algorithms and then we will [164:23] have the message the session key which [164:25] is exchanged between these users and [164:28] even further we can use something like [164:30] message counter which will increase [164:32] after each back and forth messages [164:34] between these user A and B. So this way [164:37] it's very hard even if the session key [164:40] is stolen. It will be still very hard to [164:42] decrypt each of the messages because all [164:45] for each message we are also adding the [164:47] counter for encrypting it and then we [164:49] are decryting it on the other users end. [164:53] So this is how we will ensure the [164:55] privacy between user A and B and even [164:57] the chat servers we won't be able to [165:00] read the messages in between. Now let's [165:03] move on to security practices. But in [165:05] terms of the overall system, let's start [165:08] from rate limiting and abuse prevention. [165:11] Without having any rate limits, any [165:13] hacker can start scraping data or [165:16] perform DDOS to the system. So we need [165:19] to build a protection in any kind of [165:21] system like this. First of all, we have [165:24] the user actions here which can be [165:27] messages sent from the user or files [165:30] uploaded to the media storage. For every [165:32] action type, we need to have some [165:34] protection. And these are the per user [165:37] limits. For example, that we can send [165:39] whenever users are sending messages or [165:42] they are creating groups or they are [165:44] uploading media. So whenever they are [165:46] sending a message, we can have a limit [165:48] for sending messages. We need to check [165:51] first the message token bucket. And here [165:53] we can have the rules set. For example, [165:56] these can be 100 messages per minute [165:59] that a user can sense. And anything [166:02] above that doesn't seem reasonable. So [166:04] we can apply rate limit for that [166:07] situation. And if the rate limit is [166:10] exceeded, we will block the action. So [166:12] we will notify the user and show an [166:15] error message that they exceeded the [166:16] limit. And this can be something like [166:19] too many requests in one minute try [166:21] again from 60 seconds for example. Or we [166:25] can require some capture to be solved. [166:27] So these boxes that they need to check [166:30] and let's say pick which ones are [166:32] motorcycles to verify that they are [166:34] human. And then we can also have [166:37] progressive penalty. So the first one is [166:40] the warning that we show which is too [166:43] many requests. If they start abusing the [166:45] service again like we will have [166:47] restriction for one hours. This is how [166:50] you see for example in passwords where [166:52] you're trying to type your password for [166:54] an account and you fail multiple times. [166:57] They will first restrict you from for [167:00] some time, right? The second time they [167:01] might restrict you even more like 24 [167:04] hours. And then the next step after that [167:06] is we will suspect the account and we [167:09] will deactivate it for some time for a [167:12] human to manually review their account [167:14] and verify whether this was a bot or [167:17] this was a hackers trying to steal [167:20] information or it might be still the [167:22] user who just didn't understand the [167:25] requirements and they kept abusing the [167:27] service. So we need to contact them, let [167:30] them know the issue and then they will [167:33] reopen the account. Of course with the [167:35] same rate limits applied and they might [167:38] even get a tag on their user that this [167:41] account is suspicious. So if anything [167:43] happens one more time their account [167:45] might be permanently deleted from [167:47] WhatsApp. And now this same process [167:50] repeats for all the other actions as [167:52] well. Let's say users can create group [167:54] as well. Right? So once they create a [167:57] group, we will check the group crate [167:59] counter. And this is where we need to [168:01] set reasonable limits. For example, if [168:03] they are creating five or more groups [168:06] per hour, that might seem too much. So [168:08] we can apply a rate limit for that [168:10] situation. Or if they upload media and [168:14] they upload more than let's say 10 or [168:16] 100 media files per minute, this can be [168:19] another restriction. Of course, these [168:21] numbers are just examples and these are [168:24] some things that will be clarified in [168:26] the blueprinting phase. But for each of [168:28] these actions, you need to have some [168:30] rate limit. And then the process is [168:32] usually identical for enforcement. So we [168:35] need to check if the rate limit is [168:37] exceeded. We will block it. Otherwise, [168:39] it will work as expected. So we will [168:41] allow and then we will continue with [168:44] writing to the database whether it's a [168:46] media file or it's a message or it's [168:48] creating a group. This is an example [168:51] code block like how this can be [168:53] implemented the rate limiter. So we will [168:56] initialize with the capacity and with [168:58] the refill rate which can be different [169:00] for each action and then we can have [169:03] something like tokens which show the [169:05] capacity. So for every request we will [169:08] check whether they exceeded the tokens [169:10] or they are still under the limit. So it [169:13] can be some unit like one and if this is [169:16] more than one which means they haven't [169:18] received the rate limit we will return [169:21] true and this will not really process [169:23] with other actions like storing the [169:25] message to the database and then we will [169:28] deduct one from each step. So if they [169:31] try this next time this will be zero of [169:34] for the tokens. So that will end up not [169:36] getting into this if blocks and then [169:38] instead they will get into the else [169:40] block and we will return false because [169:42] they exceeded the rate limits and this [169:45] will block them. This is the step here. [169:47] So we will block them for processing the [169:50] message or creating the group and we [169:53] will notify them and alternatively we [169:56] can also ask them to solve a capture [169:58] problem and then for that we will set a [170:01] limit right for let's say 1 hour. So [170:03] this is where we will have to refill. We [170:06] will check the time that we have now. We [170:08] will set the last refill time and then [170:10] based on this we will calculate when is [170:13] the next time they can repeat the same [170:15] action which can be from 1 hour from [170:17] now. The next action type that we need [170:20] to limit is connections because we can [170:22] have connection flow the tax and let's [170:25] say we get five connection attempts in [170:27] the single minute for different IP [170:30] addresses. This is where we need to [170:32] check their IP connection counter and [170:34] again apply the rate limit for this [170:37] specific situation. So if within a [170:40] minute they have tried to login from [170:43] five different IP addresses that is [170:45] something that looks suspicious. So this [170:47] is again where we need to apply the rate [170:50] limiting and it will follow the same [170:52] process as with the other actions. [170:55] Another example is we can have group [170:57] invite limits. So if they invite users, [171:00] they create a group and then the user [171:02] starts inviting other users to the [171:04] group. We will check the group invite [171:07] counter in this case. [171:09] And similar to all the other actions, it [171:12] will follow the same process here. Just [171:14] the counter and the limit is different [171:16] for each of the actions. [171:18] So for this to work, we need to also [171:21] have some centralized store for all [171:23] servers because each of the users is [171:25] connected to one server, right? with a [171:27] websocket connection. So all of them [171:30] need to use some distributed central [171:32] store something like radius because this [171:35] is something we keep up to let's say 24 [171:38] hours in the memory and that is in the [171:40] case they get restricted with the most [171:43] attempts or we can still use database [171:46] for keeping this data but if it's [171:48] something we are going to keep up to 24 [171:51] hours then radius might be a good option [171:53] here and then our atomic operations will [171:56] ensure that we have the right accuracy [171:58] And there is no way to bypass the server [172:01] hopping by connecting to other server [172:04] and wishing that it will work and the [172:06] rate limit isn't applied or that server. [172:10] Next, we need to scale the database as [172:12] well. Of course, with this scale now we [172:14] scaled the servers and applied rate [172:17] limiting. But the database a single [172:19] database cannot handle all this traffic. [172:22] So we need to split data across multiple [172:25] database instances which is also called [172:27] sharding. When it comes to sharding, one [172:30] of the important decisions is on which [172:33] field we are going to shard by for the [172:35] databases. So one approach can be to [172:38] shard it based on user ID. So this way [172:41] we will make sure that all the rights [172:43] all the messages for user A end up in [172:46] one shard in one database. So that [172:49] whenever we are fetching messages, we [172:51] are fetching only from one database and [172:53] we don't have to connect to multiple [172:55] shards to read the messages for the same [172:57] user. The catch here is that individual [173:00] users can become hotspots because some [173:03] users naturally use it way more often [173:06] than others. And this way these users [173:08] will start creating big databases, big [173:11] tables of messages where the users who [173:14] use it less often will create smaller [173:16] charts. So this will create an [173:19] imbalanced sharding in this case. [173:21] Another option is to shard it by message [173:24] ids. This will create even distribution [173:27] of data. But fetching all messages for a [173:30] user will require quering multiple [173:32] shards because there will be in [173:35] different shards. So your query will hit [173:37] every chart, aggregate the results and [173:40] then return them to the server. So the [173:42] downside in this case is it will become [173:44] slow and expensive compared to the user [173:47] ID. And the downside with the user ID is [173:50] that we will have imbalanced shards. So [173:52] one shard might be very small compared [173:54] to the other shard which uses users who [173:58] are using it very often. So the main [174:00] goal also for these interviews is not to [174:03] just choose one of these but just tell [174:06] them the different approaches that there [174:09] are and what is the best solution. after [174:11] all or what is the solution you will go [174:14] with? So one of the good approaches here [174:17] it can be sharding still by user ID but [174:21] applying replication. [174:23] So let's say user A is sharded to this [174:26] shard A database. All of the user A's [174:29] conversations will be stored in the [174:32] shard A. But if he talks to user B who's [174:36] connected to shard B, we will also [174:39] duplicate the rights. So we will write [174:42] the copy of each user's shard into the [174:45] shard B as well. And of course this [174:48] comes with a downside. This doubles the [174:50] right volume. But on the other hand, it [174:53] makes the reads blazingly fast. Each [174:55] user will query only their own shard. [174:57] User B will query it from shard B and [175:00] user A will query it from shard A. And [175:03] we can even replicate each of these [175:06] shards. So shard B will be replicated [175:08] into B replica and shard A will be [175:11] replicated into A replica and user A [175:15] will now read this history of messages [175:18] only from the replica A and user B will [175:22] read it from replica B. they won't [175:24] affect the shards the primary ones which [175:27] are responsible for write operations and [175:29] instead they will be reading here. So it [175:31] will increase the read performance a [175:34] lot. The trade-off here is that we are [175:36] writing each message twice. So that's [175:38] why it's also important to clarify such [175:41] things in the beginning in the [175:43] requirements phase. You need to see this [175:44] in advance like later we are going to [175:47] optimize the messages either for write [175:50] operations or for read operations and [175:52] which is the more important and if you [175:55] still failed to clarify that at the [175:57] beginning this is still a good place to [175:59] ask instead of trying to guess. So ask [176:02] them whether reading is the more [176:04] important in this situation or writing [176:06] and then based on that you will either [176:08] replicate or you will go with message ID [176:11] sharding or you can even go with [176:12] timestamp ID sharding. There are [176:15] different approaches here and the main [176:17] thing here is to understand the [176:18] tradeoffs between each approach. So [176:21] typically when it comes to keeping the [176:23] data available we usually shard for [176:26] right scalability. So if you're applying [176:28] sharding which means splitting the [176:30] database into smaller chunks this is [176:33] improving the right performance and when [176:36] we apply replication this is for [176:38] improving the read scalability. So if [176:41] you just replicate one of the shards [176:43] this is usually for the read [176:45] performance. So once you decide on the [176:48] sharding strategy and let's say you [176:50] decided to go with user ID sharding [176:55] all the client rights will arrive to the [176:58] shard to the primary node for each of [177:01] the shards. Let's say user ids from 1 to [177:05] 1,000 are stored into the shard one. [177:09] This will be the primary node and then [177:12] we can basically replicate this and use [177:15] asynchronous syncing between the primary [177:18] node and replicas. We can have three [177:20] replicas and they will be eventually [177:23] consistent with the primary node by [177:26] reading and duplicating the data into [177:28] the replicas. And then whenever we have [177:31] read operations like opening the [177:33] messages and seeing the unread messages, [177:37] this is where we will need to perform [177:38] read queries. And for read queries, we [177:40] will only use the replicas. We won't be [177:43] connecting to the primary node at all. [177:45] We will connect here only when we have [177:47] write messages which is someone writing [177:51] to the user A or user A writing to [177:54] someone else. Now next for such big [177:56] systems where we are storing too much [177:59] media files, too many messages, you need [178:02] some cleanup way, some way to clean up [178:05] the garbage data that is never read [178:08] again. So this can be expired media, old [178:11] undelivered messages or stale [178:13] connections. We need to build a cleanup [178:16] infrastructure to run some chrome job [178:19] over time and try to clean the expired [178:22] medias and stuff. So you need to have [178:24] some job scheduler something like [178:26] airflow or some chrome job that will be [178:29] scheduled every either daily basis or [178:32] weekly basis and this needs to check the [178:35] job status from monitoring. Then on this [178:38] side we have a worker pool which is [178:41] containerized job executors. [178:44] So this job queue that you see here this [178:46] is separate from message Q. This is for [178:48] incuing scheduled tasks and pulling the [178:51] jobs to the worker pool for processing. [178:54] Then we will acknowledge that we [178:56] received the message or if we failed [178:59] then we will recue it on failure and [179:01] then we will start executing the tasks [179:03] in this worker pool which can be [179:05] cleaning up expired media by checking [179:07] the date that they were published or the [179:11] undelivered messages. These can be [179:13] messages that we failed to write to the [179:15] database but we still have them [179:17] somewhere in the memory. So we can [179:19] remove this or stale connections that we [179:22] have and any other kind of data that we [179:26] don't need anymore. This is what the [179:28] responsibility of this worker pool will [179:30] be to delete all the data and then [179:33] submit the job results to the monitoring [179:35] so that we can see over time how well it [179:37] is doing and if there are any [179:39] improvements here as well because this [179:41] is not something that you will set up [179:42] once and then forget. This is also [179:44] something that will be improved over [179:46] time. So the trade-off here is that [179:49] these scheduled jobs will add [179:51] operational complexity because it is [179:54] another big thing to monitor and debug. [179:57] But the alternative to this is or the [179:59] other tradeoff is to let the garbage [180:02] accumulate until the system becomes so [180:05] big that you have to do this either way. [180:07] So regular cleanup like this is [180:09] essential. And for long-term health, [180:12] this is important to set up some worker [180:14] pool that will start cleaning up the [180:16] junk data that is not used. That's the [180:19] general flaw of designing such large [180:22] system like WhatsApp. But you won't [180:24] usually end there because that is just [180:26] the happy path of all services work [180:29] together, everyone is happy, customers [180:32] are happy, right? But that's never the [180:34] case in reality. So there can be [180:36] failures in every part of the system. [180:39] And one of the major things that you can [180:42] discuss for example for a messaging app [180:44] is cache failure. For example, what [180:47] happens to undelivered messages because [180:49] radius holds our undelivered messages [180:52] inboxes and what if the radius cluster [180:55] goes down itself. Let's start with the [180:57] problem example. Let's say user A sends [181:00] a message and they are connected to chat [181:03] server. So this is where they [181:04] established the veet communication both [181:07] ways. Now chat server will first write [181:10] the undelivered messages to the radius [181:13] inbox which is the fastest compared to [181:16] the database and then the second step is [181:18] we don't only write to the ready store [181:20] right we also write the backup which is [181:23] more durable to the database so this is [181:26] what our permanent storage is in case [181:28] the radius cluster goes down. So we will [181:32] first try to fetch the pending messages [181:35] from radius for the chat server to [181:37] forward it. But in case this is [181:39] unavailable let's say radius goes down. [181:42] We need to fall back to the database [181:44] instead. So in this case if radius is [181:46] empty or it's unavailable then we need [181:49] to fall back and read from the database. [181:52] So we need to retrieve the undelivered [181:54] message from the database and then [181:57] forward it to the server. And either [181:59] way, we need to at the end be able to [182:01] deliver the message to user B even if [182:04] they are offline and then they come back [182:06] online and fetch the messages from the [182:09] database in case the radius cache was [182:11] down. And whenever the radius recovers [182:14] and comes back online, we will first [182:17] have a background job that will sync the [182:19] inbox from database to radius. And then [182:22] we will query something like select all [182:24] from messages inbox where the delivered [182:27] is null which is not set in inbox. And [182:30] then for each undelivered message we [182:32] will write to radius inbox and then we [182:35] will keep the database record as well. [182:37] So after that we will resume to the [182:40] normal delivery which is writing to [182:42] radius and then reading from radius [182:44] again. [182:46] So to visualize this at a code level [182:48] this is for example what the write with [182:51] fallback will look like. So we will have [182:53] some function like store undelivered [182:55] message which will be the user ID of the [182:58] message and then the message itself. So [183:01] we will first try reading radius from [183:04] the cache. So we will try to get the [183:06] message with this user ID with the [183:08] message timestamp from radius cache. But [183:11] in case this goes wrong, we need to log [183:13] that the radius is unavailable and we [183:16] will need to start using database again. [183:19] So we will need to query this from [183:20] database. This can be either insert [183:23] operation if the message was written to [183:26] the other user or this can be a read [183:28] operation like select operation if the [183:30] user is trying to fetch the messages and [183:33] the radius is unavailable. [183:35] So that is the second scenario. For [183:37] example, we are trying to get [183:39] undelivered messages. So we are trying [183:40] to read it from radius. This is again [183:42] trying to read from radius because that [183:44] is the fastest if it's available first. [183:47] If we see that messages length is [183:49] greater than zero then we will map parse [183:52] and then send it to the user. But in [183:54] case something goes wrong in this try [183:56] block that means that the radius is [183:58] unavailable. So again we will log for us [184:01] for tracing it later in cloudatch or [184:04] somewhere else and after that we'll need [184:06] to again read this from the database. So [184:09] we'll do the select query instead to the [184:11] database which is again slower but it's [184:14] better than just failing the whole [184:16] operation and then once the radius is [184:18] back online it will sync with the [184:21] database have all the messages here. So [184:24] next time we will be reading from this [184:26] try block. [184:28] The next topic that you can expect to [184:30] discuss is about regions and global [184:33] distribution because WhatsApp is not [184:35] only in let's say US or it's not only in [184:38] Europe. It has users from all parts of [184:42] the world. So you need to have different [184:45] regions for each of these users. For [184:48] this we can use the regions. For [184:50] example, if you are using US, we can use [184:52] the US East one, US West 2 or these kind [184:56] of regions that are available to us in [184:58] the AWS or the cloud provider that we [185:01] are using. And each region should have [185:04] the full stack chart servers, radius [185:07] clusters, database charts so that it can [185:09] operate independently. Let's say for [185:12] example, user A opens their app and they [185:15] are located in Tokyo. So we will first [185:18] do the DNS lookup to see where they [185:21] should connect first and then they will [185:23] connect to the nearest IP address which [185:26] will be the Singapore region and in this [185:29] region we will have the full stack which [185:32] will be the load balancer that they will [185:34] first connect to. They will make a [185:35] request here to route them to the chat [185:38] server. Here we will have more chat [185:41] servers replicated and for each region [185:43] this can be different. For example, for [185:45] US maybe we are or let's say for Ireland [185:48] we have a lot of users using the app. So [185:50] here it can be many chart servers and [185:53] let's say in Tokyo or in Singapore we [185:56] have less users. So we will have less [185:58] chart server replicas. Either way they [186:00] will connect with a chat server with [186:02] websocket connection and they will have [186:05] the radius cache for each user. This is [186:08] for example where we will look up where [186:10] user B is to be able to send the message [186:13] to their region and that is how the user [186:16] B let's say in London will connect. So [186:18] first again they will do DNS lookup. [186:20] They will connect to the nearest IP [186:22] address. They will make a request to the [186:24] load balancer and they will receive the [186:26] message that was sent from user A to B [186:29] through the US West region which is the [186:32] one that they are connected to. Now [186:35] another thing that can go wrong here is [186:37] the Kafka cluster that we have. Kafka [186:40] itself is very scalable. It can handle [186:43] thousands of messages per second without [186:45] any issues. But still we need to plan [186:48] what happens in case this goes down or [186:51] something happens with the Kafka [186:52] cluster. [186:54] Inside of the cluster we typically have [186:57] multiple brokers which are called [186:59] servers in the cluster. So topics are [187:02] split into partitions and each partition [187:05] has leader and it has replicas. So [187:08] producers usually write to the leader [187:11] similar like write database actions and [187:14] replicas just provide us with [187:15] redundancy. So scenario one is that one [187:19] of the brokers fail inside of the Kafka [187:22] cluster. So if a broker dies, we need to [187:25] replace the broker. Let's say this was [187:27] broker number something like three. we [187:30] need to replace it and rejoin another [187:32] broker to the cluster. So, Kafka will [187:35] rebalance the partitions over time if we [187:38] add this broker to the cluster. But what [187:41] if the entire Kafka cluster goes down? [187:44] This is very rare, but it's still [187:46] possible. So, this means that chat [187:48] servers can't publish any messages. All [187:50] of the chat servers were sending [187:52] messages to this cluster, right? And now [187:54] they are no longer able to do that. So [187:57] message storage service can't consume [187:59] the messages which means that messages [188:02] are not persisted and this can be data [188:04] loss. In such cases we need to have a [188:07] mitigation of a local buffer. So let's [188:10] say we create a message buffer before [188:12] trying to publish them to Kafka in the [188:14] code. And this is what the publish [188:16] message would look like as a function. [188:19] So first we have a try block and this is [188:22] the happy puff which is we are trying to [188:24] publish the message to this Kafka [188:26] cluster but in case it fails it means [188:29] that this cluster is down. So we need to [188:32] have some roll back mechanism for that. [188:35] So we have the catch block for that [188:36] situation with Kafka error. So first we [188:40] will persist this error to our logging [188:42] system to then identify why this message [188:44] buffer failed. But then we will try to [188:47] push the message to the local message [188:49] buffer instead. This is just one [188:51] approach that we can use. But then this [188:54] buffer is limited. So we can store up to [188:57] let's say 100,000 messages in the [189:00] buffer. In case that also fails, we [189:02] don't fail the whole operation. So we'll [189:04] just start dropping the oldest ones from [189:07] the buffer instead. And then we will [189:09] need to add some interval for waiting [189:12] for the Kafka to recover. And once it [189:14] recovers, we will try to publish it [189:16] again to the Kafka. And this time needs [189:19] to be the exact time that takes us to [189:22] spin up a new Kafka. So if this is [189:26] entirely like destroyed or deleted, we [189:29] need to spin up a new Kafka. We need to [189:32] know exactly how much this takes. Let's [189:33] say this takes five, like not five even. [189:36] It needs to take less than five. So [189:38] let's say 2 seconds to load which is [189:41] 2,000 milliseconds. So we know that this [189:45] number is greater than the time we need [189:47] to spin up a new Kafka. So we are just [189:50] waiting for that time in the memory in [189:54] the chat server and then once this is [189:56] back online with the new Kafka we are [189:59] publishing this to that one instead of [190:01] losing the whole memory of messages. [190:04] So this prevents the memory loss in case [190:07] the Kafka goes down. But what if this [190:09] server even like both fail at the same [190:12] time like we are keeping this in the [190:14] memory but then Kafka failed and now [190:17] this server also fails. [190:20] This is where we can also store these [190:22] files into the server's local disk. So [190:25] append only the log file and whenever [190:28] the chat server also recovers we will [190:31] replay it from the disk this log file [190:34] and then we will delete the log after we [190:36] successfully publish this to the Kafka [190:39] cluster. [190:41] So in summary, this is how it will look [190:43] like. If Kafka is completely destroyed, [190:45] we are returning the messages from the [190:48] message storage service and then we [190:51] request messages since last id. We will [190:55] query the processed messages from [190:58] database and then we will check which [191:00] are the messages that are still not [191:02] published to the database that we still [191:04] have in the memory. So this is where we [191:07] will republish the missing messages into [191:09] the Kafka once it is back online and [191:13] once we write this all to the persistent [191:15] storage like a database this is where we [191:18] will resume back to the normal [191:19] consumption. So you need to have such [191:22] mechanisms to ensure that you are [191:24] avoiding data loss or at least [191:27] minimizing the data loss that can happen [191:29] in case many parts of the system fail [191:32] concurrently. And lastly, you can expect [191:34] to also talk about some feature control [191:37] in such big systems because you don't [191:40] usually release new features for all [191:42] users. You usually do gradual rollout. [191:45] In such cases, we usually use feature [191:47] flux store. All of the cloud providers [191:50] come with such solutions to provide you [191:52] with feature flux. For example, on [191:54] Azure, you have such system. for AWS you [191:57] have it and you even have external tools [191:59] that you can use like launch darkly is [192:02] one of the popular ones that we used for [192:04] example in the previous company. So [192:07] these are just tools that allow you to [192:09] set key value pairs. It can be like is [192:12] new theme enabled and it will be true or [192:15] false and the true or false is also [192:18] controlled based on the environment. So [192:20] it can be for example true for staging [192:22] but it can still be false for [192:24] production. So this way, let's say [192:26] developers are testing on staging, they [192:28] see the new feature, but users on [192:31] production don't see it. And when you [192:33] decide to also roll out it for [192:35] production, you can still set it to true [192:37] for only 20% of users. So only 20% of [192:41] users will start this beta feature [192:44] testing it. And then if it goes well, [192:46] you don't notice any failure logs. That [192:49] is when you can enable it for everyone. [192:52] So in this case the chat server will [192:54] check the supported features from this [192:56] feature flag store which can be large [192:58] dartly it can be the cloud provider's [193:00] native solution and then we will use [193:03] gradual enablement per user or per group [193:06] for example per user will be that 20% [193:08] users have access to this and then we [193:11] will return the enabled features from [193:14] this feature flag store to the chat [193:16] server and by having this for example [193:18] the chat server can send messages with [193:21] new features to the client apps that are [193:24] above this version and strip the [193:27] unsupported fields or features for the [193:29] client apps that are below 2.2 or some [193:33] version that you set. So that is for [193:36] example discussing the complete [193:38] architecture. Now we can also talk about [193:41] how usually to approach this like a [193:44] framework. So as you can see we are [193:45] starting simple. You usually start with [193:48] the clarifying requirements and then [193:51] calculating the estimates because this [193:53] is important for you to estimate what [193:56] are the storage solutions you are going [193:58] to use, what are the services are you [194:00] going to introduce a Kafka broker or [194:03] maybe it's not needed. So you don't jump [194:05] straight into radius popsup or database [194:08] sharding. You usually begin with the [194:10] high level simple architecture with just [194:13] thousands of users. Then you build it [194:16] naturally by talking with the [194:17] interviewer and then you identify [194:19] bottlenecks like you could see here [194:22] where for example the Kafka message can [194:24] break. We can have rate limiting issues. [194:27] We can have database that goes down. So [194:30] you need to start talking about this as [194:32] well. So it's not something that you can [194:34] usually memorize. you need to just uh [194:37] know the steps that are there and then [194:40] for each system you need to go with the [194:43] flow and then discuss the trade-offs. So [194:46] understanding trade-offs is another [194:48] important piece of this. Every decision [194:50] has costs. For example, if you are using [194:53] websockets, it's not all green because [194:55] for example, they use more memory than [194:58] HTTP but they provide real-time delivery [195:00] and for example for messaging app that [195:03] tradeoff is worth it. For some other [195:05] situation, the trade-off might not be [195:07] worth it. Maybe you will go with [195:09] something like just a polling option [195:11] instead of just a web socket. So you'll [195:13] need to also be ready to discuss the [195:15] back of the envelope calculations before [195:18] getting to this. So be ready to [195:21] calculate these numbers. Maybe do some [195:23] examples and then check whether you [195:26] calculated it properly or not. And then [195:29] once you have something stable, then [195:31] only you will start thinking about [195:33] failures. You don't need to rush into [195:35] that because you still didn't decide on [195:39] the main components and haven't designed [195:42] the full system. You don't want to get [195:44] into the failures before that. And don't [195:46] worry if this was too much in detail. [195:48] You are not expected to design a perfect [195:51] system in one or two hours. That's not [195:54] the goal in the interviews. They will [195:56] usually want to see how you think, how [195:58] you discuss tradeoffs, or are you just [196:01] picking something because you like it. [196:03] So you need to show your thinking [196:05] process. Say this out loud whenever you [196:07] are designing something and that will [196:09] also help you to get direction if you're [196:11] going to the wrong prof. The interviewer [196:13] will correct you and they will give you [196:16] hints on choosing the right solution for [196:19] the trade-off that you're thinking [196:20] about. By the way, if you'd like to [196:22] learn all these concepts that I'm using [196:24] for designing these systems from [196:26] scratch, I have a whole system design [196:28] guide covering all these concepts that [196:30] you need for system design interviews on [196:33] my YouTube channel. You can check out [196:35] the first link in the description to [196:37] find the link to my channel. And the [196:40] first pinned video in my channel is that [196:42] system design guide explaining all these [196:44] concepts you need for passing these [196:46] system design interviews.