Are abstractions always a good thing?

... in many cases they probably are. Cloud providers are abstracting more and more tasks. Provisioning and administration of servers and much more.

It's been a while since our last blog post and since then we have been doing a lot of research about the eligibility of a Raspberry Pi cluster and where it has its purpose in this world.

The reason for this discussion comes from a lot of questions asked when talking about our project.

What can you use a Raspberry Pi cluster for? Can I use it for testing isolated services? Can I use it as a staging environment?

That is a really good questions - and a questions that we set out to discover doing our master's thesis project. Before going further let's roll back a second and discuss our main problem formulation and discuss where we, just a few months ago, saw the eligibility of our Raspberry Pi cluster. Back then, our main focus was 'Resilient cloud computing in a controlled environment' - and we wanted to run a cloud infrastructure on Raspberry Pi clusters with the purpose of testing small microservice architectures and different communication protocols with focus on resiliency. We wanted to compare architectures that leverages some of the patterns described by Michael T. Nygard in his book 'Release It!" and actually do some academic research experiments in this controlled environment to verify the effect of applying stability patterns like the circuit breaker. Basically, the focus was of building a test bench for verifying microservices architectures with regards to resilience in a controlled environment including the visualization of how the cluster worked. This is still on the roadmap and one of our main goals of our thesis, however the Raspberry Cluster added more value in the visualization and understanding aspect than first anticipated.

Back, when on-premise hosting was a thing, and the cloud providers didn't offer 'hardware' in a pay-as-you-go fashion, it was fairly easy to understand the effects of e.g. hardware failure, loss of network connection, etc. It was there in front of you, and if you pulled the cable the server would be down. Back then, many built applications as monoliths, hopefully modularized or split in components, and the entire application was deployed on a server. If you had a lot of traffic and load you could scale out horizontally by adding more hardware that would run the replicated monolith with a load balancer in front. The figure below describes this easily understandable and graspable concept, the real-world and our mental models were in sync. This is illustrated below:

Before Cloud computing. Thanks to Designed by Freepik and Users Insights for graphical elements.

Today, we live in a world where we can provision 1000 machines with the click of a button or enter a command in the terminal. Applications are being sliced and split in to many services communicating over network and deployed to a datacenter that uses some form of best-fit scheduling algorithm for deciding where to run your application. Pure awesomeness!

However, as we are within academia and currently are writing our master's thesis in Computer Engineering, we would like to raise the question of 'Is Academia embracing this new style of computing enough?' - No doubt, there are universities out there that do this on a large scale and have a big focus on this topic, but we imagine there exist many who don't. Out hypothesis is that many developers still have the mental model shown below of how cloud computing and server provisioning works. This misconception of the real world will probably lead to problems when deploying your software in production. Now we don't have the cosy environment of method invocations and function calls but instead have to communicate via the non-deterministic network. The illustration below illustrates this misconception.

Our hypothesis about the view of cloud computing. Thanks to Designed by Freepik and Users Insights for graphical elements.

Now back to the Raspberry Pi cluster, as mentioned, the tangibility of the hardware in front of you back in the old days, gave you a good understanding of what was going on. We think that this is were a Raspberry Pi cluster can add some value in the teaching and understanding of Cloud Computing and distributed and modularized architectures. It seems that the popularity of this architectural style is gaining more and more traction, and thereby a need for developers with skills and understanding of the operational aspect of developing and deploying an application. This is also confirmed if looking for job-postings tagged with DevOps. The Raspberry Pi cluster will act as a mediator object between our mental models of the real world and the actual real world by visualizing and highlighting failures that can and will happen in the real world, such as power outage and network failures. Furthermore the cluster management, e.g. Kubernetes can show how the entire cluster handles pulling the plug of a specific node by rescheduling it to another available node. The illustration below highlights the use of a Raspberry Pi cluster as a mediator object.

The use of a Raspberry Pi cluster as a mediating learning object. Thanks to Designed by Freepik and Users Insights for graphical elements.

We don't think abstractions are bad, and especially not within Cloud Computing, it is truly awesome that we have come so far, and can get resources as we go. But we also think that there may be a gap between academia and the real world in the understanding and prioritizing of these topics.

This is meant as an open question, so please do not hesitate to comment. Are abstractions always a good thing? And do you see the value in the tangibility that a Raspberry Pi cluster, running e.g. Kubernetes, can add for learning about Cloud Computing and Infrastructure?

Over and out!