CRASH COURSE: Code your Design Diagrams with Python!

Design diagrams play a crucial role in conveying complex ideas, architectures, and systems visually and intuitively. However, for many engineers like me, the process of creating these diagrams can be a painstaking task. Spending hours piecing together elements in diagram editors, tweaking shapes and connections, only to end up with a subpar representation of the vision — it can be a frustrating experience. But fear not, my fellow engineers! In this article, I am thrilled to share with you a game-changing solution: the power of coding your design diagrams using Python.

Prerequisites

To embark on this exciting journey of coding your design diagrams, you’ll need just a few simple prerequisites.

Python: Of course, we will need this. This versatile programming language will serve as our trusty companion throughout the process. There are a lot of detailed youtube videos on how to install Python so I am not walking through that here. However, do ensure to install version 3.7 or above.

The Python Library “diagrams”: This is the magic library that provides a set of intuitive and expressive tools specifically designed for generating professional-looking diagrams programmatically. With diagrams, you’ll have the power to create a wide range of diagrams effortlessly.

To install Diagrams, simply:

pip install diagrams 

You can find more information here: https://diagrams.mingrammer.com/docs/getting-started/installation

Graphics Tool Graphviz: Graphviz is a powerful open-source graph visualization software. Diagrams need this tool to render the diagrams so this is a mandatory installation.

All macOS users can download Graphviz by typing brew install graphviz if they're using Homebrew. Similarly, Windows users with Chocolatey installed can run choco install graphviz. There are other ways of doing this here: https://graphviz.org/download/

The Crash Course

Now that we are done with the prerequisites, let’s take a quick crash course and I promise, by the end, you will know how to code your diagrams!

The Basics

The diagrams library consists of four basic things:

Diagram: This is the master context under which your diagram will be created. Think of this as a bunch of configurations for your diagram. But, most of the time you will just need it to give your diagram a name. Here is an example:

from diagrams import Diagram 
from diagrams.aws.compute import EC2 

with Diagram(“Simple Diagram”): 
  EC2(“my server”)

Here Simple Diagram is the name of the diagram. Look at how we created a component under the Diagram context. Yup, that’s all that is required. All the components will be created like that. You can find more about specific configurations available for Diagram here: https://diagrams.mingrammer.com/docs/guides/diagram

But what are these components that we will add under diagrams? That brings us to our next item.

Nodes: Nodes are the building blocks of the diagram. A node can be anything, an EC2 instance, a Lambda function, an SQS queue, or a Jenkins job to say a few. All these nodes are available under their specific packages under diagrams and can be imported when required. In the above example, we imported EC2 from diagrams.aws.compute

All these nodes can be assigned to variables as well. Here is a code snippet:

from diagrams import Diagram
from diagrams.aws.storage import S3
from diagrams.onprem.compute import Server
from diagrams.aws.analytics import Redshift

with Diagram("My Design Diagram") :
    DataService = Server("Data Service")

    S3Storage = S3("S3 Bucket")

    MasterDB = Redshift("Redshift DW")

The above code will generate this:

You can know more about nodes here: https://diagrams.mingrammer.com/docs/guides/node

You don’t have to remember the import commands for your nodes. You can find the whole list under documentation segregated by services. Here are all the on-prem ones: https://diagrams.mingrammer.com/docs/nodes/onprem

Edge: An edge is nothing but directed arrows that connect two or more nodes. You can set the colour, label, and style of your edge that connects your nodes. Let’s expand on our previous example

from diagrams import Diagram, Edge
from diagrams.aws.storage import S3
from diagrams.onprem.compute import Server
from diagrams.aws.analytics import Redshift

with Diagram("My Design Diagram") :
    DataService = Server("Data Service")

    S3Storage = S3("S3 Bucket")

    MasterDB = Redshift("Redshift DW")
    
    DataService >> Edge(label="Sending data to S3") >> S3Storage 
    S3Storage >> Edge(label="Copying data to Redshift") >> MasterDB

The above image was generated with our expanded code. Look how we imported Edge from diagrams. Also, observe how we are defining the label of an Edge. You can learn more about Edge properties here: https://diagrams.mingrammer.com/docs/guides/edge

But are you curious about why did we use “>>” or the bitwise operator? We will talk about it soon!

Cluster: Finally, this is the last basic concept. A cluster is nothing but a set of nodes that we want together. A cluster is defined using the cluster context and any node defined under that context is grouped together. You can also give it a name to define your cluster group. You can also nest clusters under clusters. Let’s expand our code further to understand this.

from diagrams import Diagram, Edge, Cluster
from diagrams.aws.storage import S3
from diagrams.onprem.compute import Server
from diagrams.aws.analytics import Redshift

with Diagram("My Design Diagram") :
    DataService = Server("Data Service")

    with Cluster('S3 Buckets') :
        S3Storage = [S3(f"S3 Bucket_{n}") for n in range(1,4)]

    MasterDB = Redshift("Redshift DW")

    DataService >> Edge(label="Sending data to S3") >> S3Storage 
    S3Storage >> Edge(label="Copying data to Redshift") >> MasterDB

This is what the code generated now:

Now observe how we imported Cluster from diagrams. Furthermore, we defined a set of S3 Bucket nodes using a for loop under the Cluster context and it all grouped together under that blue box with the label S3 Buckets. Also, we directed our Edge from a node to a list of nodes and from the list to the Redshift node and arrows got created for all the nodes in that list. Nifty, isn’t it? You can find more about clusters here: https://diagrams.mingrammer.com/docs/guides/cluster

The relationships

Now that we have established the basics, let’s talk about the next thing relationships between nodes. This can be defined by two things:

Directed Relationship: This means a node is pointing towards the next one. Without using edge, you can just use the “>>” operator for this and an arrow will be created between your nodes.

Synergic Relationship: This is to define nodes that work together as a single unit. You can define this by using the “-” operator between nodes.

Let’s go back to our earlier code and make some changes:

from diagrams import Diagram, Edge, Cluster
from diagrams.aws.storage import S3
from diagrams.onprem.compute import Server
from diagrams.aws.analytics import Redshift
from diagrams.aws.compute import LambdaFunction as Lambda

with Diagram("My Design Diagram") :
    DataService = Server("Data Service")

    with Cluster('S3 Buckets') :
        S3Storage = [S3(f"S3 Bucket_{n}") for n in range(1,4)]

    with Cluster('Lambda Process'):
        Lambdas = [Lambda(f"LambdaFunction_{n}") for n in range(1,4)]

        for n in range(0,2):
            Lambdas[n] - Lambdas[n+1]



    MasterDB = Redshift("Redshift DW")

    DataService >> Edge(label="Sending data to S3") >> S3Storage
    S3Storage >> Edge(label="Copying data to Redshift") >> MasterDB
    MasterDB >> Lambdas

And we got this by running the above:

Now, look at the above code. We defined the relationship between the lambda functions using the “-” operator and we can see that now they are in a synergic relationship and work together inside the cluster. Additionally, we just used the “>>” operator without any Edge and you can see how the Redshift DW directed to the Lambda functions without any text defining a directed relationship.

Formatting

To beautify your diagrams you will certainly need some formatting. Diagrams support all the configurations of Graphviz. Formatting can be done using the three parametersgraph_attr, node_attr and edge_attr. Here is a reference link to know what attributes are supported for formatting.

Here is an example code from the diagrams documentation for the same:

from diagrams import Diagram
from diagrams.aws.compute import EC2

graph_attr = {
    "fontsize": "45",
    "bgcolor": "transparent"
}

with Diagram("Simple Diagram", graph_attr=graph_attr):
    EC2("web")

Conclusion

I hope you’ve enjoyed delving into the world of coding design diagrams. The simplicity and power of the diagrams library, coupled with the graphics prowess of Graphviz, open up a whole new realm of possibilities for engineers like us.