I use draw.io for these kind of diagrams. It has symbols for most of the cloud services built in. It also maintains the arrow connections when we move around the elements. Very handy!
+1 for Excalidraw, especially regarding the "multiplayer" aspect.
In general, if sketching it out, whether alone or with multiple people, using something simple I can click, drag & label is more than enough and helps me keep the flow. (I would stick to actually drawing - either on a whiteboard or something digital - but my handwriting is absolutely atrocious)
As soon as I'm done sketching and move over to actually documenting a concrete solution, something code-based like Mermaid - as you said - is much nicer. But both are entirely different processes for me.
In Mermaid, while I like it a lot, you lose control of layout positioning. Does the auto-layout position large diagrams (many shapes + connectors) appropriately?
The Excalidraw export format is JSON, so you might use that in source control just as well. Or include the format in PNG or SVG export (which draw.io supports also).
The way most people use draw.io is using cloud storage integration like Google drive, hence versioning is not possible. So far I never seen people put the draw.io produced file to Git.
I'm using PlantUML in some of my projects, it solved the versioning issue. However it's lacking of lots of draw.io or even inkscape editor capability.
I do. What I like the most is the ability to include the diagram source into PNG metadata. The viewers don't need any renderer, yet it can be readily edited.
Last release was 2014. I remember liking dia, but I forget...it either crashed or didnt have all the aws icons I wanted when I used it last 5 years ago, and there hasn't been a release since.
That person’s entire criticism of Dia is “it’s old”. That’s it; not even “the program, when I run it, looks old”! They merely criticize the web page for having old screenshots and support for old operating systems on it.
As the README mentions "cloud-based" platforms, it should be noted that Diagrams.net (previously Draw.io) is available as an offline application. Inkscape is missing connection points and routing controls to make it easy to do diagrams. Also, Diagrams.net can export to SVG or PNG while keeping the original diagram embedded. I use that instead of the specific format they use.
I tried to get diagrams to pass through the container ID to the exported SVG "g" to find it's some massive java thing that hasn't been added to in years. There's a lot of scope for semantic SVG we could use in other tools.
What I generally dislike about using graphics tools for diagrams is that they have no notion about the semantics of what is getting drawn. It’s just „pictures“.
In contrast to, for example, proper modeling tools like Sparx Enterprise Architect or Visual Paradigm.
It doesn’t really matter then if you’re using an online tool or a native one.
- Can check the textual description of the diagram into version control and track how it changes over time
- Can easily diff 2 versions of the same diagram
- Adding elements to a large existing diagram using WYSIWYG tools can be tricky (eg: you may have to move or reroute existing elements to make enough space)
Another cool idea is to create multiperspective diagrams with different views for different purposes (eg: code structure, deployment, network architecture, security):
- Clarity. By breaking up diagrams into perspectives, diagram authors can bring much more clarity to a system. Individual concerns can be spread out over many perspectives, allowing each the space it needs without interfering with others.
- Extensibility. Multiperspective diagrams share a model, so creating additional perspectives from an existing diagram is incredibly easy, far easier than starting from scratch each time.
- Maintainability. A corollary to the above, individual perspectives can be modified without affecting the others. Diagrams that are easy to maintain get maintained, while diagrams that aren’t fall out of date quickly. Multiperspective diagrams greatly help in the fight to keep documentation up-to-date.
I would suggest that if your architecture diagrams are a bunch of icons provided by AWS/Azure/GCP with lines pointing at each other... you are doing it wrong.
The 'what does this element do for my system' is vastly more important than the 'which in vogue offering from my cloud provider implements it'.
I can't say using vendor specific icons is wrong, clearly a lot of people need just that, but I always miss more generic architectural components in these diagramming tools.
That's the problem I have with the `diagrams` Python package (https://pypi.org/project/diagrams/), I find it extremely useful, but the "Generic" section doesn't even include a "Queue" component, and the "OnPrem" section only includes specific queue products.
That Python package is exactly how I like to create diagrams, so I keep using it, but I always have to add my own badly-designed custom icons to it, which look so bad that I wouldn't dare send a PR to add them upstream.
What a useless crap. It’s an over engineered documentation framework focusing on all the wrong things so some agile consultant can sell more cool aid. I don’t trust anything that is designed someone who last did real projects in the 2000’s OOP Java enterprise craze.
This is the important bit. C4 in its entirety isn't useful to many orgs and teams, and can be unnecessary overkill. You might end up picking 1 or 2 layers for yourself.
You don't have to stick to their software either, use DrawIO if that's what you prefer, but understand the main concepts that C4 is trying to impart about the separation, what goes on the arrows, and how useless icons can be.
> Take the parts that work for your team and scrap the ones that don't.
Precisely. Any time someone utters the descriptor "best practice", at best, you do yourself a favor by examining it with a critical eye and judge whether it is a good practice to follow (and a good fit for your project) or not. At worst, you beat back the consultants framing it as "the one and only way to do things (with the implication that everything else is wrong)."
So so much agree with this. People who think in terms of OOP and design patterns (i.e. bandaids) will think "how else are you supposed to model solutions"?
But yes, this is fundamentally an OOP thing, and especially the component layer is very much just fantasy. In no real OOP software do these meat reusable components not have super strong binding to each other. Yeah you may have the SignIn controller and the ResetPassword controller both talking to the SecurityComponent, but in the end that's guaranteed to be implemented in such a way that you cant use any of them without all the others connected, and its likely that testing this means writing a mock for each of them that is so complex it may as well have its own tests.
They are thinking of their problem, and building their solution entirely out of blocks made from the words that make up the problem. "We need users to be able to sign in and reset their password" becomes SignIn component and ResetPassword component, when really this is just a matter of changing a hash and possibly some encryption keys, and could just be a handful of validate(), reset() et al functions on the security module.
I read this article and I'm left with the feeling that my entire application has no layer 3 i.e. the component layer. No single (useful) part of the application can be said to have a well defined interface or to be self contained. The strange thing is that I'm not actually that unhappy about that. I'm now sitting here thinking whether I should be more worried.
From my experience what you are experiencing is a case of "the current architectural best practices disagree with what we are doing successfully" which many organizations and Senior Engineers interpret as reason enough to introduce change. You will probably receive comments along those lines, be in arguments around this.
You could also actively ignore this for half a decade and see if the tides are changing again. I can almost taste a push towards "lower amounts of abstraction levels and system boundaries lead to smaller stack traces which have xyz advantages" on the horizon. Hang in there, solve the problems you have.
It kinda depends on the scale I guess? At some point in the growth of on application, pulling out the Big Tools can be helpful to get your application out of a local maximum.
Not everything needs that level of design though. Just because architects do detailed FEM analyses for skyscraper design doesn't mean your garden shed needs the same level of attention.
Depends on your application - A game engine won't break down into this 4 layer model, but lots of enterprise applications (or ecosystems) will.
If an application continues to grow, and the team building it grows, it will eventually need to be broken apart into smaller independent blocks to make it practical for a large team to work on it and maintain it. In a smaller codebase with fewer engineers, you can make different choices.
Honestly, this may be because you write procedural/functional code, aka simple code, and not enterprise (tm) (R) OOP "I've memorized all 150 design patterns" code.
A lot of these components are strongly linked, and will hold mutable references to each other, which is often pretty terrible.
In procedural code world, everything is connected to each other, but not strongly, and there are very few mutable references.
Great article. So you could say the current version is just a bunch of icons. Next step is adding meta info to the recourses. I'm currently working on a auto import funcion like you can use in Cloudcraft.
Depends on the target audience. Mostly the audience is hand wavy types that just want to point at a beautiful thing in a slide to point out "we have stuff with icons that you might know! Look at how beautiful this is. It almost looks like we know what we are doing!". Exaggerating here but anything that fits on a single slide is completely useless to an expert. Not worth drawing unless the goal is to impress non experts like that. It's a marketing tool in that case. The level of detail required for this is "a power point slide".
In companies/organizations that are large enough you always get a certain amount of project bureaucracy, ass coverage, and other documentation that doesn't technically do anything more than impress easily impressible types. But it's rarely actually useful. I usually just go straight to the code repositories and ignore things like wikis and other crap. Complete waste of time usually. Show me what you really have. I'll figure it out in no time.
Two mistakes people make with these tools:
1. Spending lots of time using them believing that the output is actually useful/valuable to engineers. Your time is more valuable. You should be doing more productive things. Minimize time you spend on drawing pretty pictures. They didn't hire you for your graphical design skills. This is of course subjective and context dependent. Sometimes it's just required to have diagrams. E.g. operations people like having good documentation just as a way to ensure that they can follow strict processes and make good decisions as to what is in their scope and what needs escalating. There's also a certain amount of impressing the customer, senior management, or other stake holders that "we have stuff". The bigger the company, the bigger the need for project bureaucracy like this. But it is bureaucracy and you want to be efficient with it. Quick and easy.
2. Believing that these are design tools. They are not. They are documentation tools. You use them after you build the thing. Before you build the thing, you use something like a white board. Or pen and paper. A napkin. Anything fast and easy that doesn't slow you down. It's transient stuff and when you start building the thing you'll realize half a dozen topics you did not take into account. So, other than as a record of all the design mistakes you are making, such diagrams have no long term value. You document the solution after you found, solved, implemented, and tested all the design problems. Not before.
Friendly reminder that the places where you are least likely to find diagrams:
- Any kind of large open source project. Or any open source project really. Just not a thing. The bigger and more complicated they are, the less likely it is to have diagrams. Reason: they are redundant and absolutely nobody volunteers to sit down and do them. Just ask yourself: "what would Linus Torvalds say when asked to provide diagrams for the linux kernel". I imagine a fair bit of cursing would happen.
- Especially open source projects that are about producing diagram tools. I personally find this highly ironic. People won't eat their own dog food when the dog food is diagramming tools. Diagramming tools are something you build for others to use. Go look for it on Github if you don't believe it. All you will find is toy examples but nothing actually documenting these tools in any level of detail.
- Small startups or other companies that are highly innovative and have a fast pace of change happening all the time. Reason: people have better things to do than mess around with diagrams. The rainy afternoon where you really have nothing more valuable than messing around with boxes and arrows to do never really happen.
Where do you find people messing around with diagrams? Bloated engineering teams in corporate situations. The more boring the company, the more useless types they employ, the more diagrams you will find. People insisting to each other that "somebody" (not them) should do a diagram. I usually just bounce the question when it comes up. "Great idea, when can you have it done?" Usually the implied suggestion is that I should sit down and waste my time doing a diagram for them that they will never even look at for more than a few seconds. It's write only documentation.
I would have to disagree. If you have a larger system, then onboarding, design discussions, and communicating changes will be more efficient if you have a consistent language and up-to-date conceptual understanding of the system compared to having to perform code archaeology and reverse-engineer the previous maintainer's intent each time you want to change something.
That's what I would call aspirational usage. Ask yourself is your best use as a 200$ and hour senior architect to be writing onboarding documentation?
I've been in plenty of companies with diagrams and universally they were incomplete, not that informative or helpful, typically outdated, and kind of obvious. That's because they are typically rush jobs exactly because the best people that could be doing these diagrams have way more important/urgent/valuable things to do.
I found a "hack" for diagrams that need to work with both white and black backgrounds was to make all your lines gray. And then when you want vary the intensity of the line you adjust the opacity
It cuts your effective dynamic range in half, but works pretty well. And adjusting opacity is actually somewhat easier than adjusting color hex values
I haven't tried it with colors though - but I expect it'd work as well
In contrast, forcing a white background on people running dark mode is kinda ugly in my opinion as you end up with a border/cutout and it doesn't flow with the rest of the content as well
I am surprised they used a PNG for the preview instead of an SVG file. I know you can embed JavaScript in SVG files, I wonder if it could react to the `prefers-color-scheme` CSS feature of the page it's embedded in.
As someone working indirectly on one of those “crap online editors”, I do wonder what benefits Inkscape has over other options for what is essentially a graph that may very well need an automatic layout. Someone else mentioned diagrams.net as well.
i use yEd from yWorks for all my diagramming.
it is based on GraphML, can import nodes/vertices from excel/XML, it has good automatic layouts for large diagrams, and can output to GraphML, SVG, JPG, and many other formats.
yEd and GraphML is very versatile indeed. Years ago I worked on a quite big mobile game project, we used graphML to be the main quest / level / story format of the game and naturally yEd was the editor of choice. Pretty scrappy but it worked, somehow.
I don't understand why more people don't care about round-tripping. Yes it was over ambitious and far too heavyweight in the 90s, but today you should be able to generate OpenAPI and its data and component equivalents from the visual graph, or inversely import your graph, then verify / update either "side" at any time. I've mentioned this before, but TogetherJ was the best example I saw, lightweight and worked well for visual/code round tripping when it was all in a predictable Java environment. Somehow we went from overdone to decades of "works for me" mucky muck.
If you have to annotate each box you might as well ditch the image. In which case you can describe what it does instead of just naming the service it uses. And use Mermaid markup for the diagrams; GitHub supports it!
I build a lot of diagrams for external audiences and I've found both Excalidraw and LucidChart to be really useful. I use Excalidraw for live diagramming and use LucidChart for our external docs or more formal presentations.
Most diagramming solutions are like freehand drawing tools. Besides text-based diagrammers (Mermaid, PlantUML, D2Lang) what are good diagramming components for integration in a web front-end that help with strict, validated diagramming? Restricted tool pallet shapes and commands. I.e. defining that a "Concept Map" only has Concepts and Relationships, where the latter must be connecting 2 concepts for the diagram to be valid. Or an "Event Storming" diagram that only has the appropriate sticky note types that must be layed out appropriately on a timeline. Bonus points for supporting collaborative editing.
React Flow [0] until now is the component that's closest to this. Others, Excalidraw [1] and Tldraw [2] may be integrated to support this, with some additional effort.
