Blog

In my journey exploring Easy Conceptual Modelling, I want to understand how AI can help. In this post I’ll look into whether a general-purpose LLM can translate a simple graph into its textual representation. Here my sample input graph.

The answer is – spoiler alert – yes. On its initial response, one such LLM – Claude 3.5 Sonnet – has produced a textual script with only one essential semantic error. It took three more prompt rounds for the perfect output below.

OK, nothing groundbreaking, feel free to skip and leave unless you’re interested in the details.

Why I am looking into this?

I believe that AI and LLMs will relieve engineers of repetitive and time-consuming tasks, allowing engineers to focus on conceptual work, like Sensemaking and Conceptual Modelling, which are the focus of my blog.

Going one step further, I wanted to check how AI can save conceptual modelers from tedious tasks – here translating a graph into its textual representation. Without claiming this is the best way – on the contrary, there are likely more efficient methods – I found this would be a nice way to start evaluating AI.

In the rest of the post

I will:

• introduce the Graphviz graph visualization software;
• explain the benchmarking procedure
• show the result for a few standard LLMs
• and conclude with what we’ve learned

What is Graphviz?

Graphviz is an open source graph visualization software. See https://graphviz.org/. A graph, like the ones shown above, has a set of nodes, which may be grouped into subgraphs, both of them may be linked by edges. DOT is the plain text graph description language that serves as input for Graphviz. Graphviz renders DOT text into visual diagrams. Below an example Graphviz DOT text.
Apart from my hand drawing, all graph images in this post have been rendered by Graphviz.

How to see the Graphviz scripts?

On my blog’s version of this post, I have included the Graphviz script as HTML ‘title’ attribute for each image, which is you can view as follows:

• On Android Chrome and Firefox, a long press on the image displays a pop up with a partial ‘title’ attribute. On Chrome, tapping the attribute again reveals the full text.
• On both Firefox and Chrome on my Linux laptop the ‘Title attribute’ pops up when I hover my mouse on the picture.

The LinkedIn version of this article may not offer this possibility.

What is AI

AI stands for Artificial Intelligence. Large Language Models (LLMs) are AIs that can process and generate text. I’ll evaluate the following LLMs:

• Claude 3.5 Sonnet (free version)
• ChatGPT GPT-4o (free version – short name for GPT-4 Omni)
• DeepSeek V3 (free version)
• Mistral AI – Le Chat (free version)
• Perplexity (payed subscription) with various options.

I ran these queries between 6 and 10 February 2025.

Benchmark procedure

I’ll first phrase a prompt requesting the LLM to convert a graphical representation of a graph into the Graphviz DOT syntax.

Then, for each LLM, starting from a blank session:

1. I’ll prompt the LLM as above;
2. I’ll render the output script with GraphvizOnline;
3. I’ll upload the resulting image on this post (see above How to see the Graphviz scripts).
4. I’ll assess briefly the result;
5. I may attempt to fix the issues;

Until I conclude the article.

Assessment Criteria

I’ll assess the quality of the conversion if the rendered graph:

1. is semantically correct – that is, the nodes, subgraphs, edges; their labels; and the direction, source & target of the edges match the source graph.
   Here I’ll distinguish between:
   • essential errors – the transcription of the graph into the meaningful list of elements listed above has errors;
   • accidental errors – the translation of the list above into Graphviz syntax has errors.
2. is readable (by a human)
3. is understandable (by a human)
4. reasonably respects the representation conventions of the input image (for example arrows going upwards).

The Target

Here is the target graph in Graphviz DOT syntax.
With “Target” I mean the version, written by me manually, which I consider would be an excellent result if it would be generated by an LLM.

digraph WhySenseMaking {
  edge[dir=back];
  compound=true;
  subgraph cluster_environment {
    World -> Machine -> User[label="interacts with" ];
    World -> User[label="is in" ];
    World -> Goal[label="in" ];
    Goal -> User [label="has" ];
    label = "The Environment";
  }
  subgraph cluster_Stakeholders {
    Sensemaking -> Engineer [label="needs" ];
    label = "The Team";
  }
  Machine -> Engineer [label="builds"];
  World -> Sensemaking [label="of" ltail= "cluster_environment"];
}

Which GraphvizOnline renders as:

PromptA

My initial prompt (PromptA) was:

Attached the image of a graph. Please convert this image into Graphviz DOT syntax.

Claude 3.5 Sonnet – PromptA

Result:

Assessment:

• Four semantic errors:
  • 3 essential errors: wrong edge directions
  • 1 accidental error: The edge “Sensemaking of” wrongly targets “Machine” instead of the subgraph “The Environment”.
• readable, understandable, respect of the conventions: not perfect. Some wriggling edges.

Even if, for a first simple prompt, I find this first result surprisingly good, I am not satisfied and I suspect my prompt needs more guidance, so I rework it.

PromptB

In this new prompt I am much more explicit to the LLM about each step it has to perform – requesting it lists, as intermediate steps, all graph elements, especially the edges, their labels and directions.

Claude 3.5 Sonnet – PromptB

Here the result of PromptB (see above How to see the Graphviz scripts?):

Assessment:

• Two semantic errors:
  • 1 essential error – that is, one error in the graph analysis: the “is in” edge between User and World is missing.
    I find this is good and very encouraging.
  • 1 accidental error – that is, one transcription error into the Graphviz syntax;
    • The “of” edge from “Sensemaking” wrongly targets a node called “cluster_environment”. This is due to Graphviz’s complex syntax for pointing to subgraphs. In my tests, no LLM has managed to succeed this syntax.
• Readable, understandable, respect of the conventions: Not yet OK. I want my vertical bottom to top input view. And the arrows pointing up.

Fixing the errors

I’ll be here taking the LLM by the hand and point at the errors one by one, explicitly and step by step. For this script size It would be faster for me to do it by hand – the LLM is no longer helping.

Here the first correcting prompt:

Please fix the two following errors:
1rst fix: The syntax from the “of” edge from “Sensemaking” to the subgraph “The Environment” is wrong. The DOT syntax of Graphviz is somewhat peculiar here: you need to set a node as target, within the intended target subgraph. Here best node would be “World”. So the corresponding DOT script would be: /// Sensemaking -> World [label=”of”, lhead=cluster_environment]; ///

2nd fix: Please add the missing edge /// User -> World [label=”is in”]; ///

Below the result – the LLM has fixed the two semantic errors as prompted. No semantic errors:

I now want to improve the rendering – my target is to reproduce the input image. Here the new prompt:

Cool, this is now semantically perfect. Please do following decoration changes:

1. remove the /// rankdir=LR; /// statement since I want the diagram to show vertically;
2. now I’d like to vertically flip the orientation of the graph, so that the User and Engineer nodes are at the bottom, and all the arrows are pointing upwards. To achieve this result, please do the following:
   • First, you revert the direction of pointing of all arrows. To do this, please insert the directive /// edge[dir=back]; /// at the top of the DOT script;
   • Then, please revert the source and target elements of all edges. As example, /// User -> Goal [label=”has”]; /// shall become /// Goal -> User [label=”has”]; ///

Having the tool take care of reverting source & target on all edges saves significant time. The result is almost perfect:

Lastly, it became confused again with the cumbersome syntax for pointing at subgraphs AND with reverted edges.

New prompt:

One last bit, please replace “lhead” by “ltail” in the script line
/// World -> Sensemaking [label=”of”, lhead=cluster_environment]; ///

Assessment:

• zero semantic errors
• readable, understandable: perfect.
• respect of the conventions: excellent.
• But – I have needed 3 correcting prompt rounds

Here the source image for comparison:

Let’s move to next LLM.

Perplexity with GPT-4 Omni – PromptB

Assessment:

• One syntax error – noticeably, this is the single time. The digraph name “Graph” is invalid because it’s a reserved keyword in the language. Changing to name “G”, I can render the graph.
• 6 semantic errors – all with edges (as most of the case).

With too many errors, I move on to the next LLM.

Perplexity, Deepseek R1 – Prompt B

I count eight semantic errors, all involving edges.
Too many errors, I move on to next LLM.

GPT-4o – PromptB

From chatgpt.com, no paying subscription. GPT-4o means “GPT-4 Omni”.

Assessment:

I count about six semantic errors. I wonder if an LLM could be used to automate error-counting – perhaps another time.

Next day, same GPT-4o LLM, same PromptB:

Although the number of errors is about the same, the result is slightly better, as only one edge is missing – I believe. Many errors in edge direction.

DeepSeek V3 – PromptB

From chat.deepseek.com, non paying subscription.

Assessment:

Too many errors – I stopped counting.
For the first time the labels are wrong. And a node gets the name of an edge. This is far worse than previous LLMs.

Mistral AI Le Chat

From https://chat.mistral.ai, non paying subscription.

I counted nine semantic errors, which is poor.

Conclusion

The question was whether LLMs available today support the conversion of a simple graph into its textual representation.

In short, yes. Reasonably.

Here my learning:

• Some LLMs available today can perform the task, with the best model producing only one essential semantic error on the initial prompt (Claude – PromptB – Result1 above).
• Claude 3.5 Sonnet ranks best, ChatGPT 4o second. Some LLMs perform poorly.
• The prompt matters. A step-by-step approach improved the results.
• Most errors occur during semantic extraction of edges – particularly in determining their direction.
• Nodes, subgraphs, and labels were generally recognized accurately, except by DeepSeek V3.
• The translation of these elements into the Graphviz syntax works also almost perfectly.
• In other words, LLMs are freeing me from tedious transcription and syntax issues.
• This is a tiny step in a huge domain, there would be much more to explore, e.g. larger graphs and different diagrams.
• More efficient and reliable tools likely exist. One LLM suggested a multi-step process involving image processing, text recognition, and graph reconstruction, using Python libraries for OCR, node, and edge detection.
• I’m excited to explore future applications, such as using LLMs to generate conceptual models from existing knowledge in free text.

I look forward to your comments on the LinkedIn post. You can also find more information on my blog: https://recognizing.ch

#AI, #LLM, #Modelling, #Graphviz, #Engineering