Why Prompts Are the Future of Knowledge Graphs

I thought, briefly, of doing another big animated piece for this issue of The Cagle Report. Still, I suspect I’m suffering the after-effects of the latest round of Covid, and figured I’d got just enough energy to spend it typing. Warning: Covid and heart disease suck. I’d not recommend it.

The Problem with Query Languages

I’ve seen a couple of posts on the topic of prompts and their relationship to knowledge graphs, but I wanted to put my five cents in (was two cents, but inflation’s even hitting giving advice). There is a tendency that people here, upon hearing about prompts, to tend to dismiss them as not being real programming, that real programmers use SQL, or SPARQL, or R, or Python, or … fill in the blanks. Yet it is worth understanding that when every one of these languages came out, their inventors claimed that the language would make it much easier for non-programmers to query databases. For their time, they were considered “natural language” queries, and if Max, the business analyst, couldn’t understand how to use the language, he must be something of an idiot.

The reality is very different. To understand SQL, you need to intimately understand the table structure and the arcane art of joining tables across keys to create working datasets. An entire industry evolved in order to simplify the making of queries using this supposedly simple query language.

SPARQL isn’t much better – it necessitates that you have a pretty in-depth understanding of any number of different ontologies, predicate relationships, namespaces, and the understanding of how to work with sets of assertions as tuples, how to work with set operations, and how to transform these into and out of potentially uncountable forms. Given that the preferred mechanism for populating knowledge graphs seems to be slurping up Excel documents, this argues that the knowledge graph community hasn’t been all that much more successful at creating a reasonable standard for interaction than the SQL folk have.

Languages such as GraphQL are becoming more popular as they provide a way to do model discovery and data shaping. Still, even there, they are programmer tools: GraphiQL makes several assumptions about data, including that the consumer of that data will be a process-consuming JSON and that users understand the GraphQL process. These are not unrealistic expectations if you happen to be a developer, but if you aren’t, that JSON may as well be Linear B.

Moreover, once you have retrieved this content, there is a question about what you do with the resulting content. This expectation is one that developers do not necessarily even recognize as existing, as they typically have another processor that will take this content and map it to a UI or other service. These become invisible barriers because until now, the idea that people wouldn’t have a programmer in the middle likely never occurred to them.

Note: I will be talking primarily about SPARQL, but I want to emphasize that the following points apply to most current query/mutation/generation forms.

Understanding Prompts

When you get down to it, a prompt is a dialog for retrieving a data set. It can involve a large language model (LLM), but the prompt itself is a statement or set of statements indicating several key pieces of information:

• Context. What specific conditions exist that hold prior to processing the prompt?
• Type. What are the kinds of things you are looking for?
• Identity. Are there labels or identifiers that specifically identify an instance?
• Constraints and Validation. What relationships restrict the content being returned?
• Source. Where are you drawing information from?
• Order. What determines the priority of results?
• Partitioning. How many responses are returned?
• Persistence. How to provide a handle to a retrieved set?
• Transformation (Presentation and Generation). How is the information presented to the user?
• Garbage Collection. Once a conversation is concluded, how does the system dispose of intermediate products?

Some of these show up in a standard SPARQL query, though SPARQL QUERY by itself tends to be pretty anemic. Beyond some caching, most queries still focus on identifying the names of classes to retrieve content, and the context, source, presentation, and persistence are mostly handled outside UIX. However, once you start to incorporate SPARQL UPDATE to create (semi-) persistent graphs, you can build context, maintain source information, identify and apply relevant transformations to generate multiple presentation layers and maintain persistence. In other words, by taking advantage of the full capabilities of SPARQL, you can get much closer to developing interactive prompt systems.

The two areas that tend to be most complex from a prompt standpoint are parsing and presentation. Each of these is worth exploring in some detail.

Parsing Prompts

It is possible to use a machine learning approach to create a decent prompt engine, but for the most part, what such an algorithm will do will be to identify, for given nodes, the traversal paths for those nodes and edges and store them in an indexable form. This can even be simplified if you have SHACL shapes defined for your system, to use the SHACL shapes to determine the classes (or class-like structures) that underlie the model, generate relevant paths between such class nodes, and then use text searches to find labels or annotational data within respective instances to stabilize the paths then. For instance, a SHACL property shape includes both the sh:path and sh:target fields, which can construct an extended path when walked recursively, while sh:datatype can be used with sh:path to determine literals (leaves, in graph parlance). Finally, the sh:name parlance can be used to glean names of properties.

If you are working with SHACL, property order (via the sh:order property) can also be used as a proxy for weight or significance in interpreting the parsing of prepositions. Context clues can be gleaned from prepositions, especially ones that indicate complete or partial containership (a is in b, a is outside of b, etc.), but whether such context provides value beyond walking property paths is arguable.

Most prompts also work upon a principle of diminishing priority, broken up into comma or period-separated clauses (this holds more true with diffusion-type prompts than chat types but applies there as well. The results of a prompt will usually not be a single path but instead a set of paths with different probabilities, from most likely to least likely.

The results of parsing such prompts should generally be a set of properties and associated parametric values (likely as some javascript or Python object), which can, in turn, be turned into an appropriate query in a language such as SPARQL.

Note: I’m deliberately handwaving code here – the details of writing a prompt parser will be kept for a future article.

Persistence

Most queries do not take advantage of persistence. I believe this is because of the lingering perception (with some justification) that write operations are far more expensive than read operations and should not be done at all. However, the ability to create an intermediate context is critical to using knowledge graphs as compositional machines.

A composition occurs when you apply the results of one function as a parameter to another function. With respect to SPARQL, you can think of a composition as passing one or more IRIs of named graphs to another query to create another named graph. This can then be used to generate workflows. This technique is used for validation using SHACL and SPARQL, and it can be used to fuse data from heterogeneous graphs with dependencies. In this case, the named graphs help provide a context critical for managing interrogative sessions where state information must be maintained from one query to the next.

In general, persistence tries to keep as much of the set operations within the SPARQL invocations because serialization is expensive and creates dependencies that make chat-type applications much more difficult to generalize. SPARQL does not have an intrinsic pipelining mechanism, but many triple stores provide tools to build such pipelines, typically in Javascript, Python, or Lisp.

Presentation and Generation

Presentation and generation are similar operations and typically are carried out after the query and persistence phase.

Once one or more intermediate graphs are created, and it is time to pass these products back to the outside world, the transformation phase commences. Most older triple stores provide a number of formats for the output of queries, including Turtle (and its cousin Trig), nt, rdf-xml, rdf-json, and in many cases CSV or text delimited tables. Newer systems provide support for json-ld as a means of serializing named graphs.

Increasingly, especially in the era of ChatGPT and related tools, you’re also seeing the advent of MarkDown text (often unofficially tagged as having the datatype rdf:markdown). This provides a convenient middle ground for incorporating summary and annotational information from external sources (including ChatGPT). Additionally, such markdown is usually lighter than HTML and may contain Markdown for describing diagrams, mathematical notation (LateX), tables, image grids, etc. Because this kind of content is increasingly generated by external processes, this minimizes the post-processing that needs to occur once the dataset is returned to the user.

From a conceptual state, the serialization process can be seen as the reduce part of a map-reduce operation. What is returned is likely a JSON block with both summarized content (in markdown) and data preserved in a corresponding array of object links. It’s worth drawing attention to the similarity between a prompt response and an Atom or RSS feed response, a pattern that occurs frequently in semantic systems.

Garbage Collection

As can be expected, intermediate files consume memory. Chats differ from more traditional queries in that they rely upon context, and as such when these queries are complete, there needs to be a way to safely release the graphs created as part of the underlying session. This is not all that different from any other connection-based operation. Other than that, in some cases, permanent changes pass back into the knowledge graph – the graph learns over time. This means protocols must be established to transform relevant artifacts from the session into “new” knowledge while determining which temporary graphs can be deallocated.

Summary

Nothing discussed here is necessarily new, but it does represent a shift in the way that view and use knowledge graphs, especially as it becomes likely that such knowledge graphs may end up becoming the primary feeding mechanisms for LLMs, which otherwise require costly and less efficient retraining to rebuild its high count parametric models. Regardless, as chat models become more standardized, users’ expectations will strengthen the case for prompt-based queries so that people can interact with knowledge graphs in the same ways they do with LLMs.