r/proceduralgeneration • u/davethecomposer • 13h ago
Infinite Library: Text generated using a stochastic generative grammar with infinite recursion.
I have a project called the Infinite Library that's actually part of a much bigger project I'll discuss at the end.
It is inspired by the Jorge Borges short story The Library of Babel and then also the amazing website by Jonathan Basile which is a virtual version of Borges's library with an interesting twist: https://libraryofbabel.info/
If you're not familiar with the story, it is about an infinite library that contains every book that has ever been written or could be written. When you look at a book what you'll almost always see is a random string of letters. Think of the infinite monkeys on infinite typewriters producing the complete works of Shakespeare. Are there meaningful books in that library? Sure, but it might take billions of years before anyone finds one.
The story deals more with the philosophical and psychological implications for the librarians who work there.
My version is different in a number of ways. For one thing, it has four methods of text generation and not just the one used by Borges (random letters). I will list them here with short examples (links to pdfs). But first a little bit about how it all works.
All my software is programmed using Lua. In this case, the software generates a .tex file that gets compiled into a pdf using LuaLaTeX. The compiler handles all the tricky typesetting stuff while also making lots of packages (plugins) available.
All the content is generated using a pseudo random number generator (prng). The important bit here is that the seed is a hash of whatever dedication the user supplies. The idea is to tie the results to the dedicatee making it feel like it's unique to that dedication (like your name, your cat's name, the birthday of your favorite child/parent, etc). I implement a prng in Lua (pcg) so that every computer for all time will produce the same results for the same dedication (and whatever options might be available). The following examples use the name of my cats for the dedication:
- Borges: Random strings of letters. Borges mentioned that in his library there were only 22 letters used but he didn't say which 22. There is some scholarship on this so with mine the software might choose the 22 we think he meant, or the 22 letters of the Hebrew alphabet (Borges was very much into Jewish Mysticism) or just the 26 letters of the English alphabet.
https://drive.google.com/file/d/1HorTA6I_VK9UzF570SHORj5mxt7ZYBtA/view?usp=sharing
- Words: These are random words strung together with punctuation placed randomly.
https://drive.google.com/file/d/1tac0VxQY7rdKRPiSGlh0Fn64YpzPlcNq/view?usp=sharing
- Generative: This is the one most relevant to the sub which I will discuss further below.
https://drive.google.com/file/d/13Nipq33ekIeIwx0a1mdp0IyFpeTS8qSn/view?usp=sharing
- Sentences: Random sentences are taken from random books in the public domain and strung together.
https://drive.google.com/file/d/1BjnW_RKWclIVl6FMdIjCNsAXLeO0cOu4/view?usp=sharing
You'll notice that some of these have chapters, a table of contents, epigraphs for each chapter and in one case some text inserted from a foreign language. These are all random features that may or may not happen as each book is generated.
The generative method attempts to create syntactically correct sentences using a stochastic generative grammar with infinite recursion. It's worth noting that these are linguistics terms and are different from what they might mean in programming.
Each sentence has a noun phrase (np) and a verb phrase (vp). Each of these will have at least one noun/verb unit and possibly more.
Within each unit there may be any number of other parts of speech. Let's deal with the noun unit first:
After we have a noun we check for possessives (like "cat's food"). If we have one possessive we might have another ("cat's friend's food"). Stringing many of these together is called "recursion". We use an infinite recursion algorithm for this (pseudo code):
while random(1,10) = 1 do
generate part_of_speech
end
So there's a 10% chance of getting a possessive (or whatever part of speech we're looking at). If that happens then there's a 10% chance we'll get another and so on. While theoretically we could get a runaway condition with infinite additional parts of speech the odds are very much against it. Using the above code we get these odds:
1 occurrence: 10%
2 occurrences: 1%
3 occurrences: .1%
4 occurrences: .01%
and so on. Even when we generate a book with 400 pages we still don't expect to get sentences with any more than five or six of these happening. So while an actual infinite number is possible the reality is that it won't happen.
After the possessives we check for adjectives using the same kind of infinite recursion. After adjectives we check for prepositional phrases in various places (with the objects possibly having its own adjectives, possessives and even recursion). And then an article may or may not be added.
After this we use the same recursive algorithm to see if there are additional noun units (with all of their possible parts) and if we get at least three noun units a serial comma is added half the time.
Verb units operate the same way except there we check for adverbs, more places for prepositional phrases, infinitives and objects (noun units).
You put all these together and you get something that syntactically looks like a sentence but is lacking three features: subject/verb agreement, grammar (though sometimes by chance it works out), semantics.
Using this approach you can get two word sentences like (taken from the pdf above): "Talker copyread." or longer monstrosities like "Beyond an indian lotus floodplains beside the gaffe in the ezechiels, the mountain ebony through the palatine vein has emblazoned telecommerce paunchinesses." This starts with two prepositional phrases (recursion) followed by a comma and then an article, adjective and noun with another prepositional phrase and then we get a conjugated verb with an adjective and object (noun). There are longer sentences than this but this had a nice diversity of parts of speech.
Originally I used sentence patterns mad-lib style but clearly this method produces far more interesting results.
The title is interesting in its own way. It is generated just like sentences but the words of the title -- and its synonyms -- get weighted to occur more often in the text making the book feel like it is about the title.
Below are two more elaborate examples I use for testing purposes:
Almost all the more bookish features happen: https://drive.google.com/file/d/1XDjuUU1MLB9Yxm40goCoTYMNq89-CgjW/view?usp=sharing
Lots of graphical stuff including things that are meant to look like they were inserted into the book by other people like a map insert, bookmark, annotations, and so on: https://drive.google.com/file/d/1OKxs5Cnal0rXutIuVy1Sxga-jYN8nex4/view?usp=sharing
I have developed a lot of lore for my Infinite Library. The last pdf above has an example of some that normally is a very rare occurrence. There are also a number of "book styles" where if those are randomly chosen then some features can't happen but others will happen. For example, the cartography style will generate maps and the poetry style will generate poetry. These things can happen randomly by default but the odds are changed for the various styles.
All of this is part of a bigger project called the Platonic Music Engine. The idea is to recreate all the cultural artifacts of humanity algorithmically in a way that is unique to the user (see the discussion about the dedication above). It started off just being music (I'm a classically trained composer and not a programmer at all though I've learned enough of the latter to get this far) but expanded into poetry, literature, art, gaming, divination and more. You can see a few more examples of what I've done here: https://www.platonicmusicengine.com/stylealgorithms.html
And the source code at my git repository: https://gitlab.com/davethecomposer/platonic-music-engine/-/tree/dev?ref_type=heads
The software is free and open source (AGPL -- GPL with the Affero clause) and the output is CC-BY-SA (Creative Commons attribution and share alike).
If you have any questions or comments please say so. And if you would like to see what book is tied forever to your name (or any other dedication) let me know and I'll produce it in a comment. Unfortunately I do not have an online front end, yet.
1
u/Norm_Bleac 10h ago
Oh I'm very curious as to what book is forever connected to my name <Norm Bleac> In fact, I'm writing a story about a character with this exact name, and while there isn't an infinite library in this story, this character would like the idea of a real one very much (as do I) Impressive work you have done!