[Enhancement] Allow linking to inexistent notes

[adam-devel]

This Enhancement/Feature request doesn't involve the command-line interface, cosmoscope or documentation. It's mainly about error reporting and linking files.

TLDR

Support links to inexistent files or at least reduce errors relating to inexistent files to warnings.

Why

The note taking app Obsdian allows linking to inexistent notes which later on may get created.

When writing about something you have two options.

1. stop whenever you want to create a link to another topic. check if the file for that topic exists already. if it doesn't then create it. repeat everytime you want to link to another note
2. or just write your entire note with all the links then go over the text and find every link to a file and check if it exists yet and make sure to create it if it doesn't

both scenarios are tedious. the former is an obstacle to writing and latter is a chore that could be automated or avoided altogether. this really is a necessary feature.

Linking to inexistent files is very common. it's not just a random Obsidian feature. Wikipedia supports linking to pages that may not exist yet as well

Possible Approaches

1. ignore links to inexistent files and not show them in the graph
2. or it could display a warning instead of an error
3. or it could support the feature and show the inexistent files in the graph
4. or it could provide a command that finds links to inexistent files and create said files.

[infologie]

@adam-devel Cosma deliberately doesn't have functionality based on links to nonexistent notes, because the underlying needs can be addressed in more generic ways, which I encourage:

• If you need to link to something but don't need to describe it, just create an empty note under that name. Think about the usefulness of creating a note, even if it's empty: you may want to categorize it, add some tags to it, move it into a specific folder. (Remember that Cosma is designed for knowledge work: organizing is the core of the work!)

• If you need to link to something you will eventually describe, but you don't want to do it yet, and you'd like to find it easily later, use an appropriate tag (e.g. “stub”, “seedling” or whatever, to identify this early epistemic status—Maggie Appleton says “growth stage”).

These actions are more resilient to technical changes: an empty note can be handled like any other file; tags are universally supported, in various ways. In contrast, only a few apps support linking to nonexistent notes. So I'd rather have a tool that makes creating notes a very quick process, as easy as linking, and that drives me towards explicit knowledge organization, so that the work can be picked up easily in a simple text editor if needed.

If you have suggestions on how to reduce friction in the note creation process for Cosma, that would be awesome. (Also, like the other issue you created, I have strong opinions but I'm keen to discuss.)

[adam-devel]

If you need to link to something but don't need to describe it, just create an empty note under that name

you should be able to see how flow-breaking that is. imagine a file with 4 links. I have to stop typing. create a new file, fill it with the necessary boilerplate then come back to the original file. again and again for every new link. and god forbid I decide a minute later that actually I should have a different title.

compare that to just writing a wikilink and moving on, you can then just focus on writing your note.

you might want to categorize it

in my opinion this just adds unnecessary friction and also adds the overhead of pruning and updating one's categories. I rarely use categories.
if two topics are sufficiently different (a thesaurus of related words and notes on foundations of mathematics) then they warrant being separated into their own folders. if there's an overarching topic that encompasse a bunch of other topics, then it's more appropriate to dedicate a note to said topic then to link back the parent topic from its sub-topics

If you need to link to something you will eventually describe, but you don't want to do it yet, and you'd like to find it easily later, use an appropriate tag (e.g. “stub”, “seedling” or whatever, to identify this early epistemic status—Maggie Appleton says “growth stage”).

yes I could signal that with a tag or I could use what I already have: wikilinks.
It seems redundant to add something to my toolkit that serve a sub-purpose of another tool. it doesn't make a pretty ontology, that maybe a stretch however.

So I'd rather have a tool that makes creating notes a very quick process, as easy as linking

I don't think you can get more quick and frictionless than typing the wikilink directly into your editor and moving on. I suppose you could use external tools to automatically create the missing notes for you at the press of a button (when your cursor is over a link to a no existent file) and prompting for a category if you want that, but at that point you are admitting the usefulness of my proposal, you are just doing it in a roundabout way. in fact this is what obsidian does, when you click on, or try to access a non-existing link, it automatically creates the file for it. whether the file actually exist or not is an implementation detail at that point. the result is same.

---

I am sorry for the walls of text, I am too verbose for my liking as well. it takes re-reading and recontextualizing before I can make the text concise. I am happy to do so at your request. if it doesn't bother you however, I can respond faster at the cost being verbose

[infologie]

Compared to file names, I am less susceptible to change opinions about this topic. But I'm grateful for the opportunity to express and refine them, and maybe discover something new.

---

About breaking the flow. I don't have the same experience. If I write a paragraph that should contain links to 4 new notes, then I finish typing my paragraph and go create the 4 notes. Each note may spark another train of thoughts. Yes it can be a rabbit hole. But you can always walk back each train of thought to the initial note and continue. I don't really see the problem.

About categories and tags. I use them for organizational concepts (and you should too!): not things that I would write a note about, but ways in which I organize my notes. e.g. reading notes, definitions notes, biographical notes, etc. It complements what you describe: you can have notes on various topics and sub-topics, and the links create the hierarchy; but in each topic you may have notes which serve different purposes, and this is where categories/tags are useful. They are key tools for knowledge organization. (Cosma is my attempt to show people who only use categories/tags that they should also use links… and vice-versa!)

About the importance of creating the file :

I don't think you can get more quick and frictionless than typing the wikilink directly into your editor and moving on

That's the thing: you shouldn't move on! Not if you're working at a deliberate pace, being mindful of how you extend your graph over time. You should stop and think about that note. (This is related to the first point about flow. I look at it this way: A two-hour writing session is one flow, but inside are multiple little moments of flow, as you branch out from notes and create new ones. You walk back each train of thought to where you were when you branched off, and you continue. You're still in the flow of that session. But this may be more down to personal ways of "feeling" the work.)

whether the file actually exist or not is an implementation detail at that point

It's not a trivial difference: you cannot add metadata to a nonexistent file!

To explain why this is important to me: when I create a note, I immediately assign a category to it. My system only has a handful of categories. I've thought about them long and hard. It's very rare that a note doesn't fit: it usually means I'm about to add a new category, and it's only happened a couple times since I chose the initial set of categories. So when I create a note, even before adding content, it fits into the system. This is the same as when people say that notes should always be linked to at least another note, otherwise it floats away into space, and you have to rely on search to (maybe) find it again one day: a note should be fitted into the rest of your system, whether via links or categories, and ideally, both.

(For additional context, you could check out How To Write A Thesis by Umberto Eco, if you don't know it. Cosma was influenced by academic note-making traditions. Ways of organizing knowledge are central to the program's philosophy.)

About the speed of creating notes with Cosma. This is what I'm most interested in discussing further. We provide two commands to create notes. If we can improve them, we will try. And maybe there are ways to speed things up with command line tricks, shortcuts to call Cosma from a writing environments, etc. I'm going to think about this. But if anyone has suggestions I will take them gladly.

[adam-devel]

I am enjoying the back and fourth I hope it's not taxing you.

for now I just want to note a very clear difference in approache between me and you. later on I will comment more broadly.

I don't mind initial chaos (I used to mind it). I have a lot of floating files without any connections right now.

I write, then I edit and organize. I move things around, I merge similar notes, I separate notes that seem too busy. eventually it may become evident that I need an overarching note encompassing a bunch of notes, either linking to them or is linked back to from them.

I rely on my intuition for categorizing novel ideas to reduce friction between my mental categories and the categories in my notes. and I use standard categories for well established topics.

as a side note, I don't understand why one would have tags AND categories

[infologie]

Addressing only the side note:

I don't understand why one would have tags AND categories

They are the same conceptually. Let's call them classifications. The benefit of having different classifications types is that you have type-dependent features. Two examples:

• A classification type is associated with colors in an interface (like categories in Cosma) while others aren't. As I've mentioned, I use categories for kinds of notes: reading notes, concept notes, biographical notes, etc. These translate to colors, which is a nice visual aid, especially in the graph. Since there aren't too many categories, there aren't too many colors. And I use tags for projects: I tag notes with the name of a publication or a course I'm preparing, for which the note is relevant. (Like the different kinds of notes, projects are things I don't write notes about, I use them purely as an organizational concept. I do have project folders, READMEs etc. but they live elsewhere. Here it's just one more identifier.) Since these projects tend to multiply, it's better if they don't translate to colors, because it would make things less readable.

• A classification type is inserted by adding key: value in the YAML header, while another is inserted by adding #tag anywhere in the document. The former fits a high-level type of classification. The latter is convenient if you want to use it as a form of indexing. This is the way tags work in Zettlr, and someone requested their support in Cosma (issue [Enhancement] Detect #tags for customization #40).

[adam-devel]

The benefit of having different classifications types is that you have type-dependent features

Makes perfect sense. it's more convenient expose different use cases of the same structure as separate features

I don't share the use cases you mentioned below but they make sense

[adam-devel]

If I write a paragraph that should contain links to 4 new notes, then I finish typing my paragraph and go create the 4 notes

Yes. you can either create the note the moment you type the link to it or you can add to your workflow the additional task of reviewing the text for links to missing notes. the first option introduces friction and the second one introduces additional steps. you described the second option.

You hold that these unconvinces are necessary for maintaining a well categorized knowledge base. you believe categorization should be dealt with at the moment of creation of the link or soon after it. what follows is your support for this way of doing thing:

Each note may spark another train of thoughts. Yes it can be a rabbit hole. But you can always walk back each train of thought to the initial note and continue. I don't really see the problem.

the problem is: one doesn't always write about already established knowledge. sometimes you just don't yet know the boundaries of a given topic until you write a lot about it. the difference between writing about novel things and established topics is night and day. important relavant categories emerge as time go. they aren't preemptive.
you have to write and rewrite and remove links and add new ones. now compare editing text, where the moment you delete all links to a nonexistent note it vanishes, and the moment you edit the name of the link, the name of the nonexistent note also changes. compare that the empty note that remains dangling if I stop referring to it. the notes that requires renaming or editing as a file.

I use them for organizational concepts (and you should too!): not things that I would write a note about

you already have your filesystem for that. I use the graph view to see possible interesting connections between my notes. the sort of organization you are talking about should indeed be chosen at the time of creation. the mental burden of choosing a category is low because the categories are mostly disjointed. I agree with you on that. what I disagree with you about is the way you went about implementing this.

my critique is the following: since the categories are largly disjointed, it's unlikely for a note from one category to link to a note from another category. why choose the category every single time a note is created when you can safely assume the category of the new note to be the same as the note linking to it. you shouldn't introduce friction for frequent yet easily predictable data, chances are you are just writing the same category again and again for every link you create. introduce friction for exceptional scenarios.

This is how I do it: I just enter a directory and start writing and linking to notes. the notes are automatically assumed to be in the same category (the one represented by the directory I am in) because they actually are, I don't have to even think about categories. if I want to link to something outside this category (which is the exception, which is when it's appropriate to introduce friction), I simply prepend the link with the name of the category. for example [[categoryB/note title]]. this works because programs like obsidian allow to link to a note either by file name or by absolute path, the root of the absolute path is the directory that contains all my notes and each category of notes is under a directory representing said category. so the directory path directly translates to the category.

once you are convinced the user shouldn't be burdened with choosing a category for every single note instead of choosing a categiry at the beginning of a session and having all upcoming notes be part of that category. you no longer have a reason to require the manual creation of notes.

What i am trying to convince you of is that the created/uncreated distinction is an irrelevant one, I shouldn't be burdened with tracking that. it's much more comfortable to only think about "notes" and their relationships and not worry about the details of whether they are created or not.

[infologie]

@adam-devel: since the categories are largly disjointed, it's unlikely for a note from one category to link to a note from another category … I just enter a directory and start writing and linking to notes. the notes are automatically assumed to be in the same category

Most people who do academic research (our core users) don't work this way: they use categories to classify the types of things they write notes about, and a typical work session usually involves creating notes in several of these categories. For you, notes from one category usually don't link to notes from another category. But in a researcher's case, it happens all the time.

To give you an example, here are some of the categories I use: reading note, concept note, idea note (not exhaustive, it's just to give a quick example). A typical work session would be:

• I'm reading a paper, like "What is knowledge organization?" by Birger Hjørland; I start a "reading note" in which I copy quotes from the paper with my comments in between, linking to existing notes as appropriate;

• while reading, I come across the concept of "knowledge organization process"; I don't have a note on it yet, so I create one, with the "concept note" category; this usually repeats a few times;

• at some point, the accumulation of reading, writing and identifying concepts triggers a thought, for instance "hey, linking is a knowledge organization process too!" This I put in an "idea note". The idea is often expressed as a composition of various concepts into one statement, like [[Linking]] is a [[KOP]].

Anyway, coming back to this:

What i am trying to convince you of is that the created/uncreated distinction is an irrelevant one

The created/uncreated distinction may be irrelevant to you but it's relevant to Cosma's core audience. There's no right or wrong way here. It's just that Cosma deliberately supports only one way. This is because it was designed for people who rely on metadata. It's fine if you don't, but ultimately it means you have a different use case from us.

[adam-devel]

We take notes differently and we do it for different purposes, I don't think I would be able to convince you.

I want to ask however, what use do these categories serve ? Why do you want to have a separate category for readings, concepts and ideas ? I would collapse all of these under a broad "thoughts" or "ideation" category ?