I might be out of topic, or have a complete misunderstanding of the subject at hand, however this is my experience regarding PKM. I've dabbled with PKM for my day to day work, as well as tasks, knowledge and lessons gained from topics I've either discussed in depth, or experienced, that I should hold to a higher standard than simply trying to internalize & "forget." I try to use a sort of primitive PKB to hold all of these topics. However, it seems like, at least in the tool I use, which is Joplin, there's friction in the way of trying to "tag" these notes smartly. I can categorize them in a general sense, e.g. these notes are for "work", these ones are for "family" etc. however any further association is tough to do; The biggest problem I've faced is "how fine-grained should I tag my notes according to their contents"? It's tough for me to answer this; I don't have any professional experience in this area for organizational KM or societal KM. It's also alot of work, because if all you want to take are notes/excerpts from a source you're reading and you have to tag them & ask yourself multiple times the question of "to what degree" should you be tagging the note itself, this becomes bothersome and probably prohibitely expensive regarding "time taken", depending on the length of the note of course. Tagging'll still be a fraction of the time it took myself to write the note initially, however I cannot see the benefits because whenever I've tried to use these tags somehow, e.g. either by filtering by tag to get back to something old, I've found that oftentimes I've most likely not tagged it "correctly" for my question at the time. What I want to get at is this: knowledge is as useful as the metadata that describes the knowledge itself. This is especially true the bigger the data gets; it is prohibitively expensive in terms of time and money to do this manually, especially for PKM since you're not dealing with "business data" often that are well defined and categorized. My couple of questions is this: do you have a practical scenario that PKM could be used in day to day life for regular people (not work necessarily)? Do you think the "tagging" issue I aforementioned, assuming it is valid and fair, will be one that'll be solved with LLMs eventually, or a simpler but just as good narrow AI? EDIT: it's worthwhile to note that, as you mentioned, the traversibility of the graph is important, however I'd like to add presentation as a second point, if we're able to make any sense of a given topic. Given that our interests, and perhaps our personal notes and not professional ones, wide and shallow instead of narrow and deep, this might even make it borderline impossible to use a "sane" (perhaps static) schema to usefully define our notes, if we're to use such a PKB to query such a huge variety of topics whose nodes might be partially interconnected.
What works for me daily are PKM tools that are block-centric (every paragraph is a node in the graph). My system of tagging is based on an ontology(evolving but controlled) and taxonomy(open and flexible). The most useful "tags" so far are some classes (Issue, Event etc), people, dates, and projects (including mini-projects like writing an essay).
@@kvistgaart Hi, thanks for taking the time to reply. Since writing this comment, I've given up on tagging notes by any sort of criteria, and I've just used the 'Global Search' of Joplin, which does search contents. This works well for anything that does not involve historical data, e.g. versioning of a note if any details changed since etc. I came to the conclusion that, personally at least, manual PKM w/ respect to time taken, is impossible. Since then, with the release of LLMs that can be self-hosted, and have a huge context length, models like llama3.1:70b work surprisingly well for most tasks regarding PKM, apart from versioning. Even though I am very critical and very skeptic against LLMs due to hallucinations, costs and other issues, completely offloading the cognitive burden of organizing and filtering your stream of conciousness is the way to go. RAG (Retrieval Augmented Generation) w/ 'efficient' (models in the range of 20b-80b range) looks ideal. Honestly, PKM as a manual process should fade in time. Unstructured data was never a good fit for structured processes, and the granularity is a perpertual process you cannot ever win; the chaotic nature of LLMs seem to be the ideal solution here, just how NoSQL was the solution to a similar issue for data storage. Their usefulness for 'chatting' with resources & interconnected topics seems to be most evident when using services like Phind & Perplexity, and I'm convinced this is the way going forward. I wish Obsidian and other PKM tools/applications start experimenting with this approach, as it seems far easier & far more organic to 'chat' with your most related notes (per query), and by extension your stream of conciousness at the time of writing the notes. I've read several esoteric blog posts regarding this topic, and judging by my experience w/ the state-of-the-art RAG (so-called agentic RAG), it's truly remarkable what these tools can do. Apologies if this sounds like some sort of sales pitch, but it's not often I get to share my excitement for a viable esoteric solution to an already esoteric topic :). Thanks again for taking the time to reply!
Fantastic talk thank you.
I might be out of topic, or have a complete misunderstanding of the subject at hand, however this is my experience regarding PKM. I've dabbled with PKM for my day to day work, as well as tasks, knowledge and lessons gained from topics I've either discussed in depth, or experienced, that I should hold to a higher standard than simply trying to internalize & "forget." I try to use a sort of primitive PKB to hold all of these topics. However, it seems like, at least in the tool I use, which is Joplin, there's friction in the way of trying to "tag" these notes smartly. I can categorize them in a general sense, e.g. these notes are for "work", these ones are for "family" etc. however any further association is tough to do; The biggest problem I've faced is "how fine-grained should I tag my notes according to their contents"? It's tough for me to answer this; I don't have any professional experience in this area for organizational KM or societal KM. It's also alot of work, because if all you want to take are notes/excerpts from a source you're reading and you have to tag them & ask yourself multiple times the question of "to what degree" should you be tagging the note itself, this becomes bothersome and probably prohibitely expensive regarding "time taken", depending on the length of the note of course. Tagging'll still be a fraction of the time it took myself to write the note initially, however I cannot see the benefits because whenever I've tried to use these tags somehow, e.g. either by filtering by tag to get back to something old, I've found that oftentimes I've most likely not tagged it "correctly" for my question at the time.
What I want to get at is this: knowledge is as useful as the metadata that describes the knowledge itself. This is especially true the bigger the data gets; it is prohibitively expensive in terms of time and money to do this manually, especially for PKM since you're not dealing with "business data" often that are well defined and categorized.
My couple of questions is this: do you have a practical scenario that PKM could be used in day to day life for regular people (not work necessarily)? Do you think the "tagging" issue I aforementioned, assuming it is valid and fair, will be one that'll be solved with LLMs eventually, or a simpler but just as good narrow AI?
EDIT: it's worthwhile to note that, as you mentioned, the traversibility of the graph is important, however I'd like to add presentation as a second point, if we're able to make any sense of a given topic. Given that our interests, and perhaps our personal notes and not professional ones, wide and shallow instead of narrow and deep, this might even make it borderline impossible to use a "sane" (perhaps static) schema to usefully define our notes, if we're to use such a PKB to query such a huge variety of topics whose nodes might be partially interconnected.
What works for me daily are PKM tools that are block-centric (every paragraph is a node in the graph). My system of tagging is based on an ontology(evolving but controlled) and taxonomy(open and flexible). The most useful "tags" so far are some classes (Issue, Event etc), people, dates, and projects (including mini-projects like writing an essay).
@@kvistgaart Hi, thanks for taking the time to reply.
Since writing this comment, I've given up on tagging notes by any sort of criteria, and I've just used the 'Global Search' of Joplin, which does search contents. This works well for anything that does not involve historical data, e.g. versioning of a note if any details changed since etc.
I came to the conclusion that, personally at least, manual PKM w/ respect to time taken, is impossible. Since then, with the release of LLMs that can be self-hosted, and have a huge context length, models like llama3.1:70b work surprisingly well for most tasks regarding PKM, apart from versioning.
Even though I am very critical and very skeptic against LLMs due to hallucinations, costs and other issues, completely offloading the cognitive burden of organizing and filtering your stream of conciousness is the way to go. RAG (Retrieval Augmented Generation) w/ 'efficient' (models in the range of 20b-80b range) looks ideal.
Honestly, PKM as a manual process should fade in time.
Unstructured data was never a good fit for structured processes, and the granularity is a perpertual process you cannot ever win; the chaotic nature of LLMs seem to be the ideal solution here, just how NoSQL was the solution to a similar issue for data storage.
Their usefulness for 'chatting' with resources & interconnected topics seems to be most evident when using services like Phind & Perplexity, and I'm convinced this is the way going forward.
I wish Obsidian and other PKM tools/applications start experimenting with this approach, as it seems far easier & far more organic to 'chat' with your most related notes (per query), and by extension your stream of conciousness at the time of writing the notes.
I've read several esoteric blog posts regarding this topic, and judging by my experience w/ the state-of-the-art RAG (so-called agentic RAG), it's truly remarkable what these tools can do.
Apologies if this sounds like some sort of sales pitch, but it's not often I get to share my excitement for a viable esoteric solution to an already esoteric topic :).
Thanks again for taking the time to reply!