Regular org mode in emacs is very powerful. But if you want to build a zettelkasten then the emacs package org-roam is extremely powerful. org-roam combined org-roam-ui will give you all the functionality of obsidian, including the fancy graph view, but also with all the benefits of emacs and orgmode.
I love obsidian! The whole zettelkasten approach helped me tremendously in my studies for university. It feels like I'm building a very specialized personal wiki for highly technical and scientific topics. You collect and parse the ideas on your own words and externalize the work done by you in written Markdown. You're actively engaging with the content by continuously rearranging and connecting ideas from different fields and further iterating on that knowledge base. For me this helps with learning like nothing else. And if you forget details of concepts over time you can reread and understand immediately those concepts because you're already written these in YOUR OWN WORDS. And you know which explanations work for you best. Feels fucking great. Everyone should try some form of zettelkasten.
So excited for the 'Home Server' series and one of the most important topics to start with! Definitely lacked on documenting my stuff and wasting time researching the same topics over and over. I am using BookStack as KB now, it's pretty basic but covers my needs and supports popular export formats. For files I got Nextcloud with a folder structure and file naming scheme.
Glad to hear BookStack is covering your needs! My main use-case when building BookStack was a mixed-skill internal/team-based environment. It may fall short in a lot of areas as a personal knowledge base for very-technical focused people (this video's audience) but it does suit some people very well. Many of the complex features are often hidden a layer behind-the-scenes. At the personal level I have heard some positive stories of people using and getting their family on-board to document recipes and household information!
@@BookStackApp First of all, thanks a lot for all the work you have put into this project and offering it as FOSS to the community! It really is a great tool and has helped me a lot so far! Well yes, it may not be the best for people who want to have a tool that does more than just basic note taking with the most common functions. Sometimes I run into limitations myself and would wish for more functions, but those are usually cosmetics or non crucial functions. One example would be linking attachments with a preview of it on the document instead of having to use the text link format, so nothing that is really impacting my experience that much. Also, I don't think techy people always search for software that is highly customizable and stuff. I would consider myself rather thech savy and also run a whole rack full of homeservers and I was actually explicitly looking for an easy to use tool. I just wanted a simple way to document all my configurations, structures, guides, problems and other useful stuff and being able to search it. As a side note, sometimes I would like to have a deeper hierarchy inside BookStack, but somehow I feel like it actually stops me from creating a labyrinth. :D
@@h4X0r99221 Thanks! On that last point, I did originally build it with an infinitely nestable page structure but realised it could easily destroy discoverability/accessibility of content, instead becoming much like the deeply nested folder structure of word docs I was attempting to replace, hence I settled on the three distinct levels.
Wendell at around 16:35 talks about the remote VSCode deployment as though it might be difficult for a beginner to achieve. I used my $100 of Linode credit from the Level 1 sponsorship to do exactly that for my first deployment of anything ever (…and note that my dayjob is teaching business at a university and has nothing to do with tech). Basically, I got a remote Linode server and my own domain from Hover, and learned how to get all of that setup and how to ‘SSH’ into my new server through Linode’s own website tutorials. Then I learned about ‘Docker’ and ‘Docker Compose’. In the end, I deployed a ‘Docker Compose’ network that had the following containers: - A ’nginx’-based container acting as a reverse proxy, which handled https/SSL termination (with ‘certbot’ running on a ‘cron-job’ to renew the ‘LetsEncrypt’ SSL certificates periodically) - A ‘postgres’-based container for my database - A basic webapp backend (written in Haskell, lol) serving up the API and handling ‘websocket’ connections that were forwarded by the ‘nginx’ container, as well as handling all of the database ‘CRUD’ actions. ‘ A basic PureScript (like Haskell but compiles to JavaScript) ‘SPA’ page for my frontend, which I have the ability to update and regenerate from my ‘code-server’ container (this is the remote VSCode instance, based on the VSCodium open source thing Wendell mentioned). And I do basically all of my editing through browser-based remote VSCode. And because I went through the process to get SSL/TLS working through that ‘nginx’ container, Apple doesn’t have a cry and so will let me use my iPad browser to connect to my website over https and use VSCode in my iPad browser, which is pretty great. ANYWAY… everything within ‘marks’ were terms I hadn’t heard of before I got my $100 of Linode credit. So thanks Level 1. It was a very busy month but I learned a lot :)
I've been trying Zettlekasten on and off for three years now - it hasn't "stuck" for me yet, but I keep trying, because I really do need a better way to keep track of all my projects and all the information I collect for them. The best resource I can recommend on this is a book by Sönke Ahrens titled How to Take Smart Notes. This book, more than any other source I found, actually explained both why and how quite well. It's also a short book, 178 pages, so you won't be drowned in detail.
This series is absolute GOLD. I happened to be looking at Obsidian, Notion, and logseq over the past week. I dismissed Notion early on since it wants to keep my files on it's servers. Obsidian wins on most features, but there are several ideas in logseq that have merit... I just wish the editor was less 'problematic'. Thanks Wendell!
We all appear to be on the same righteous quest as Wendell; to find the best available tool for the task. So far I've experimented with Evernote, Joplin, LogSeq, and Obsidian. None do everything I want, but Obsidian is what I'm leaning towards right now. It would be interesting to hear what people see as the strenths and weaknesses of other similar tools: - Roam Research - Notion - Obsidian - LogSeq - Joplin - Athens - Org-mode - Evernote - Bookstack - neorg - Emanote - Org-roam - Vimwiki - Craft - Workflowy - Trilium Notes - others? Opinions❓
@@michaellessard2011 i use logseq on the daily, and in the last four months, it's come a long way. the way the two programs do links is different, and there are still some major teething pains when working with the same graph (project) on two computers. (clobber hell in some cases) so long as the sync of my choice is working and i have the GIT functionality on (set to 600 seconds, which is 10 minutes, the max duration between auto commits) everything seems to be gravy. about a month in and no regrets.
Obsidian is used by CGP Grey. You can listen to him talk about it on the ‘Cortex’ podcast quite frequently. Along with many other “I’m glad I don’t inhabit his head” related behaviours.
Still loving this series! Guess I have been Zettelkastening for the last 10 years without even knowing it ;) I even use symbolic links at my own peril. The ultimate for me would is a universal extensible file container for ANY type of file(that of course supports extensible tags/metadata). Many applications (including my own brain) are still looking at file extensions for hints. And the problem with that is you can only have one per file. The other is that when you move a file, any kind of database app that you might be using also needs to to be told about it. If you can tag the files themselves then reorganization becomes easy with any application you want to use...on any filesystem
Dammit I knew it was going to be sooner than later that I would end up migrating everything from bookstack! But yeah seeing what has happened to my RUclips favorites and recipes bookmarks really reinforces the idea that if you don't have a local copy then you haven't really saved it at all.
Putting files in folders is very different from the zettelkasten method. Rather, it's often discouraged, because no categorization is perfect and might hamper free association. The main feauture of a zettelkasten is the interlinking between notes. It's purpose is reminding you of other bits of information that are related to the original topic.
I was quite surprised to see obsidian is closed source, I had them confused for a different obsidian. I like org-roam but I have been keeping an eye out for options that dont tie me to Emacs. Athens looks like a nice open Obsidian alternative. Emanote looks interesting and editor agnostic but I dont think it has the fancy graphs I like from these roam clones
I've been quite happy with Logseq as an open alternative to Obsidian and Roam. I tried Athens, but progress was much slower and the community was dwarfed by that of Logseq.
Great video, data hoarding is not about some flex on how much storage you are using but it is great becuase you have essentially decentralized all your media:music, movies, tv-series. Also I think your audio is a little muted, don't know if you noticed that.
I seriously wish I didn't work as much as I do so I could dedicate time to developing a system like this for myself.. still may consider it in the future. Thank you, Wendell _engagement_
try logseq, it encourages a daily note style/outliner workflow that comes out of the package. U just start using it and its very intuitive! Obsidian is better at longer form writing but logseq is so good. Also id give amplenote a go too but i already got too many apps
I've been using directory structures as a knowledge storage system and bookmarking system for over 30 years. The problem with indexing the connections between numerous sources lies with the relationships between the content within the files within the directories or the content within the webpage of the bookmark. Keeping track of all these nodes and edges, even with a simplified tagging system, requires a graph database like Neo4j.
I've enjoyed using zim wiki + git. I like having everything locally in the file system to sync over nextcloud or syncthing. It's a WYSIWYG that uses a modified DokuWiki syntax and stores it as .txt files. It supports plugins like code blocks / syntax and other nice features.
Imagine this combined with some sort of open source, locally run on a home server, text based assistant AI. Everything that makes this sort of knowledge mapping process useful for humans to keep track of information would map directly to what an AI would need to map and navigate data. You're not only helping yourself remember, you're helping the AI help you, building your own little Watson with your cookie crumb trail of notes you leave behind. We're not there yet, but it can't be that much longer before there are open source chatgpt equivalents you can run on your own hardware, and from there it's just a matter of hooking it up into this sort of personal knowledge network and you've literally got a personal fully configurable secretary that exists solely to help you manage whatever you need to manage. It's almost terrifying to think about, really.
Having a photo library, you find it super hard to categorize photos for later viewing. You have to tag subjects, dates, events, locations, and sometimes other unrelated topics to the photo. You need to use metadata tags and find methods for creating and searching using those metatags. Add video clips to that. Add PDFs to that. Then you want to search for a project you did years ago and want all the associated photos, videos, PDFs, spreadsheets, word documents, and email correspondence. You want some way to pull it all together back together cohesively so you can review it all. You can go mad trying to think of all the possible future uses or needs you may have to recall the information in the future. instead of recalling a single project, you might want to recall all instances you worked at a location, on a type of equipment, or worked on a specific problem type, what the resolution was, and the research you originally did that helped you resolve the issue.
Life advice right here. There is nothing worse than spending days/weeks/months hacking something together so it'll run for a few years then having to relearn it all over again just to figure out that something simple went wrong. I don't use software for this, just a complex folder structure and .txt's with instructions I've written to myself as well as all the data hording (i'm not a code god though sadly so no software for this anytime soon) I do appreciate the struggle though. My latest set of notes is around all the settings for each device i own (mainly my core clocks / volts for each gpu in the mine) (it can be very annoying to have to test clocks on the same card just so you can get to testing clocks on a new card) (or a bios config from your old ddr2 motherboard, you don't want to be trying to google some obscure setting that you already had figured out 15 years ago) I wish i thought like this back then but I'm learning this more and more these days. (i've lost so much information in my lifetime)
The basic principles of mind-mapping (as opposed to any specific mind-mapping app) I'd have thought of as fairly foundational in this context? Beyond that you need a way of linking branches on the map to folders and sub-branches to specific files, or pages/paragraphs/images on a specific file or files, and then to link from those files back to the map or to other files. I've never seen an app that does what I'm talking about, but would like to know if there is one?
I started using obsidian after coming across it in the very forum thread you mention. One easy way for syncing I use is the Obsidian Git plugin: just point it to a git repository (I use GitLab) and it automatically pushes all my notes to it. Any files under 100mb (mostly images,audio files and pdfs for me ) also get backed up this way as well. You can also embed YT videos in your notes directly as it supports html embeds
This raises important questions, such as: Does a knowledge system that contains all knowledge systems which do not contain themselves, contain itself? If only Bertrand Russell was alive, he would work it out for us...
The only thing missing from the file storage approach is tagging. I want something like FileMeta (on Github) that persists to something less fragile than NTFS alternate data streams...a cross-platform/file system indexer that stores a blob in the root directory.
Zfs does this and I used that for a while but ultimately it's pretty easy to add tags to the markdown then just index the text files. Search is instant
this feels like a bit of a call-out to get my notes together.. I have been using obsidian with "some" organisation.. most of the time it's just made on the spot if I don't have anything where the new thing fits.. never really "went back" to reorganise my notes into something sensible.. every time I need something I know is there, I just dive into the pile of knowledge, dig the information out, and leave.. Same thing with "putting" new information there.. it's more of and just throwing it where it roughly fits and moving on maybe this is the time to finally organise everything... (I mean start new project of organising everything.. that I'll definitely finish)
Hahahaha, just had a Total Commander popup while re-organizing data that I though was you and Obsidian related as it was in topic (until the image on the video changed and I noticed, oops that is mine)
In general, I have never found trying to organize things in a hierarchical structure, like directories, to be worth it beyond a few levels. I think tagging and linking is the way to go. Really google email does a good job of this, although don't really trust them. I think a file system that supports tagging would help finding things tremendously.
Vimwiki is a great, lightweight tool that lives inside vim. It can understand a few different formats (it’s own, markdown, restructured text) and was easy to migrate away from Obsidian. I just git to sync the files and different vim plugins for searching / distraction free writing.
Seeing a way to DIY syncing an Obsidian vault would be interesting. Right now I'm just using the "obsidian-git" plugin and my GitLab server to sync my two vaults.
13:18 Emacs is great, after you get used to it. People also love to talk about vim, I was never able to get my head wrapped around it, I just default to emacs every time.
i never heard of ZettelKasten untill now, but the way i would create a dir structures is like this, and back in about 2001-2002 i created a mysql database and a corresponding website for me to easly orgenise and retrieve all the pdf's, programs, etc i had, stored on my hd's. sadly i lost the db and website, due to a double hd failure, and never bothered recreating it about 10yrs ago.
Firstly, love the content, long time lurker, I try to subscribe in floatplane when possible! Thanks so much for everything you and the team do, it’s saved me immense amounts of time and helped incredibly with ideation & execution. Came back to follow up and act on this video. Your video barely came up in my search results when I looked for obsidian. I think this has actually been a pattern across the channel (thinking back to the IT/SysAdmin automation with Microsoft 1-2 years ago). Is there a reason you don’t put the software name in the title? It might help with search results, even at the cost of bringing misaligned viewers to your channel (niche audience). Or is that something that could actually negatively impact you with the YT algo?
My biggest issue over decades has been loss of storage devices. Not having a proper raid or backup and losing digital properties to mistakes, rot, or worse failure of hardware.
@@bassplaya69er Now I do, but no reason to go into that here :) Above is cautionary for anyone thinking they can just do this on a computer without proper planning.
I'm about to adopt obsidian as my primary command center. Although the Api is new, I can compose all my databases in markdown (though a workflow automator) with links depending on views in the database. If that doesnt work, the outliner plugin will. I can ditch my main mindmap thanks to the plugin markmind. I can relay anything in obsidian to a gatsby admin panel with gatsbyGarden plugin. And backup with github of course. Auxilary programs also integregrated with obsidian include logseq for task management, and raindrop for bookmarks. I'll let you know if this obsidian stack works. I hope to build a dataOcean with it. Composing all data of my life, and visualize it.
Yeah, FS structures will persist and if you want my option the main difference will be then they become "3D in a sense (not counting Time as a dimension, that would be versioning which we already have in a number of file systems)"
Really very interesting. I've been using Dokuwiki. I very much like the idea of the software tying in with a directory & file structure. This is also ties with having a logical file & folder, directory naming system - convention say the ISO standard?. I did look at Bookstack & Obsidian but definitely er to open source. Will revisit. It would be Very useful to have more discussion on what is out there, in particular user friendly & flexible vendor & platform agnostic solutions that can be cloud & self hosted simply, and don't have barriers to synchronising & sharing across multiple platforms. Linux, Mac, Android, Windows, Ios etc. I thought the commenter talking about Mind Maps & linking fascinating too. Thank you.
I don't know why I haven't seen you on youtube yet but I like your philosophies. Someday we may chat. But for now I have something interesting. I've weighed using obsidian and other knowledge types of mapping software heavily. Their's nothing out there that does it all yet. But besides obsidian there is one closer to a more visual organizer. It's called Knowledgebase builder by Ingo Straub. Fanatastic piece of software. Allows hyperlinking, strong xml support and built using sqlite. He's very active right now and releasing a lot of examples preparing for his next release I believe. I use it for organizing my ideas especially if I am preparing to make something public. The price so cheap right now and it's not a cloud only SAAS. It has that as an ability if you are prepared to make something public. I am not selling or affiliated with him for money in anyway. I believe that's his only weakness is the marketing the software. Works excellent as an Internet, Intranet or private knowledge map. His company I believe started with a card catalog system probably based on the zettelkasten idea. He would be familiar with this as he is a german software developer.
I'll take a look at Foam.. have used Dendron for the past year and like it's file name refactoring features. It's also a set of vscode extensions and markdown, and looks a lot like Obsidian. It's not folders, though, but one vault of notes with file name conventions, although you could create separate vaults and link to assets elsewhere (PDFs, etc.)
Yeah, sort of one of those... And having a knowledge repo, scientific and engineering mostly would be my goal. Need to find a nurse so I build her the Medical repository...
fascinating stuff. Obsidian sounds pretty similiar. of what i recently considered having by using a personal wiki, like Moin. Does somebody have experience with that?
was keen until I saw it is not a web app, thought it would be cool to host it on my home server. So I will stick to my KISS text file wiki system I started in 2004, just a bunch of files named by date of creation and displayed by my php script which does automatic html linking (I don't have to think about links) and automatic list numbering and formatting and so on (dont have to remember MD). Search does a grep across the files so very quick across the 817 files. However, obsidians nifty graph thingy is very cool.
I've tried so many knowledge management apps at this point. And frankly, I see this is as an unsaturated market cause none of the options convinced me. Almost all lack a good UI and that is actually appealing to work with, or there are quirks here and there that makes the experience quite rough. Obsidian, for example, is not a proper outliner and that's the biggest drawback for me. Meaning it can't behave like Roam Research or even Notion or Craft or Workflowy. There is an option to make headings and lists foldable, but it's very janky. And indenting and outdenting is very rough. What drives me crazy is the fact that folding arrows are misaligned with bullet points. I've spoken about this issue for months now. Someone really has to develop a PKM app and pool together the good things from all of these apps and eliminate the bad.
I assume other folks have similar issues because most of these systems force you to use their one editor. A bring-your-own editor solution would probably win big time.
Same for me. I really REALLY tried to go used to Obsidian since everyone is recommending it, but, I just couldn't get used to the interface. I didn't like it at all. And there are few quirks here and there that makes the experience quite rough.
@@Neumah it's too alien compared to the rest of my system's interface which I've spent a few too many hours into crafting because I'm a /r/unixporn fanboy
When Wendell mentioned "the discipline to put those files in the directory" it got me thinking about how I overheard some Gen Z techs talking about how they didn't really understand "folders" until they started working in tech. It seems that search has gotten so good (and mobile OS paradigms so ubiquitous) as actual physical file drawers and folders fade away that the metaphor isn't fully crossing the generation gap. I'm forced to wonder if it should; and if we should all expect what the kids seem to expect -- categorical views driven purely by file metadata and content. Functionally, the directory is just another file acting like a foreign key that the files share. As computers evolve, it's possible that the only enduring value of namespacing for file organization is that it's better for terminal interfaces and programming. For applications like a knowledge base, maybe we can do better.
Obsidian is missing 2 things. A web gui for those of us with corporate computers, and google/icloud integration so I can't get my Linux install to sync with my Phone.
I seen the forum blog you setup for this topic. It is just a bunch of people chiming in making the discussion even more complicated. You want to structure all you data, files, etc, why not start by structuring this forum topic at least. First by pinning a list of idea / software something to the start which grows, and through time amend features of each item so you can actually get some possible answers instead of none stop talk and growing forum topic. Perhaps your goal is to grow your forum and not come up with an answer to this very interesting topic?
even the best system won't get me to emacs, sorry. I use a dokwiki on my NAS which might be not that neatly organized, but I am used to it, and I like its simplicity. and even if all fails, its stored data is easily extracted and readable. (simple txt files with easy markup)
It's very interesting how the restrictive ecosystem of Apple basically obliterated all organizational skills in the younger generation. They don't understand file structures, and instead simply rely on search because that's what they've been trained to do. It's interesting to think how much efficiency is lost due to capability increase. Back in the day, it would be ridiculous to search all files for a specific string of text, but now that can be done easily by modern processors. I'm seeing this a lot in many spaces, with increasing capability, strategy of optimization is lost
File directory does not work for me, I need full text search. I know there are tools for full text search, but they are not useful when cloud directories also come into picture. Like I can't search on my phone and pc at the same time. I don't pay for cloud to utilize their full text search functionality and dump everything there. Currently evernote does it for me. Actually it is perfect, I can send content and tag and full text search images and text at the same time. The only downside is some content is not ideal to have it stored there, like you can't store large media files, which you shouldn't want to in the first place, but I am just giving an example where you can't use it as if it is a file directory. Everytime I watch a video on this knowledge management concept, everyone is talking about note taking and zettelkasten stuff. As if everyone is this note taking genius and needs to have their linked notes to manage their ideas. I am sure most of you all out there are lazy to just open a notepad let alone link knowledge together. I am sure there are people that need this because of the type of work they do, but I just want to emphasize how I feel full text search is the king of any feature one could come to ask from a knowledge management system. After that tagging comes next. Guys we don't even need to take notes here, all of us are bunch of hoarders anyways. So please stop with taking notes, all of you just store links and password. That is all for your note taking needs. Don't lie to yourself. Focus on what matters, so instead of having another note taking app released we can come to have some apps released for full text search.
I think the last tip in the video is very nice. High fidelity vscode over internet sounds great, I'll seriously consider it. What I additionally want to share is that if we are going to self host anyways ,we can have a nas drive that has cloud drive functionality and has full text search indexing also available. I think that is the best scenario here.
Great content as usual but I will be asking this of a lot of youtubers, please donate a slot to the Ukraine even if you are demonitized for it. if you are not open to a ban for politicising your channel. It's gone beyond meme's and jokes about incompetent, ienept and corrupt governments, this shit is real.
Regular org mode in emacs is very powerful. But if you want to build a zettelkasten then the emacs package org-roam is extremely powerful. org-roam combined org-roam-ui will give you all the functionality of obsidian, including the fancy graph view, but also with all the benefits of emacs and orgmode.
I love obsidian! The whole zettelkasten approach helped me tremendously in my studies for university. It feels like I'm building a very specialized personal wiki for highly technical and scientific topics. You collect and parse the ideas on your own words and externalize the work done by you in written Markdown. You're actively engaging with the content by continuously rearranging and connecting ideas from different fields and further iterating on that knowledge base. For me this helps with learning like nothing else. And if you forget details of concepts over time you can reread and understand immediately those concepts because you're already written these in YOUR OWN WORDS. And you know which explanations work for you best. Feels fucking great.
Everyone should try some form of zettelkasten.
So excited for the 'Home Server' series and one of the most important topics to start with! Definitely lacked on documenting my stuff and wasting time researching the same topics over and over. I am using BookStack as KB now, it's pretty basic but covers my needs and supports popular export formats. For files I got Nextcloud with a folder structure and file naming scheme.
Glad to hear BookStack is covering your needs! My main use-case when building BookStack was a mixed-skill internal/team-based environment. It may fall short in a lot of areas as a personal knowledge base for very-technical focused people (this video's audience) but it does suit some people very well. Many of the complex features are often hidden a layer behind-the-scenes. At the personal level I have heard some positive stories of people using and getting their family on-board to document recipes and household information!
@@BookStackApp First of all, thanks a lot for all the work you have put into this project and offering it as FOSS to the community! It really is a great tool and has helped me a lot so far!
Well yes, it may not be the best for people who want to have a tool that does more than just basic note taking with the most common functions. Sometimes I run into limitations myself and would wish for more functions, but those are usually cosmetics or non crucial functions.
One example would be linking attachments with a preview of it on the document instead of having to use the text link format, so nothing that is really impacting my experience that much.
Also, I don't think techy people always search for software that is highly customizable and stuff. I would consider myself rather thech savy and also run a whole rack full of homeservers and I was actually explicitly looking for an easy to use tool. I just wanted a simple way to document all my configurations, structures, guides, problems and other useful stuff and being able to search it.
As a side note, sometimes I would like to have a deeper hierarchy inside BookStack, but somehow I feel like it actually stops me from creating a labyrinth. :D
@@h4X0r99221 Thanks! On that last point, I did originally build it with an infinitely nestable page structure but realised it could easily destroy discoverability/accessibility of content, instead becoming much like the deeply nested folder structure of word docs I was attempting to replace, hence I settled on the three distinct levels.
@@BookStackApp Wasn't a bad decision, definitely forces me to think of a more streamlined structure. :)
Wendell at around 16:35 talks about the remote VSCode deployment as though it might be difficult for a beginner to achieve. I used my $100 of Linode credit from the Level 1 sponsorship to do exactly that for my first deployment of anything ever (…and note that my dayjob is teaching business at a university and has nothing to do with tech).
Basically, I got a remote Linode server and my own domain from Hover, and learned how to get all of that setup and how to ‘SSH’ into my new server through Linode’s own website tutorials.
Then I learned about ‘Docker’ and ‘Docker Compose’. In the end, I deployed a ‘Docker Compose’ network that had the following containers:
- A ’nginx’-based container acting as a reverse proxy, which handled https/SSL termination (with ‘certbot’ running on a ‘cron-job’ to renew the ‘LetsEncrypt’ SSL certificates periodically)
- A ‘postgres’-based container for my database
- A basic webapp backend (written in Haskell, lol) serving up the API and handling ‘websocket’ connections that were forwarded by the ‘nginx’ container, as well as handling all of the database ‘CRUD’ actions.
‘ A basic PureScript (like Haskell but compiles to JavaScript) ‘SPA’ page for my frontend, which I have the ability to update and regenerate from my ‘code-server’ container (this is the remote VSCode instance, based on the VSCodium open source thing Wendell mentioned). And I do basically all of my editing through browser-based remote VSCode.
And because I went through the process to get SSL/TLS working through that ‘nginx’ container, Apple doesn’t have a cry and so will let me use my iPad browser to connect to my website over https and use VSCode in my iPad browser, which is pretty great.
ANYWAY… everything within ‘marks’ were terms I hadn’t heard of before I got my $100 of Linode credit. So thanks Level 1. It was a very busy month but I learned a lot :)
I've been trying Zettlekasten on and off for three years now - it hasn't "stuck" for me yet, but I keep trying, because I really do need a better way to keep track of all my projects and all the information I collect for them. The best resource I can recommend on this is a book by Sönke Ahrens titled How to Take Smart Notes. This book, more than any other source I found, actually explained both why and how quite well. It's also a short book, 178 pages, so you won't be drowned in detail.
This series is absolute GOLD. I happened to be looking at Obsidian, Notion, and logseq over the past week. I dismissed Notion early on since it wants to keep my files on it's servers. Obsidian wins on most features, but there are several ideas in logseq that have merit... I just wish the editor was less 'problematic'. Thanks Wendell!
We all appear to be on the same righteous quest as Wendell; to find the best available tool for the task. So far I've experimented with Evernote, Joplin, LogSeq, and Obsidian. None do everything I want, but Obsidian is what I'm leaning towards right now. It would be interesting to hear what people see as the strenths and weaknesses of other similar tools:
- Roam Research
- Notion
- Obsidian
- LogSeq
- Joplin
- Athens
- Org-mode
- Evernote
- Bookstack
- neorg
- Emanote
- Org-roam
- Vimwiki
- Craft
- Workflowy
- Trilium Notes
- others?
Opinions❓
@@michaellessard2011 i use logseq on the daily, and in the last four months, it's come a long way. the way the two programs do links is different, and there are still some major teething pains when working with the same graph (project) on two computers. (clobber hell in some cases) so long as the sync of my choice is working and i have the GIT functionality on (set to 600 seconds, which is 10 minutes, the max duration between auto commits) everything seems to be gravy. about a month in and no regrets.
Obsidian is used by CGP Grey.
You can listen to him talk about it on the ‘Cortex’ podcast quite frequently.
Along with many other “I’m glad I don’t inhabit his head” related behaviours.
Still loving this series! Guess I have been Zettelkastening for the last 10 years without even knowing it ;) I even use symbolic links at my own peril.
The ultimate for me would is a universal extensible file container for ANY type of file(that of course supports extensible tags/metadata). Many applications (including my own brain) are still looking at file extensions for hints. And the problem with that is you can only have one per file. The other is that when you move a file, any kind of database app that you might be using also needs to to be told about it. If you can tag the files themselves then reorganization becomes easy with any application you want to use...on any filesystem
Dammit I knew it was going to be sooner than later that I would end up migrating everything from bookstack! But yeah seeing what has happened to my RUclips favorites and recipes bookmarks really reinforces the idea that if you don't have a local copy then you haven't really saved it at all.
Putting files in folders is very different from the zettelkasten method. Rather, it's often discouraged, because no categorization is perfect and might hamper free association.
The main feauture of a zettelkasten is the interlinking between notes. It's purpose is reminding you of other bits of information that are related to the original topic.
I was quite surprised to see obsidian is closed source, I had them confused for a different obsidian.
I like org-roam but I have been keeping an eye out for options that dont tie me to Emacs.
Athens looks like a nice open Obsidian alternative. Emanote looks interesting and editor agnostic but I dont think it has the fancy graphs I like from these roam clones
I've been quite happy with Logseq as an open alternative to Obsidian and Roam. I tried Athens, but progress was much slower and the community was dwarfed by that of Logseq.
Joplin deserves a mention too, been using it for years.
I keep coming back to this video. I really feel like we're on the verge of something great here.
Great video, data hoarding is not about some flex on how much storage you are using but it is great becuase you have essentially decentralized all your media:music, movies, tv-series.
Also I think your audio is a little muted, don't know if you noticed that.
I seriously wish I didn't work as much as I do so I could dedicate time to developing a system like this for myself.. still may consider it in the future. Thank you, Wendell
_engagement_
try logseq, it encourages a daily note style/outliner workflow that comes out of the package. U just start using it and its very intuitive! Obsidian is better at longer form writing but logseq is so good. Also id give amplenote a go too but i already got too many apps
I've been using directory structures as a knowledge storage system and bookmarking system for over 30 years. The problem with indexing the connections between numerous sources lies with the relationships between the content within the files within the directories or the content within the webpage of the bookmark. Keeping track of all these nodes and edges, even with a simplified tagging system, requires a graph database like Neo4j.
I've enjoyed using zim wiki + git. I like having everything locally in the file system to sync over nextcloud or syncthing. It's a WYSIWYG that uses a modified DokuWiki syntax and stores it as .txt files. It supports plugins like code blocks / syntax and other nice features.
Imagine this combined with some sort of open source, locally run on a home server, text based assistant AI. Everything that makes this sort of knowledge mapping process useful for humans to keep track of information would map directly to what an AI would need to map and navigate data. You're not only helping yourself remember, you're helping the AI help you, building your own little Watson with your cookie crumb trail of notes you leave behind.
We're not there yet, but it can't be that much longer before there are open source chatgpt equivalents you can run on your own hardware, and from there it's just a matter of hooking it up into this sort of personal knowledge network and you've literally got a personal fully configurable secretary that exists solely to help you manage whatever you need to manage. It's almost terrifying to think about, really.
Having a photo library, you find it super hard to categorize photos for later viewing. You have to tag subjects, dates, events, locations, and sometimes other unrelated topics to the photo. You need to use metadata tags and find methods for creating and searching using those metatags. Add video clips to that. Add PDFs to that. Then you want to search for a project you did years ago and want all the associated photos, videos, PDFs, spreadsheets, word documents, and email correspondence. You want some way to pull it all together back together cohesively so you can review it all. You can go mad trying to think of all the possible future uses or needs you may have to recall the information in the future. instead of recalling a single project, you might want to recall all instances you worked at a location, on a type of equipment, or worked on a specific problem type, what the resolution was, and the research you originally did that helped you resolve the issue.
Life advice right here. There is nothing worse than spending days/weeks/months hacking something together so it'll run for a few years then having to relearn it all over again just to figure out that something simple went wrong.
I don't use software for this, just a complex folder structure and .txt's with instructions I've written to myself as well as all the data hording (i'm not a code god though sadly so no software for this anytime soon)
I do appreciate the struggle though.
My latest set of notes is around all the settings for each device i own (mainly my core clocks / volts for each gpu in the mine) (it can be very annoying to have to test clocks on the same card just so you can get to testing clocks on a new card) (or a bios config from your old ddr2 motherboard, you don't want to be trying to google some obscure setting that you already had figured out 15 years ago)
I wish i thought like this back then but I'm learning this more and more these days. (i've lost so much information in my lifetime)
The basic principles of mind-mapping (as opposed to any specific mind-mapping app) I'd have thought of as fairly foundational in this context? Beyond that you need a way of linking branches on the map to folders and sub-branches to specific files, or pages/paragraphs/images on a specific file or files, and then to link from those files back to the map or to other files. I've never seen an app that does what I'm talking about, but would like to know if there is one?
I started using obsidian after coming across it in the very forum thread you mention. One easy way for syncing I use is the Obsidian Git plugin: just point it to a git repository (I use GitLab) and it automatically pushes all my notes to it. Any files under 100mb (mostly images,audio files and pdfs for me ) also get backed up this way as well. You can also embed YT videos in your notes directly as it supports html embeds
I love Obsidian! I didn't know they had a git plugin, I was just manually pushing to a private repo for backup!
I use Syncthing to achieve the same!
I'm a simple grad student. I see obsidian md, my ZK software, I click.
This raises important questions, such as: Does a knowledge system that contains all knowledge systems which do not contain themselves, contain itself? If only Bertrand Russell was alive, he would work it out for us...
The only thing missing from the file storage approach is tagging. I want something like FileMeta (on Github) that persists to something less fragile than NTFS alternate data streams...a cross-platform/file system indexer that stores a blob in the root directory.
Zfs does this and I used that for a while but ultimately it's pretty easy to add tags to the markdown then just index the text files. Search is instant
I've been using Wekan (Kanban software) as a similar system for this, great advice!
this feels like a bit of a call-out to get my notes together.. I have been using obsidian with "some" organisation.. most of the time it's just made on the spot if I don't have anything where the new thing fits.. never really "went back" to reorganise my notes into something sensible.. every time I need something I know is there, I just dive into the pile of knowledge, dig the information out, and leave.. Same thing with "putting" new information there.. it's more of and just throwing it where it roughly fits and moving on
maybe this is the time to finally organise everything... (I mean start new project of organising everything.. that I'll definitely finish)
I am excited about the home server series, would love to see a walk through of Proxmox installation or TrueNAS Scale along with hardware selection.
I use the relatively new Neorg on Neovim. It has some rough edges still but it's very promising.
I'll have to look into this. This whole comments section is all emacs. Gotta give vim some love!
Hahahaha, just had a Total Commander popup while re-organizing data that I though was you and Obsidian related as it was in topic (until the image on the video changed and I noticed, oops that is mine)
In general, I have never found trying to organize things in a hierarchical structure, like directories, to be worth it beyond a few levels. I think tagging and linking is the way to go. Really google email does a good job of this, although don't really trust them. I think a file system that supports tagging would help finding things tremendously.
Vimwiki is a great, lightweight tool that lives inside vim. It can understand a few different formats (it’s own, markdown, restructured text) and was easy to migrate away from Obsidian. I just git to sync the files and different vim plugins for searching / distraction free writing.
Seeing a way to DIY syncing an Obsidian vault would be interesting. Right now I'm just using the "obsidian-git" plugin and my GitLab server to sync my two vaults.
13:18 Emacs is great, after you get used to it.
People also love to talk about vim, I was never able to get my head wrapped around it, I just default to emacs every time.
i never heard of ZettelKasten untill now, but the way i would create a dir structures is like this, and back in about 2001-2002 i created a mysql database and a corresponding website for me to easly orgenise and retrieve all the pdf's, programs, etc i had, stored on my hd's. sadly i lost the db and website, due to a double hd failure, and never bothered recreating it about 10yrs ago.
System Crafters has a series of Emacs org-roam mode, which is very detailed. I recommend you watch that thing
It’d be really interesting to revisit this using some locally run LLMs
One way to sync mbile obsidian vaults is to cmit them with working copy ( paid app) to git. You can link a repository to your obsidian folder.
I am using Trilium Notes right now as I can easily host it myself just nice having an online editor.
Another program I like is Anytype, it's still on alpha but has many features to make databases and it uses IPFS
Firstly, love the content, long time lurker, I try to subscribe in floatplane when possible! Thanks so much for everything you and the team do, it’s saved me immense amounts of time and helped incredibly with ideation & execution.
Came back to follow up and act on this video. Your video barely came up in my search results when I looked for obsidian. I think this has actually been a pattern across the channel (thinking back to the IT/SysAdmin automation with Microsoft 1-2 years ago).
Is there a reason you don’t put the software name in the title? It might help with search results, even at the cost of bringing misaligned viewers to your channel (niche audience). Or is that something that could actually negatively impact you with the YT algo?
now how do we combine obsidian with encarta97 for a flash 256 color GUI and wiki like interfaces?
Thanks, I'm going to check it out!
My biggest issue over decades has been loss of storage devices. Not having a proper raid or backup and losing digital properties to mistakes, rot, or worse failure of hardware.
then you need a proper 321 backup statergy
@@bassplaya69er Now I do, but no reason to go into that here :) Above is cautionary for anyone thinking they can just do this on a computer without proper planning.
This is very relevant to my interests and the URL for this video (the end part) is kind of creeping me out.
at this point Google knows me better than any of my loved ones.
I'm about to adopt obsidian as my primary command center. Although the Api is new, I can compose all my databases in markdown (though a workflow automator) with links depending on views in the database. If that doesnt work, the outliner plugin will. I can ditch my main mindmap thanks to the plugin markmind. I can relay anything in obsidian to a gatsby admin panel with gatsbyGarden plugin. And backup with github of course. Auxilary programs also integregrated with obsidian include logseq for task management, and raindrop for bookmarks. I'll let you know if this obsidian stack works. I hope to build a dataOcean with it. Composing all data of my life, and visualize it.
Yeah, FS structures will persist and if you want my option the main difference will be then they become "3D in a sense (not counting Time as a dimension, that would be versioning which we already have in a number of file systems)"
Really very interesting. I've been using Dokuwiki. I very much like the idea of the software tying in with a directory & file structure. This is also ties with having a logical file & folder, directory naming system - convention say the ISO standard?. I did look at Bookstack & Obsidian but definitely er to open source. Will revisit. It would be Very useful to have more discussion on what is out there, in particular user friendly & flexible vendor & platform agnostic solutions that can be cloud & self hosted simply, and don't have barriers to synchronising & sharing across multiple platforms. Linux, Mac, Android, Windows, Ios etc. I thought the commenter talking about Mind Maps & linking fascinating too. Thank you.
I personally like to use Git and Marktext to document.
Had yet to hear about Obsidian
I don't know why I haven't seen you on youtube yet but I like your philosophies. Someday we may chat. But for now I have something interesting. I've weighed using obsidian and other knowledge types of mapping software heavily. Their's nothing out there that does it all yet. But besides obsidian there is one closer to a more visual organizer. It's called Knowledgebase builder by Ingo Straub. Fanatastic piece of software. Allows hyperlinking, strong xml support and built using sqlite. He's very active right now and releasing a lot of examples preparing for his next release I believe. I use it for organizing my ideas especially if I am preparing to make something public. The price so cheap right now and it's not a cloud only SAAS. It has that as an ability if you are prepared to make something public. I am not selling or affiliated with him for money in anyway. I believe that's his only weakness is the marketing the software. Works excellent as an Internet, Intranet or private knowledge map. His company I believe started with a card catalog system probably based on the zettelkasten idea. He would be familiar with this as he is a german software developer.
I'll take a look at Foam.. have used Dendron for the past year and like it's file name refactoring features. It's also a set of vscode extensions and markdown, and looks a lot like Obsidian. It's not folders, though, but one vault of notes with file name conventions, although you could create separate vaults and link to assets elsewhere (PDFs, etc.)
I remember I tried BookStack a while ago and it was promising but a little too early in development. I wonder if its worth a revisit
Have been moving forward steadily for over 6 years now! It's still quite an opinionated platform though (In terms of design and structure).
Yeah, sort of one of those... And having a knowledge repo, scientific and engineering mostly would be my goal. Need to find a nurse so I build her the Medical repository...
fascinating stuff. Obsidian sounds pretty similiar. of what i recently considered having by using a personal wiki, like Moin. Does somebody have experience with that?
was keen until I saw it is not a web app, thought it would be cool to host it on my home server. So I will stick to my KISS text file wiki system I started in 2004, just a bunch of files named by date of creation and displayed by my php script which does automatic html linking (I don't have to think about links) and automatic list numbering and formatting and so on (dont have to remember MD). Search does a grep across the files so very quick across the 817 files. However, obsidians nifty graph thingy is very cool.
Take a look at Joplin I use it for organizing and it s works on most platforms and work whth Next Cloud
I've tried so many knowledge management apps at this point. And frankly, I see this is as an unsaturated market cause none of the options convinced me. Almost all lack a good UI and that is actually appealing to work with, or there are quirks here and there that makes the experience quite rough.
Obsidian, for example, is not a proper outliner and that's the biggest drawback for me. Meaning it can't behave like Roam Research or even Notion or Craft or Workflowy. There is an option to make headings and lists foldable, but it's very janky. And indenting and outdenting is very rough. What drives me crazy is the fact that folding arrows are misaligned with bullet points. I've spoken about this issue for months now.
Someone really has to develop a PKM app and pool together the good things from all of these apps and eliminate the bad.
I assume other folks have similar issues because most of these systems force you to use their one editor. A bring-your-own editor solution would probably win big time.
@@MaxPrehl but you'd always loose on integration with other features :/
oh what I would pay for access to Wendell's knowledge repo directory
I was thinking the same thing. ... Also his self hosted media libraries 😂
Is it possible to use org mode with doom emacs? I'm way more speedy with vim bindings for day to day editing.
Personally I really dislike Obsidian's interface but do agree it's a solid product. That said emac Org mode is amazing
Same for me. I really REALLY tried to go used to Obsidian since everyone is recommending it, but, I just couldn't get used to the interface. I didn't like it at all. And there are few quirks here and there that makes the experience quite rough.
Just out of curiosity, what is it you don't like about the Obsidian interface?
@@Neumah it's too alien compared to the rest of my system's interface which I've spent a few too many hours into crafting because I'm a /r/unixporn fanboy
Obsidian is amazing.
This is invaluable information for organizing my porn.
Thanks for all the info !
Pair obsidian with hugo, you can publish your stuff too
When Wendell mentioned "the discipline to put those files in the directory" it got me thinking about how I overheard some Gen Z techs talking about how they didn't really understand "folders" until they started working in tech.
It seems that search has gotten so good (and mobile OS paradigms so ubiquitous) as actual physical file drawers and folders fade away that the metaphor isn't fully crossing the generation gap.
I'm forced to wonder if it should; and if we should all expect what the kids seem to expect -- categorical views driven purely by file metadata and content. Functionally, the directory is just another file acting like a foreign key that the files share.
As computers evolve, it's possible that the only enduring value of namespacing for file organization is that it's better for terminal interfaces and programming. For applications like a knowledge base, maybe we can do better.
Templater angry Data view plugins are amazing
What do people use to write/edit/sync/upload files/notes/second brain from mobile to their homeserver?
resilio sync for general files. google drive for one button document scanning with the included widget on android
But Wendell, I need the Matrix upload feature!
8:30 Hey look, it's Metroid Prime 2
Obsidian, isn't it built on electron?
Edit: Yes it is, and that is bad.
It is
So is VsCode
@@MaxPrehl Yes. But that is overrated crap. This would actually fill niche if it wasn't written on bloated pile of junk.
So is obsidian like notion?
It's pronounced Tsettel
Im from the future - in 2024, RAG seems like a powerful way to search through your internal documentation.
Obsidian is missing 2 things. A web gui for those of us with corporate computers, and google/icloud integration so I can't get my Linux install to sync with my Phone.
Take a look at syncthing
My always go to is org-mode and nextcloud.
another one of those videos that pops up just after thinking about the exact same niche subject
I seen the forum blog you setup for this topic. It is just a bunch of people chiming in making the discussion even more complicated.
You want to structure all you data, files, etc, why not start by structuring this forum topic at least. First by pinning a list of idea / software something to the start which grows, and through time amend features of each item so you can actually get some possible answers instead of none stop talk and growing forum topic. Perhaps your goal is to grow your forum and not come up with an answer to this very interesting topic?
even the best system won't get me to emacs, sorry. I use a dokwiki on my NAS which might be not that neatly organized, but I am used to it, and I like its simplicity. and even if all fails, its stored data is easily extracted and readable. (simple txt files with easy markup)
What editor do you prefer? Do you ever take notes from your phone?
Nice!
I found a great blog detailing a zettelkasten in vanilla vim.
Search and you will find.
Remember HyperCard?
Links to multiple locations is easy, in Linux Filesystems that is...
?
he uses arch, btw
It's very interesting how the restrictive ecosystem of Apple basically obliterated all organizational skills in the younger generation. They don't understand file structures, and instead simply rely on search because that's what they've been trained to do. It's interesting to think how much efficiency is lost due to capability increase. Back in the day, it would be ridiculous to search all files for a specific string of text, but now that can be done easily by modern processors. I'm seeing this a lot in many spaces, with increasing capability, strategy of optimization is lost
File directory does not work for me, I need full text search. I know there are tools for full text search, but they are not useful when cloud directories also come into picture. Like I can't search on my phone and pc at the same time. I don't pay for cloud to utilize their full text search functionality and dump everything there. Currently evernote does it for me. Actually it is perfect, I can send content and tag and full text search images and text at the same time. The only downside is some content is not ideal to have it stored there, like you can't store large media files, which you shouldn't want to in the first place, but I am just giving an example where you can't use it as if it is a file directory.
Everytime I watch a video on this knowledge management concept, everyone is talking about note taking and zettelkasten stuff. As if everyone is this note taking genius and needs to have their linked notes to manage their ideas. I am sure most of you all out there are lazy to just open a notepad let alone link knowledge together. I am sure there are people that need this because of the type of work they do, but I just want to emphasize how I feel full text search is the king of any feature one could come to ask from a knowledge management system. After that tagging comes next. Guys we don't even need to take notes here, all of us are bunch of hoarders anyways. So please stop with taking notes, all of you just store links and password. That is all for your note taking needs. Don't lie to yourself. Focus on what matters, so instead of having another note taking app released we can come to have some apps released for full text search.
I think the last tip in the video is very nice. High fidelity vscode over internet sounds great, I'll seriously consider it. What I additionally want to share is that if we are going to self host anyways ,we can have a nas drive that has cloud drive functionality and has full text search indexing also available. I think that is the best scenario here.
One interesting tidbit is that the German "Z" is spoken like "Ts" so "Zettelkasten" would be more like "Tsettelkasten" when spoken.
Don't use that software. But do some similar
I just watched it to see how he pronounces "Zettelkasten" ....
org-mode works better for me
Great content as usual but I will be asking this of a lot of youtubers, please donate a slot to the Ukraine even if you are demonitized for it. if you are not open to a ban for politicising your channel. It's gone beyond meme's and jokes about incompetent, ienept and corrupt governments, this shit is real.
I am using zim but hating every minute of their wiki syntax/wysiwyg. I want my markdown damn it! Nowhere near obsidian ofc