Trying to be productive, efficient, and organized is an Achilles’ Heel like no other—it suckers you into endless procrastination in the pursuit of a method to end procrastination once and for all, creating a vicious cycle of identifying a system for getting things done, implementing it to the best of your ability, then discovering a different potentially better system that requires you to subtly (or substantially) modify all the work you’ve already done for the sake of becoming more efficient and productive than you were before.
At least, that’s how it’s been for me working with Obsidian (and Roam Research, Drafts, Things, and Ulysses before it). Even Bullet Journaling can get out of hand if you have the misfortune of finding the “BuJo” subreddit or hashtag on Instagram. In the productivity community, there are as many acronyms for productivity systems as there are letters in the alphabet—and they’re catchy. From GTD to LYT, PKM to PARA, it seems like there’s always a better way lurking somewhere on the horizon, and that can lead you to forever chasing the dragon of “better” or, worst of all, “best.”
And I get it—a lot of these systems are advocated by people who make at least some of their living writing or teaching about these subjects. In the words of Joker, “If you’re good at something, never do it for free.” There’s real money to be made in showing scatterbrained people (like me) how simple and easy things could be, how productive they could be, if only they would enroll in a short webinar on personal knowledge management for the astonishingly reasonable price of $299.
Full disclosure: I have never paid for knowledge management services or purchased a book on a personal productivity system. I own The Bullet Journal Method by Ryder Carroll and a collection of essays titled Manage Your Day-to-Day: Build Your Routine, Find Your Focus & Sharpen Your Creative Mind, but that’s the extent of my financial investment in learning how to organize my thoughts.
However, that’s not to say that I haven’t invested an inordinate amount of time and energy in studying how others organize their thoughts and get things done. If you’d like to peek into the rabbit holes down which I dove, here are the people whose methods I studied (in no particular order):
- Tiago Forte (PARA)
- David Allen (GTD)
- Nick Milo (LYT)
- Johnny Noble (Johnny Decimal)
- Eleanor Konik
- Andy Matuschak
- Ellane W
- Mark McElroy
- Jamie Todd Rubin
These folks have been wonderful guides, and each has a unique approach that I, at one time or another, thought was the perfect model to emulate when trying to make sense of my own database of notes. But none of those models was perfect.
The Perfect Personal Knowledge Management File System Note Taking Method
…does not exist. I wish that it did, I really do. I wish there was some mountain to climb, atop which was some monastery where wisened monks cultivated the ancient wisdom of how to conquer your own thoughts once and for all, such that all you had to do was learn its location, make the pilgrimage, and prostrate yourself before that ultimate truth. But there isn’t.
The quest for the perfect personal knowledge management file system note taking method is a snipe hunt. It’s a wild goose chase. It’s a pipe dream. It’s an elusive dragon that always hovers just out of reach, taunting you by appearing imminently attainable, but never allowing itself to actually be caught. It’s the pot of gold at the end of a rainbow.
A rainbow is an optical illusion that depends on the vantage point of the observer to be visible, meaning if you move, it moves, and the rainbow will look slightly different, in a slightly different place, to different observers. They can both refer to “the” rainbow, but there’s no singular object or phenomenon they’re observing.
Similarly, the perfect Bullet Journal spread or the perfect knowledge management system might exist for an individual insofar as there can be a particular method that works particularly well for a particular individual, but there’s no Holy Grail or silver bullet that’s going to work for everyone. Even a method that works well for an individual won’t necessarily work well for them forever—it’s highly context-dependent.
All that being said, I have adopted a method that seems to work especially well for my needs, and which I hope will be flexible enough to adapt to future use cases as my needs change over time. I’m sharing it here not because I think it’s something you should do, but more because I think it highlights a bit of the process by which I arrived at this method. One practice in particular is something I wish I had been doing from the very beginning.
The Filesystem Method that Works Best for Ian L Hayes
Trying a bunch of different approaches was instructive in that it let me discover what worked for me and what didn’t. After noticing I wasn’t getting much done whenever I opened my Vault and it had begun to just stress me out in general, I tried to articulate what specifically was bothering me about it and come up with targeted solutions for those problems. What follows are my personal pain points and the solutions I’m experimenting with to address them.
Pain Points and Their Solutions
My Daily Notes Page was Overwhelming and Nearly Unreadable Outside of Obsidian
My first Obsidian Vault, titled “Vault 111,” cobbled together all sorts of different note-taking systems as I learned about them. It started with a full import of all my Day One journal entries which were, themselves, transcriptions of my paper journals and various weblogs going back to approximately 2003 (as I go into more detail about here.)
After importing all of my journal entries, I established new Daily Note Page templates that relied heavily on various Dataview code blocks and snippets to offer a dynamic dashboard with every journal entry (as detailed in this other post). I thought this was going to help me stay focused on what I wanted to work on by surfacing relevant tasks and simulating some of the features I liked about Day One. I was particularly fond of their “On This Day” feature, letting me see what I was writing about in my journal on a given date in past years.
In reality, I was overwhelming myself before I had even sat down to work. I would open my Daily Notes Page to jot something down and be bombarded with things my past self thought would be urgent. The Dataview queries also required me to be very specific in the way I recorded data, so I created a couple dozen QuickAdd actions to ease the strain of having to remember all that. Unfortunately, QuickAdd (at the time, anyway,) had a lot of difficulty keeping synced between my Mac, iPhone, and iPad where I was accessing Vault 111.
I ended up having to spend a bunch of time tweaking my actions, troubleshooting the sync, and having to cross-reference a bunch of different Dataview code blocks throughout my Vault just to make sure my queries weren’t letting anything slip through the cracks. What should have been a simple act of recording something in my journal quickly became performing a bunch of maintenance just to make my Daily Notes Page work right, and by the time I put out all of the fires, I had forgotten what I even opened Obsidian to do in the first place.
Furthermore, I briefly experimented with using Apple Notes as my primary note-taking app, and when I ported over a bunch of notes from Vault 111, those Dataview code blocks obfuscated the files’ meaningful content and left me with a bunch of plugin-specific noise to intentionally ignore. The whole reason I got into Obsidian in the first place was to avoid being locked into some proprietary file format, and even though they were still technically “just Markdown files,” it was clear that I would have to do a lot of cleanup to make them readable to other apps that don’t use Dataview.
Pain Point #1: Cluttered Daily Notes Page kept me from actually writing in it.
Solution: No Daily Notes Page Template
Currently, I have no template set for my Daily Notes Page. It’s just a blank note every time I open it. I still use Drafts for quick capture, and I have a Drafts Action set up to append a given draft’s contents to the Daily Note corresponding to its creation date, but the only structural formatting I give my Daily Notes is a second-level header with a timestamp.
To keep things simple, sustainable, and somewhat future-proof, I aim to create dedicated notes for any Dataview queries I’d like to make, then I transclude them to the notes where I want to see them. All of these Dataview notes live in a single folder in my new Vault, quarantined in the event I move on from Dataview at some point in the future.
My Folder System Was Incoherent
Look, I got into this thing with the best intentions. I made folders based on things I thought went well together, and also based on where the source material originated. I’ve been using Scrivener for well over a decade and Ulysses for almost as long, so when I imported files from those apps, I remembered them best by the app where they originated. I also had project-specific folders, information silo folders, and folders that (I thought) would help me quickly and easily retrieve what I was looking for.
After about a year of constantly misremembering where I had stored a given file, I thought I would convert my folder hierarchy over to the Johnny Decimal System, dividing my Vault into ten top-level folders and ensuring each of those contained no more than ten subfolders. (This, of course, broke nearly all of my QuickAdd actions and templates, causing a bunch of secondary work getting them all functional again.) I was also really persuaded by the idea that having a specific folder hierarchy would give me something of a geographic memory of where my files lived in a truly spatial sense. The numbers keep the folders in order so you know what’s at the “top,” what’s in the “middle” and what’s at the bottom.
The problem was that I had trouble coming up with categories that were sufficiently specific and distinct—remember, every file can only ever belong to a single folder! I would get stuck with something like a writing textbook I was working through. Do I file it under “70 References/72 Annotated” or in my “100 Personal Projects” folder, since it’s both a book I’m reading and the source of a series of creative writing assignments? Where do quotes from other sources go, “70 References/71 External References” or “60 Information/63 Claims?”
It was hard enough to find the right place to store a note, then I had to go through the whole process again when I wanted to retrieve it! Where did I put that idea for a story, is it in “40 Production Writing/41 Premises” or “40 Production Writing/ 42 Postulates?” Maddening!
Pain Point 2: Incoherent folder system complicated storage and obfuscated retrieval.
Solution: Seven Folders for Everything
Really, there are four folders, and three of them function more as workspaces than anything. I wanted to believe that things could be as simple as “Projects,” “Areas,” “Resources,” “Archive,” but my life is not complicated enough to have enough areas to warrant a dedicated folder containing them. Also, I’ve never been entirely clear on the distinction between the “Resources” and “Archive” directories. As a result, I have a folder system that’s specific to my needs, and although aspects of it resemble the Johnny Decimal and PARA systems, it’s different enough that I can’t comfortably classify it as either.
Obsidian sorts folders alphabetically, so using numbers allows me to fix their order regardless of their titles. I named them to be as unambiguous and descriptive as possible—”Active Projects” is for projects I want to actively focus on, “Active Resources” are the notes that are currently relevant for those projects, and everything gets returned to the “Archives” when not in use. I’ll explain that “99 Temp” folder in the next section.
Only the three middle folders have any subfolders at all, and this is by design. I’m tired of looking for things everywhere. My “Journal” folder is subdivided by year; it’s purely chronological. The “My Writing” folder has subfolders for each distinct writing project, since I’m most likely to reference similar files related to the same project. “Vault Management” has purely maintenance folders like “Indices,” “Attachments” (any files that aren’t markdown or canvases,) “Templates” and “Dataviews.”
I get a lot of mileage out of that “Indices” folder. The main reason I was able to get rid of my previous extensive folder hierarchy was that I decided to start manually creating indexes of related files. It’s more work than using a plugin like Zoottelkeeper or Waypoint, which do the same thing automatically, but that extra work helps me cognitively associate those files with each other. When I let the plugins do all the work, I stopped connecting the ideas myself, and that’s the whole point of a personal knowledge system, right?
Now, squirreled away in the “Dataviews” folder is a note that looks for notes without any inlinks—just to make sure I don’t forget any—but that’s a failsafe, not a primary strategy.
I Couldn’t Readily Determine What I Was Looking at With Most Files
So far, these were small potatoes. It’s hard to create a bunch of different folders, but it’s easy to dump all your files into one “Archives” directory. It’s hard to trial-and-error different Dataview queries to dial in the “perfect” Daily Notes Page, but it’s easy to deselect the template in Obsidian’s settings. My goal in simplifying my folder hierarchy was to rely on my indices and the Search function to find what I was looking for—and this set me on the most time-consuming part of this whole Vault overhaul.
As you can see in the screenshot, I have 2,700 notes in my “Archives.” Some are ideas I wrote down, some are quotations or excerpts I found reading other people’s work, and some are entire articles I saved with a web clipper to highlight and parse out later. When I searched for a note, there was no telling when I created it—if it was years ago, before I even started using Obsidian, or just in the past couple of months—or if it’s even an idea that originated with me without opening a bunch of false positives and getting sidetracked from the reason I was searching for it in the first place. Worse, remember how I imported a bunch of files from Scrivener and Ulysses? I was going back and forth between those two apps while working on two different novels over the course of nine years with no consistent attempt at versioning. All told, there were roughly five hundred duplicated files I had to purge once I finally figured out which ones were redundant.
More descriptive note titles could only take me so far. What I did to solve this particular problem was far more drastic.
Pain Point 3: Inconsistent nomenclature and years of accumulation left me with inscrutable filenames that didn’t tell me what I needed to know.
Solution: Dates and Categories Encoded in the Filename
Okay, I’ll admit it—this is the one thing I wish I had done from the very beginning. It would have been so much easier to simply prepend each filename with the date it was created and the simple key I devised than to have to retroactively rename nearly three thousand files.
Plugins helped a lot, but it was still a woefully manual process. I used Linter to pull the creation date for each file from its metadata and stick it in the YAML via Obsidian’s new Properties feature. Smart Rename enabled me to rename each file manually, after which it would add an alias for the original filename to the Properties and go through the entire Vault renaming each link to it to display that alias. It wasn’t always clean—some of the files that came from Ulysses and Scrivener had inaccurate creation dates in their metadata, so I had to cross-reference them with my journal—but I was able to get every file that wasn’t in my journal assigned a YYYYMMDD date code.
Everything is so much easier now. I set the “Temp” folder as the default location for new files, and I use the Templater plugin’s “Folder Templates” feature to trigger the renaming process whenever I start a new note. It adds the date automatically in YYYYMMDD format and asks me which of five categories define the note. Again, if I had done this from the beginning, it would have been a cakewalk to maintain.
Credit for this approach goes to Douglas Barone’s File System Infobase Manager concept, which he adapted from a Scrivener user known as AmberV. My categories don’t match theirs exactly, but the concept is the same.
I consider every note as belonging to one of five top-level categories. I kept it at five because each has two primary subcategories, and I have ten fingers. The whole point here is to make things simpler, and I didn’t trust myself to remember a shorthand with more than ten variables. Each category is represented by a single letter, and its subcategories are described by a 1 (originated with me) or a 2 (originated elsewhere), with two notable exceptions I’ll mention in a moment.
C is for correspondence, like a letter or a draft of an email. C1 is outgoing correspondence, C2 is incoming correspondence. Simple.
N is an internal note, meaning it’s something I typed up myself rather than being something I copied or saved. N1 is a note that originated with me—it’s my own idea or question—and N2 is a note that I created in response to something else. For example, I have an N2 note about Doug’s “File System Infobase Manager,” because it’s not an idea I came up with—I just have thoughts about it.
R refers to records, whether those are events, indices, or discrete entities. R1 is an archived project (either completed or abandoned) and R2 is an index or other discrete entity (like a person, place, book, or organization). The exception to my subcategory binary is the bat (!) character—I use R! to indicate a project that is neither complete nor abandoned (but which may, nonetheless, not warrant inclusion in my “Active Projects” folder on the grounds that I am not, well, actively working on it.)
W is for my works of creation. W1 stands for nonfiction while W2 stands for fiction. Here, again, I use the bat (!) to indicate something I created specifically for use within Obsidian, like a specific Dataview note, a template, or a canvas. That way, if I ever have to move my files into a different system altogether, it’s one query to identify everything Obsidian-dependent.
X is for external notes. These might be standalone quotations, articles I clipped from the web, or other such works that originated elsewhere. X1 refers to such files that I have annotated with my own highlights or commentary, and X2 is simply an unaltered external work. I don’t have a lot of the latter, and what I do have are usually excerpts from books I’m reading on which I have yet to add commentary.
As convoluted as it may seem, having dates and this five-category shorthand allow me to instantly distinguish between “20231111-W1 My Obsidian File and Folder Organization System” (this file, a nonfiction weblog post) and “20231002-N2 File System Information Manager” (my note about Doug and Amber’s respective approaches to cataloguing) without even having to preview them. There’s still something to be said for concise and descriptive file names, but another advantage of this method is it puts all my thoughts inline with each other.
There’s something autobiographically satisfying about being able to see a train of thought developing over time from notes about an article to a new work of creation, the end result of which is right here on my weblog.
That only left one more small problem to resolve.
I Kept Forgetting the Conventions I Adopted for My Own Vault!
As the work grows, it gets harder to control; it is a lion growing in strength. You must visit it every day and reassert your mastery over it. If you skip a day, you are, quite rightly, afraid to open the door to its room.
Annie Dillard, “The Writing Life”
If I ever put Obsidian away for a few days, I would come back and just stare at my files a little bewildered and unsure of where to begin. Whatever I sat down to do, it was supplanted by the task of figuring out how to make sense of my Vault, and that began to grate on me.
Thankfully, I haven’t had this problem since streamlining my folders and meticulously naming all my files with the aforementioned conventions, but true future-proofing means making sure I—or my survivors, should I meet some untimely demise—had a clear way to interpret the contents of my Vault.
Pain Point 4: I kept forgetting my own organization schema and what I was supposed to be working on within my Vault
Solution: A Single File in My Root Directory Explaining Everything to Myself
“! 20230605-N1 About These Notes” is exactly what it sounds like—a note that explains my folder hierarchy and naming conventions, what’s “Top of Mind” (which is a section I update as frequently as I can to reflect my medium and long term priorities), and how “My Digital Notebook”—the name of this new Vault— relates to my analog notebooks sitting in a drawer in my study. Whether I’m looking to reorient myself after a day or a week without opening Obsidian or my dear sweet wife is left with only my son and my computer, having a simple, clear, and concise note explaining everything is a valuable resource.
Relatedly, I have also enabled the Homepage plugin and assigned a specific note with a couple of choice Dataview code blocks to appear whenever I open an empty tab. I call it my “Workshop” and it contains only a link to the current Daily Notes Page, a Dataview codeblock showing my four most recently modified files (in reverse chronological order,) another showing my next four tasks, and a row of buttons that allow me to quickly make a new task, search my Vault, add a quick time-stamped note to my Daily Notes Page without opening it, launch Obsidian’s Quick Switcher, or open an experimental note I made that presents a semi-randomized selection of notes from my Vault as if it were a social media feed to try to clean up my habits.
I’m Sticking to Rivers and Lakes From Now On
So that’s the whole kit and kaboodle. I’m done chasing waterfalls, rainbows, and big purple dragons. I don’t want to spend all my time tweaking my workflow or rearranging my files—I just want to follow my thoughts where they lead, find them again when I want to revisit them, and turn them into something meaningful that might have a chance at helping someone else. No more searching for perfection, just focusing on what works for me in the context of the work I’m trying to do.
[…] and establish conventions for adding to the pile that is sustainable and scalable. So that’s what I did. It took me a little more than two weeks, but I have finally cleared out the clutter of my digital […]