- data
- process
Organizing my files
For 30 years I have been generating digital computer files and storing them in various ways. Over time I created heirarchies of related information, much of it overlapping and without any formal naming convention. Methods evolved over time as did my interests and eventually I found myself searching for stuff that I knew I had just generated but could not find. Since I tend to constantly move from machine to machine even through the course of a day the situation was just becoming worse as I tried to back that stuff up and ended up creating duplicates.
Getting Organized
I was very happy to discover johnny.decimal which describes a methodology for reviewing and organizing files that really worked for me. The concept is pretty simple, and there's really no need to discuss it in depth here since its very well documented on that website, but suffice to say it forces you to carve up your stuff into discrete categories of things and in the process of sorting through all your stuff you discover and eliminate duplication and create a simpler indexing system.
I've now applied this method to my personal files for home as well as work which gave me the perspective of sorting through vastly different file stores and I can tell you that as a digital packrat this was no small undertaking. Overcoming the inertia of 30 years of filing things a certain way takes effort. There is a fear that you will never be able to find any of it again that you have to push through. My work and home data structure look very different of course because the content is very different, as would your own filing system be. The final organizational structure is only created by working the process and if your experience is like mine, you will be tweaking it for some time even after you think you're all done.
Since my main store of files (at home) is on a networked NAS which isn't always on, I still have the old problem where I'll create files on a laptop and then need later to back that up to the NAS. I tend to use rsync to get that stuff onto the NAS when I can and I'm hoping the new organizational structure which I am now using by default on all new machines will make it easier to sync stuff back.
Note taking and an index
In my case files are separate from notes and I have amassed an extensive amount of notes over the years. I was always leery of locking my note-taking up in word processor formats, and am glad of that now because after 30 years many of those formats are now defunct. While LibreOffice can still open pretty much any format - being able to operate on plain text is important to me. I've moved toward using markdown over the last few years but that is still plain text. Keeping my notes in markdown allows me to use programs that respect this format and I've tried a bunch. I'm writing this (in markdown) using Kate, an editor I've grown to like quite a lot since moving into KDE Plasma at home but my go to software for writing and maintaining what really is a database of notes usually is Obsidian. Yes, its a proprietary software but I've tried a couple of the open source alternatives and just really like Obsidian more.
All of my notes are just plain text files with a .md instead of .txt at the end. These are then recognized by Obsidian as markdown and indexed in its database. I can search, link, and syntactically highlight sections easily so that code snippets are formatted legibly in my notes. Since I tend to copy, compose, and archive a lot of code and commands in my notes this is huge.
My notes are organized and named exactly the same as my file structure (or at least thats the goal) so it serves as an index to find files or notes as the case may be. I decided to keep them separate folder structures so I could more easily check my notes into a git repo. This is mostly because I'm always using different machines and its just easier to check the notes in and out on all these different machines than try to syncronize them along with a huge file store. No, I'm not interested in using some cloud provider solution like OneNote or GoogleDrive for this - and the git repo is self hosted. I hope to write something up about that soon.
Verdict
I'm super happy with my new organizational methods. It's been as "zen-like" as manipulating a bunch of computer files can be and I'm able to find stuff much more quickly now. Also, I know where most anything new should go without having to think too much about it. Since I do work in the command line a lot, tab completion in getting to the right folder is also faster.