My Notebook System

As I noted above, I started this system with the cheap (under 1 USD) spiral-bound notebooks that can be found in every gas station or grocery store in North America. I pulled the pages out as I transcribed them, so the notebook got thinner and thinner as I used it. Unfortunately, the metal spiral remained bulky and would even snag on clothing.

The Field Notes notebooks have always been weirdly pleasurable objects for me. I’ve suffered from the "this is too nice to write in" problem with notebooks over the years, but I managed to get over that with Field Notes. Now I see them as a fun utility. I even managed to start using the subscription limited editions when I accidentally ran out of the regular kraft paper notebooks…​and I’ve been working off of a couple year’s worth of limited editions ever since!

I started with pencils. I used them for years. I tried several dozen different mechanical and woodcase pencils. I used them all to complete physical failure.

Woodcase pencils are: reliable, foolproof, ubiquitous, and a total pain to sharpen in the field. I carry a pocketknife, but you can’t always be whipping out a knife to whittle wood shavings at every venue.

Mechanical pencils are: always sharp, carry a ton of graphite, can be really nice, and will eventually fail in the punishing conditions of the pants pocket. The best mechanical pencils I ever carried were the Caran d’Ache 849 (carandache.com) models. These are very durable, attractive, and a great size for pocket carry. I eventually destroyed two of them.

Graphite doesn’t run when it gets wet, is erasable, and you can write upside down if you have to. But graphite does get lighter and lighter as the pages of a pocket notebook rub against each other. And erasing is way slower than just crossing out a mistake, which is what I do now that I’ve switched to pen.

Tried many pens before I settled on the Pilot Juice. The .38mm ball was a complete revelation for me. I’ve always considered myself to be a medium or even bold writer. But in a pocket notebook, the thinner line lets me fit tiny writing at the bottom of a page or in the margins when I need to amend an entry - or even sketch a surprisingly detailed drawing for a new project idea in a couple inches of space. It’s the first pen I’d ever used with a sub-.5mm ball that doesn’t feel scratchy or produce skipped lines or anything of that sort.

The Pilot Juice is an incredible pen for pocket notebook use for several reasons: it’s lightweight, it’s extremely reliable, the ink dries quickly and there’s lots of it, and the pen body has been very durable (though I’ve seen hairline cracks towards the ends of their lives). And the icing on the cake is the excellent spring-loaded clip! In the past, all pen clips have failed me - even the spring-steel ones eventually lose their shape. But this cheap plastic pen can grip the full 40+ pages plus cardstock cover of a notebook in my pocket and survive for months and months and months. I’ve been spoiled by the Juice’s clip and it’s now my gold standard.

In short, the Juice may not be the most elegant writing tool I’ve ever carried, but I no longer care. It performs like a masterpiece.

The idea was to parse these times and categories, total them, and get a sense for where my time was being spent each day.

As chance would have it, the above entry actually captures the creation of the first tool I wrote to extract the info: a TCL script that simply gave grand totals for any given category. Getting spans of time from the timestamps is pretty easy. Turn everything to minutes by multiplying the first two digits by 60 and adding the second two digits: 0630 becomes "390 minutes since midnight". Do the same for another number and subtract to find the difference: 0630 to 0715 is 435 - 390 = 45, so an activity that started at the first timestamp and ended at the second (some other activity is starting) lasted 45 minutes (which you can easily verify at a glance).

This is one of many, many advantages of 24-hour times as opposed to the abomination of AM/PM. And don’t even get me started on date formats (xkcd.com)

Timezones also presented a challenge. Imagine you’re going on vacation for a week. Do you write your entries in "home time" and translate every entry, or do you start writing entries in local time and translate later…​or neither and try to fudge the data later? Yeah, it sounds silly, but imagine you’ve been scrupulously keeping track of every minute for a couple years. This sort of thing bothers you. For a brief time, I even experimented with calculating times in a canonical UTC form and setting timezone changes for a block of entries with an encoding (TIMEZONE: +7).

I wrote several generations of report tools (most of which exported to HTML for pretty graphs and word clouds by frequency of use). These were always fun to look at. I discuss more about "tooling" later.

For the first three years, all entries went at the top of a single text file. The original log.txt file spans from May 2014 to October 2017 and has 46,000 lines.

As I mentioned in the beginning, the nature of attempting such a hardcore logging method was that once I started, it required constant vigilance. If I missed even a single entry, it could make it hard to piece together the rest of the sequence of the day.

The consequence of this was that I quickly built up a habit of pulling out the notebook and making entries all the time. After a lifetime of wanting to "get better" about journaling or note-taking, I was suddenly doing it dozens of times a day.

The result was two-fold:

A paper notebook is a wonderfully handy thing to have around, but it only works if you fill it with stuff. Don’t be afraid to throw anything and everything in one. You can always separate the wheat from the chaff later. And notebooks are (usually) cheap.

My first scripts simply summarized the time categories in various ways. I’ve had terminal output with neat columns of data and pretty HTML reports.

For a while, I was also keeping a directory alongside the log with one small image for every day. The images might be photos or screenshots, but mostly they were little drawings. A script collated those with the daily log entries and create a sort of digital scrapbook where each day was a tiny thumbnail image which could expand into the full entry (and full-sized image). That was pretty neat.

I’ve written a couple search tools. I was even looking into a dedicated full-text fuzzy search system based on Apache Solr, but decided that was not the sort of tool I was willing to support in the long run.

These days, I just have a Bash alias that searches the log (and my wiki) with ag, The Silver Searcher (github.com).

But the truth is, it’s often surprisingly hard to search the log. Sometimes I’m just certain that I’ve written about something. And I search and search and nothing comes up. Finally, I narrow it down to a date range and brute-force read the entries until I find it. And darn it, there it is, and I somehow didn’t use any of the keywords I had tried. It happens all the time and I have no idea how to solve it without some sort of agent with natural language intelligence.

To tackle the findability problem from another angle, I’ve tried numerous tagging systems to help categorize and search, but I’ve yet to stick with anything systematized and thorough enough that it could be used for ad hoc reporting or automation.

Another thing I’ve tried to do with the log system is track goals both big and small.

One of my favorite systems scanned for items that started with "TODO <keyword>" and email me the list of all items. An item would be automatically struck from the list when I entered another entry with "DONE <keyword>". That worked really well until something changed in my setup and the emails stopped being sent from my local machine to my mail host.

I’ve also tried, unsuccessfully, to enter a goal and track its progress (percentage of completion or simply time spent) in later entries. So far, my attempts have all been too "heavy" and they all fell under their own weight and often just served to discourage me when it turned out I wasn’t spending my free time the way I’d originally thought I "should".

Of course, there’s a larger issue of finding the right balance between spending your free time relaxing, or following whatever new passion strikes your fancy, or pursuing more lofty lifetime goals. But that’s surely outside the scope of even this sprawling page.

So what I currently do is write immediate TODOs as I discover them. Then I copy over the outstanding ones into the next day’s page. I do this until they’re done. Or I give up on them. This is very much like the Bullet Journal system, as I understand it.

For the longer-term goals, I write them on the last page of my notebook. When I start a new notebook, I copy the outstanding goals to the next one.

These two levels of reminder work pretty well.

As for tracking things I’ve done, see the discussion about "wrap-ups" later.

I’d love to find a way to add future calendar events in my paper notebook. But nothing has stuck yet. The closest I ever got was having a trimmed index card with all future events written on it, which I would move from day to day. Since I was forced to interact with it on a frequent basis, it reminded me to look at it. But since it was just one card, I had to write events as I got them, not the order they’d take place. So it became increasingly hard to scan for events at a glance.

Storing dates on paper either means having space allocated for every day in advance (and hoping it’s enough space!) or writing deadline/appointment entries as they come, which means they’re not going to be in order and that pretty much guarantees you’re not going to see the right one on the day it’s due. (Like a dentist appointment that’s scheduled six months in advance).

So, like it or not, I use the Google Calendar app and web interface to keep track of future appointments and such. This is simply something that works better on the phone for me. :-(

I’ve had funky ideas about tying my aspirations and interests to some form of "natural time" like moon phases to ground my whole system to something outside of the artificial constructs of digital clocks and calendars (not to mention having a free, universal reminder right up there in the sky!)

I would also love to cycle between indoor/outdoor activities and distribute my creative energies between endeavors such as writing, art, and programming in a way that makes me feel like I’m not short-changing any one area for too long. It seems like a seasonal plan would be really interesting. Maybe when the birds migrate back in the spring, I would start doing woodworking projects or something like that? Who knows. It sounds neat.

My most recent stab at time tracking was creating a hardware timer with color-coded glowing keys for time categories and an LCD display with minute totals. It’s based on a Raspberry Pi Pico microcontroller. I call it my "multitimer" and it works really well.

Check out the multitimer project here.

The huge advantage of the hardware device is that it takes a split second to tap a key to switch tasks (it also keeps track of the totals, eliminating the entire tedious and error-prone transcription process!). That’s important because I recently wanted to track chaotic distractions. Most of the distractions were short, just a couple minutes. But sometimes they would stretch into half an hour. And there was no way of knowing which kind it would be beforehand!

To make matters worse, I was trying to track two different projects, and often switching between them.

Trying to write down the task switching wasn’t working because the distractions would come without warning and needed to be dealt with immediately. Trying to figure them out after the fact (or worse, at the end of the day), resulted in a lot of guesswork and I often found that the numbers just didn’t add up. It is always disturbing to lose account of my most precious of resources: time!

I really wanted to get to the bottom of it with a definitive record. As a bonus, maybe I could even debug my perception of time while accounting for it.

The multitimer really does meet this need. It works!

The big disadvantage of the multitimer in its current incarnation is that it’s a bulky build intended to live on a desk. I didn’t build a power supply, so it also has to remain plugged into a USB outlet to function. So it’s completely impractical unless you’re okay with using it only while tethered to a small area.

A micro version of the multitimer (almost certainly with a custom PCB) that fits in pocket or on a wrist would be awesome and way more practical. I’m sure regular surface-mount tactile momentary switches would be fine, particularly if they were somewhat recessed. And the big LCD could be replaced with a tiny OLED display that comes on only when needed. Frankly, I’m still incredulous that nobody has build such a thing. But I guess cell phones have eaten the market for a lot of wearable hardware. Let me know if you know of one!

These are fantastic! They’re a chance to celebrate success, re-visit ideas I had during the week, and pluck the important stuff from the entries to make something more closely approaching a narrative. If there was a pattern or theme during the week, it’ll appear now.

What’s really wild about the weekly review is how it helps put daily highs and lows into perspective. I know this is a really, really obvious thing to say. But it’s amazing how a terrible week turns out, when I can see the it as a whole, to have had just two crummy days. The rest were all good or even great. I wonder if doing this enough will eventually help me put those things in perspective while they’re happening?

2023 Update: I had no luck keeping up with the weekly wrap-ups. I still think they’re a great idea and I’m sure they’ll work for other people. However, I am doing monthly wrap-ups this year and find them to be no problem at all since I have a whole month to complete one before the next month is complete and I’m "behind".

2023 Update: As mentioned above, I’ve removed the weeklies entirely. It turns out that creating monthly wrap-ups wasn’t as big a deal as I’d made it out to be. In fact, at the end of March, I still hadn’t done the February one. So I wrote them both and timed myself. They averaged only 16 minutes each. No problem! So most of what I had to say about the monthly wrap-ups here has changed.

2024 Update: I stuck with it and made my first yearly wrap-up: Wrapping up 2023, plans for 2024.

I have less experience with these, I’ll admit, but the concept is the same as the weekly. The time scale is just bigger. Instead of filtering the big events from the daily entries, now I’m filtering the big events from the weeklies. Patterns become even more apparent and everything is put in a larger perspective. As I said, I’ve managed to complete some of these in the past and it felt amazing.

The weekly wrap-ups are essential for me to accomplish the monthlies. If I had to read through a month’s worth of daily entries, it just wouldn’t get done. Oh sure, I could do it once or twice. But it would not be sustainable. Heck, some days I barely muster the will to transcribe my log from the previous day!

But there’s this irksome thing about trying to use weeklies to construct monthlies. This has has always bothered me and please tell me I’m not alone. Weeks don’t start and end on month boundaries! There are 52 weeks in a year and 12 months in a year. Any way you try to slice it, there’s just no way to get those 4.3 weeks per month to work out neatly.

What this means is that if you make your wrap-ups fall on a traditional Mon-Sun (or Sun-Sat) week, you can’t really use them to make the monthly wrap-ups!

Let’s take the end of August, 2022 for example:

The weekly wrap-up for the week ending on August 28 is fine. You can use that for the August wrap-up. But the last three days, 29-31, are not yet summarized. "Well, that’s no problem," you say. "Just add the interesting stuff from those last three days into the monthly wrap-up." And you’re right. That’s fine.

The real problem comes when you try to do the September wrap-up. Now you’ve got a week that was already half accounted for in August. To really do it right, you’ve got to cross-check September’s wrap-up to avoid duplication on the overlap. Do do it right, you’ll pretty much have to re-visit the dailies to get the dates right. Annoying!

After agonizing over this for longer than I’d like to admit, I’ve settled on 7-day wrap-up "weeks" that start when the month starts. Now let’s see how August and September work out with this system:

The last week wrap-up in August is guaranteed to end before September starts. I’ll just need to include anything big from the last couple "extra" days, but that’s easy to remember because they’ll be the ones I most recently transcribed. September starts with a fresh new week wrap-up with no overlap.

All months get exactly four whole "weeks" plus three or less "extra" days at the end. It ain’t perfect, but it’ll do.

(Update: And now that I’ve dropped the weekly wrap-ups, this problem has disappeared entirely.)

I have yet to actually do one of these. But the idea is to continue the filtering process to the next stage: the "big stuff" from the dailies go into the weeklies and the big stuff from the weeklies go into the monthlies, and then the big stuff from the monthlies goes into the yearlies. The perspective keeps widening at each step.

I’m hoping a yearly review helps me see the bigger picture. I want to hone my sense for which projects to take on and whether or not I’m actually making progress on the things I want to accomplish with my time on this planet.

Lots of people make New Year’s resolutions and we all joke about them. I have often thought in terms of, "what I’m going to get done this year." Like most folks, my track record so far is…​not great.

There’s a quote that apparently nobody famous ever said (quoteinvestigator.com) in this exact form, but I like the sentiment anyway:

If I can get a handle on my months and years, then maybe I can even start to tackle the scope of decades.

When comparing old log entries with my memory, I trust the log as the truth. I know how fallible my memory is! Over time, my memory is reshaped by what I’ve written (presumably to become more accurate!) In some ways, every entry is a way to shape my memory of the event.

My early transcriptions were 100% verbatim what I’d written in the notebook.

Then a move, Covid, and an onslaught of other challenges made life difficult and I got a little out of my transcription routine. I got caught up. Then I lost it again. And again. Somehow I’ve found myself with over 300 untranscribed entries in chunks spread over the last three years.

(I’m trying to tackle this by transcribing two entries per day: one recent and one from a gap in the past. But that’s not going great and at my current rate, it’s going to be another two years before I’m caught up. Sobs.)

Where I’m going with this is that I found myself transcribing hard times from three years ago. And, gosh, reading and writing those entries brought back three year old pain and frustration like it was yesterday.

And I just can’t see the value in revisiting those negative emotions now or in the future.

Also, since I have children, I’m cognizant of the fact that someday I may be leaving this record to them. (Whether or not they would ever be interested in actually reading it is hard to say.) How much should I censor myself for their sake?

I don’t really want the kids to read my old grievances - things I’ve already mostly forgotten - and think, "Gosh, Dad sure was an anxious, angry guy!"

Well, first of all, everything must go into the notebook. I think that’s really important for a capture system. The paper notebooks have got to be able to accept all manner of foul language and angry scribbled drawings. That’s sacred.

But for the final record in the transcriptions, it seems like there’s a balance to be found:

So I’ve given myself permission, when I’m transcribing, to be factual about what’s happened. But I’m leaving out the emotional first-person prose that brings it all back.

I’m leaving out: "Biff made another sly remark about my dinosaur and I HATE HIM SO MUCH! I’ll get my revenge! Now everything is crap!"

Instead, I write: "Really frustrated with Biff. I let it bother me for the rest of the day."

I keep whatever part I think actually matters. I just take the heat out of it.

Now the log has given me yet another opportunity to see things in perspective and learn from them. Not fuel to rekindle old angers over petty crap that really doesn’t matter in the long run.