Bookkeeping with awk and Jupyter notebooks

Motivation

If Socrates had lived in the age of big data, would he be famous for saying "the unquantified life is not worth living"? In the second decade of the 21st century we already find it completely natural to measure with the utmost precision our distances walked, energy taken in through meals, oxygen inhaled through exercise, blood pressure and heart rate, all recorded automatically by the portable computers we insist on calling "phones" (despite almost never using them to conduct voice conversations across great distances) and other wearable devices. In the preceding decade, while software engineers were laying the groundwork for this trend, I had already allowed my personal record-keeping to encroach on such territory as cash transactions, SmarTrip travels, and music listening choices, but only with the primitive tools of pencil and paper. For easier post-processing of the data, I started making these entries in dedicated software programs, including Grisbi and Gnucash for the monetary transactions. When the lag time of X forwarding and the unavailability of such specialized programs on campus computer labs limited the use of such graphical interfaces to the ever-decreasing hours spent at home, I sought out a record-keeping system that would work within the text-based environment of a terminal window.

Implementation details

For each account, a tab-separated text file requires the following fields, using a separate line for each transaction (longer comments or unaccommodating text editor widths require you to turn off automatic line feed):

The awk script

takes the

account file

as its first positional parameter. The output of the script is a summary of month-by-month totals, as well as the current balances (what the bank knows about, versus what you know about).

The python script

processes a collection of files (one for each bank or credit card account), populating a vector of year-to-date totals for each expense category. The last cell uses matplotlib to generate a colorful visualization of year-to-date cash flow, sorted by category.

Transferrable skills learned from this project

Back to Extracurriculars