Xinye Tao | Blog - Collection - Résumé | RSS

Introducing My Knowledge Lakehouse

# non-fiction, 2024-01-13

1 There’re only three entities: log, list, structure.

1.1 Everything goes into the log chronologically. It is the source of truth.

1.11 We refer to the smallest unit of content in log as entry.

1.111 entry can be anything really: an idea, a memory, a piece of information, a progress update of a task, a TODO item, a reference to another entry

1.12 Immediacy and immutability matters. entrys are captured as they happen, never modified.

1.121 There’s no room for auditing or forgetting. entrys are an authentic reflection of reality.

1.122 Efforts should be put to optimize the latency of the capture.

1.1221 Sometimes the primary tool is not available. An “always-on” secondary should take its place.

1.1222 Sometimes log from different tools or locations need to be “merged”, “persisted”. The general principle is to reduce manual steps (e.g. use a Web App that merges automatically), and do them daily at a fixed time and place (e.g. push to a git repository before going to bed).

1.123 In this sense, log is very similar to Lab Notebooks (Sam Bleckley, 2020)

1.13 External context (if any) is part of the entry.

1.131 At the minimum, the title, author, and year of publish should be cited.

1.132 Better yet, use a personal URI system to reference local copies of the context, e.g. Zotero snapshot, image, book.

1.14 There can be different types of logs. Different logs should represent completely different domains. Thoughts in these domains usually happen at different times and places.

1.142 I currently have three types of logs: journal, reading, worklog.

1.2 list and structure are derived from the logs.

1.21 A list references multiple entrys in the log.

1.211 list is a flat collection of entrys, contains nothing else.

1.212 The order of entrys in a list may be important, may be not. This is to say, you can treat the list either as an Array, or as a Set.

1.213 I have some lists here, if you want to see.

1.22 A structure references multiple entrys in the log, in a hierarchical way.

1.221 The construction of a structure involves many thoughts that function as “glue”. They can exist outside the logs.

1.2211 Think of the “glue” thoughts as the data member names when programming a struct. A name represents a particular angle to view an entry, an incomplete slice of an entry’s whole.

1.23 list and structure don’t need to be exaustive in what they are trying to collect. Unlike logs, the creation and maintenance of list and structure are performed asynchronously.

1.24 It is idiomatic to create lists using automation. For example, use a search and de-duplication script to create a list of all unfinished TODO items.

1.3 Yes, it’s like a data lakehouse. list and structure are materialized views over the unstructured data stored in logs.

2 Reference to an internal entry is simply a pair of content and date.

2.1 The content is a copy or a paraphrase of the original entry.

2.11 Do repeat yourself, as contrary to DRY. Think of the content as a cache of the full entry.

2.2 The date is the creation time of the original entry.

2.21 The fact that unpacking a reference requires the reader to examine all logs from that entire day, facilitates re-discovery of old ideas.

2.22 References to an entry can be found easily with a search of that date. These references are commonly referred to as backlinks, or webmention in the context of netpages.

2.221 For logs that are not searchable (e.g. paper notebook), a searchable list needs to be created to bookkeep all references in them. It contains a set of date1 (date2), where date1 is the day being referenced, and date2 is the day this reference is created. By searching for date1, we know date2 is its backlink.

3 All logs are to be reviewed regularly, and randomly.

3.1 Through the course of re-reading, learn to remember past ambititions and melancholies. Learn to disagree with them, and in disagreement find a way forward.

3.2 Digital logs should be printed to physical books for the sake of reviewing.

3.21 Nothing compares to the feel of a physical book, especially when the purpose of reading is not to consume and complete.