Alright! Hi everyone! Happy EmacsConf!
I am so excited to be here.
It's surreal to be part
of the conference itself
in addition to being a viewer.
EmacsConf is like Christmas to me
and I am so excited when it comes around every year.
Today, my talk is on a programming methodology
that I've created, discovered, stumbled upon.
I call it Book Club Tapas.
Before we get into that,
let me introduce myself.
My name is Mattie Sullivan,
and my pronouns are she, her.
I go by the handle ElephantErgonomics,
which is shortened down to ElephantErgo in the IRC.
You can reach out to me after the talk
for questions, comments,
or just to say hello by reaching out
to hello at elephantergonomics.com.
So this software development strategy I found.
It's inspired by literate programming and agile.
So what exactly is Book Club Tapas?
Book Club Tapas is a conversation
that you have with yourself.
It's a log and a ledger
of your intentions, hopes, dreams,
and what you've learned
over the course of development.
Book Club Tapas is an oracle
you can consult
about the state of and the strategies
behind your development process.
Book Club Tapas is also a peer programming partner
that helps you decide how to best put forward your efforts
and how to best pull together
what you're working on.
Book Club Tapas will also help you to understand
how to tailor scope to your needs
and how to have the best parts
of your program shine through clearly.
Book Club Tapas consists of two parts,
Book Club and Tapas,
but what does that mean exactly though?
Book club is a reverse literate
development strategy.
Book club is a time for you to write,
and then read and reflect.
It's like a book club,
but it's for your program.
Instead of inserting narration into your code
to narrativize what you're accomplishing,
you are instead inserting snippets of code
into your narrative to make it come alive.
So what are we narrativizing, exactly?
What sort of story are we telling?
Book club is the story of you, your program,
and how your efforts are allowing your program
to come into the world.
Software developers naturally have to wear
a lot of different hats
and take on a lot of different roles.
We apply ourselves into a lot
of different contexts.
We do research, interface architecture design, mathematics, philosophy.
We take in the world around us
and then build abstractions to model it.
We translate the abstract
into the concrete,
and then when we're trying to teach software how to be smart,
we translate the concrete
back into the abstract.
I can't help but feel like so much of what makes software development difficult
is just trying to remember
and keep track of everything.
We have to try and remember
so many different implementational details.
We have to remember how our own code works,
how the API of our dependencies work,
how relevant real-world constraints behave,
what the standards lay out,
and how our data structures are laid out.
When we're debugging,
we simultaneously have to remember
how our program is currently behaving,
as well as how the program ought to behave
in order to get a chance
to reconcile that gap.
It's honestly all way too much.
We need a ledger of what we're actually doing
in order to stay sane.
I think a really effective way to make sense of things that are complex and important
is to narrativize them,
to turn them into stories.
This is a strategy that humans have been using for a long time.
Mnemonic devices, metaphors,
and drawing parallels
are all different ways of doing just this.
Telling stories helps us to understand
things that are big and complex
by grounding them in our own experience
and making it fit into our scale.
So because the way that everyone
naturally tells stories
is going to be a little different,
because the details that strike us as important and worth focusing on
are going to be different for different people,
I'm not going to say
that there are hard and fast rules
about how Book Club should work,
because how it should work is however it best fits your needs.
Different people and different projects
have different backgrounds and mindsets.
And I don't think it's my place to say what strategy is correct as a universal law.
You know, because Book Club Tapas is, after all,
just something I've sort of stumbled into.
Book Club is intrinsically ad hoc.
My providing a prescription of strategy
is basically going to begin and end with the idea
that you write a reverse literate document
that illustrates how you've gone
about writing your program.
But. All of that being said,
I'm going to talk about
how I've laid out my book club files
and why I think this is a solid place
from which to get started.
So my stock off-the-shelf suggestions
for just getting started
is to have sections for our overarching goal, our development goals,
a place for scratch work, a test suite, research,
And then finally sections for variables, functions, and macros.
So we have our starter kit sections.
How do we go about using them?
How do we get started?
Well, we write them, you know,
out in our org document,
but then what do we do?
We start by writing what we know.
We have a spark, a vision.
We had the beginning of an idea
of what we wanted our program to do.
Alternatively, maybe we had
a client layer goals out.
Either way, we have some idea
of how we want our program to be shaped.
Let's start by writing that down.
What are we trying to do?
What is our goal?
After that, we're probably wondering to ourselves,
okay, we have our goal,
but how do we get there?
That's when we start writing
our development focuses.
If we have bursts of intuition
about what functions to write,
questions that we want to answer through research,
we start enumerating those every time they hit us.
Our goal is to write them
all down in a checklist
in order to turn them from daydreams
into courses of action.
If we aren't having development focuses
hit us right away, that's okay.
If we just stare at the goal for long enough,
I think it's inevitable
that the Muse will speak
and we'll get a clear lead
on a path forward.
So now what?
Now that we have our development focuses,
we want to go ahead and create
the rest of the headings for ourselves
so we can act upon them.
We go ahead and write the rest
of the file structure ad hoc
in a way that will serve our needs for now.
If it's not fitting us well later on,
we can just go ahead and change it.
There's no pressure.
That's the beauty of having this
all be in a plain org document.
If we're doing something consistently,
we probably want to have a heading for it.
We'll go ahead and create homes
for our variables, our functions, our macros.
We'll want to create a spot for scratch work
to sort of like stretch our legs
and lament in a stream of consciousness
sort of format about how
a particular piece of design ought to work.
Basically, anytime we wear a different hat
or we take on a different role as a developer,
it's worth considering creating a category for it.
The best way for us to figure out
what headings to fill in
and how to fill them in
is to just go ahead and act upon our development goals.
If we have a question we want to answer,
we'll want to create a research heading
so we can go ahead and have a spot
for a scratch work for reasoning things out.
If we want to write the first draft
of a function we want,
We'll want to create a heading for functions
and then a subheading for that function in particular.
So now that we've filled in our sections,
what do we do now?
Our idea for a program
has been turned into a story,
but what does that actually get us?
To me, a lot of what's exciting about Book Club
is that novelization goes in
and a peer programming partner comes out.
As we loop through reviewing our document,
as we scan it up and down,
we're able to engage in conversationality with our past self
because of how verbose
we've been in our notes.
We can ask our past self questions
and get back answers.
We've turned our past self
into a peer programming partner.
If we're wondering what to do next,
we can check our development focuses.
If we're wondering how something works,
we can read documentation
embedded in our function drafts.
Or we can read the outcomes of tests
that we've performed in our research.
We can ask ourselves questions and get answers.
Some of what's most exciting
about peer programming to me
is having fresh perspective
and alternate context.
We have a fresh set of eyes
on the program that aren't our own.
And with that set of eyes
comes someone else to share the burden
of trying to remember everything.
With Book Club, instead of having
a peer programmer
that exists in physical space,
we have one that's, to get all sci-fi for a moment,
reaching forward towards us
from backward in time.
We're asynchronously working
with our past selves
as an equal role collaborative
partner in development.
We have their perspective,
their fresh memories of the code as it was written,
and their focus on what was worth worrying about
at a different point in time.
We can ask them questions and get answers.
We can ask them questions like,
well, what do I do now?
How does this data structure work?
What types does this third-party library take?
By asking these questions,
I can even stay fresh
on development progress
that I last touched months ago.
It's really easy to duplicate work,
forget how things work,
lose track of priorities.
Book Club helps keep us focused.
It keeps us accountable.
It even keeps us company.
One of the most immediately useful things about Book Club,
in my opinion, is that we immediately have a list of actionable items.
Every time I have a little pain point,
I go ahead and write it down,
and I write down all of the things
that would be nice to have done someday.
So you might be wondering,
and it's fair to wonder this,
isn't this effectively just the GitHub issue model?
We're listing out bug requests,
issue requests, feature requests.
It's not exactly a new idea,
and it's pretty intuitive.
I think the important consideration here
is that having really formalized apparatus
for entering in our thoughts
can be an unnecessary source of friction.
Bug listings don't tend to be
a great fit for daydreaming
or verbose considerations of philosophy.
Bug listings tend to be reserved
for catastrophes.
I feel like a lot of the tooling
that we currently use
really struggles with creating ergonomics
that make taking frictionless notes difficult.
We have systems where all the disparate
parts of what we're working on
feel really far away from each other.
We're pushed away from engaging
in conversations with ourselves
as a result of how disparate
all of our tooling feels,
how the process of working with it
is incongruent.
My hope is that we can instead
engage with a process
that makes it really trivial
to write impulsive journaling
about what we're doing.
So much of design is ultimately
just daydreaming.
Good ideas tend to strike us hard
in a momentary flash of inspiration,
and then they fade just as quickly.
Anyone who's had an idea all at once
in the middle of the night
knows that they're going to have to choose
between either committing to writing it down
or accept that by morning
they'll have lost it.
If we're not writing
what strikes us as important
at the same moment that it's happening,
we're going to lose it.
It's not realistic to expect ourselves
to hold onto our ideas forever
with the same precision
as when we were first inspired.
Okay. I'm gonna call you out real quick.
If I ask all of you, who wants to read
really excellent documentation,
I imagine that everyone here
is raising their hand.
We want code to make sense,
and we want to know what
the original developer had in mind.
Even the original developer themselves
would want this just for their own sake.
I know that for me, I can even feel
things becoming less fresh
just after a couple months away
from my codebase.
And that was me from a couple months ago.
They're not around anymore.
Now, here's the rough part.
Here's what I'm really gonna call you all out.
Who wants to write really excellent documentation?
Now, I don't know what's happening on your end,
but I'm imagining crickets,
silence, tumbleweeds
blowing through to the horizon.
It's a tough ask.
It's not generally all that rewarding.
If you're writing docs from scratch,
a lot of it involves relearning
the intentions behind crusty old code.
For me, it hurts to not spend that same time
implementing bug fixes and new features.
It just doesn't feel like
a great use of my time.
Even if it's strictly for my own code base
for my own use, it's hard to sit down and do it
even when I know how much I would benefit from it.
My thinking is that when you write rough,
piecewise daydreaming as you go,
it's so much easier to not only
begin writing documentation early in your process,
but also to stay consistent about not slouching into
an accumulation of a backlog.
So not only does writing documentation early
make us more likely to keep that habit going,
but it also makes the documentation
we do write way more robust.
When fiction meets reality
and we start writing out code
that is constrained by the real world
and not just our imagination,
we learn that things we assumed about our design
aren't going to work out in practice.
Because of this, we can enter
into a sort of situation
akin to boiling a frog in a pot of water.
Frogs don't notice that they're being boiled
if the water is only heated gradually enough.
We decide to adjust our design only a little bit
without changing the documentation right away.
Doing that once is fine,
but I don't believe for a second
that we're only going to do it once.
We can find ourselves surprised
that as time goes on,
our code looks nothing like our spec,
and we lose the thread of what our code
was supposed to do in the first place.
When we stake our intentions clearly and early,
You ground yourself in them.
You reduce the risk of straying from them.
You have clear reference
for what you want your code to do,
and you reduce the risk
of having its purpose shift over time.
When we take turns alternating
between writing code and documentation
rather than acting, you know,
as having it all as one step,
we risk taking turns just moving
our goal post back and forth.
So we've seen how book club files get us
all sorts of amazing features
and practical benefits.
But we might be starting to notice a pattern
as we continue to engage in conversation
and work with our document
and watch it grow in size.
We originally created our book club file
with the hope to reduce what we would need to keep track of
and to reduce our level of overwhelm.
We might find that as our book club file grows,
We're encountering more detail
than we can practically parse, manage,
and decipher intention from.
It can be easy to enter into a situation
where we're drowning in the breadth of our notes,
and in doing so we've recreated the same problem
we originally set out to solve.
Writing out every single detail helps us a lot
to make sense of things at first,
but then after a while we can encounter
a signal-to-noise problem
when we try to make meaning from too many details.
This is where tapas come in.
So tapas in Spanish cuisine are appetizers.
What's notable about tapas
is that you can bring a bunch of them together
to make a full meal.
In the context of book club tapas,
they serve a similar role.
The idea is that we write flavorful libraries
that together form a full program.
We have a full program,
but it's made from discrete modules.
The idea behind Tapas is that instead of creating
one perfect, solves-everything codebase,
we want to create a whole bunch
of separate libraries
that themselves nail a specific subdomain.
And once these libraries
are all brought together,
they form the whole that we're seeking.
Once our book club file becomes big enough
such that we feel like our scope can be split
into multiple libraries,
that's when we want to take the opportunity
to split our program up into parts, into Tapas.
So, maybe one of the best ways
to understand what makes a good Tapa
is to first examine what does not make a good Tapa.
The single most important thing
to understand about Tapas
is that they themselves are substantial.
There's a lot of back and forth
on the idea of microlibraries,
their merits, their dangers,
and when and where they kind of work best.
I think the distinction
that I would like to draw
is that I think that tapas belong in the larger end
of scale and complexity for microlibraries
rather than the smaller end.
I think particularly small helpers
like npm's is-odd
are a good example of something
I think does not constitute a good tapa.
Meanwhile, I think Python's request library
is a really good example of a top-up.
I believe requests only does HTTP connections,
but I feel like that's not so simple and straightforward
that you can just go ahead and implement it
on your own real quick.
A real danger of creating
helper libraries that are too small
is that we don't remove abstraction
nearly as much as we postpone it.
If our libraries are small,
but the glue code that binds them is large,
we haven't done anything
to reduce complexity
or employ abstraction in a meaningful way.
If all of the complexity exists in our glue code,
we've simply replaced our functions
with libraries of the same size and purpose.
Our codebase is still monolithic
instead of having meaningfully divided scope.
I think that a good tapa
ought to feel like augmentations
or extensions to the standard library.
You know, maybe something kind of
akin to Scheme's Surfy system.
I think that the goal of good tapa s
is not to solve a particular problem,
but instead to solve a particular class of problem.
The goal of a well-written tapa
is to solve needing to do hard work in general
rather than solving what can only really be
an individual need
of an individual program.
I feel like tapas are most helpful
when we instead seek to solve
a larger overarching problem
that intersects with the problem space of our codebase.
When we have a handful of tapas
that are roughly the same size and scale,
the glue code that marries them
is also roughly the same size and scale.
As a heuristic, I try to aim for any function
being approximately 3 calls in length,
and then any tapa being between 6
and 12 functions in length.
The number of tapas themselves
can be as many or as few as you need,
but then your tapas can split into
their own separate tapas as needed.
My hope is that the collection of our tapas,
especially as we create
dependency chains among them,
is that each next tapa is a trivial case
of the one prerequisite to it.
Every tapa is a meaningful
human-readable abstraction
that enables us to feel confident about our tooling
without drowning in detail.
The whole stack can be understood by humans,
but we only have to focus on
any one piece of it at a time,
rather than focusing on the entire stack all at once.
We can practically achieve
a huge final product,
but each individual step
in working towards that goal
is still at a human scale.
One thing I want to make sure to point out,
one thing I want to make sure
to point out explicitly, real quick,
is that having access
to a hygienic macro system
like the ones that we have in Lisps
makes for an amazing experience
for creating tapas.
The types of abstractions that we can do
by modifying syntax at compile time
makes for incredibly intuitive
and ergonomic tooling.
So we've talked quite a bit about
what I think makes a tapa good,
but I think maybe the best way
to understand the concept
is to have a look at the whole workflow in practice.
I've been working on this currently
unnamed Elisp program recently.
It's a validator for the file tags lines
of my Org Mode files.
So I have Org Mode files
under my directory.
I want the headers of my org files to be tagged
in accordance with the sequence
of the names of the directories.
I do this by having a file text line at the top of the file
just list the path segments in order.
If I have an org file in the directory
Yeah, tilde, documents, foo, bar.
The file tags line has the tags foo and bar.
This is totally fine to do by hand,
but I want a program
that recursively searches through my directories
to validate that the tags are correct
because it's easy to drop something.
This scale of problem is actually kind of perfect
for demonstrating how book club tapas work in action.
We have a problem
that's mostly rather simple,
but it has a lot of moving pieces.
We want to iterate over directories recursively.
We want to do string manipulation.
We want to parse buffers
and we want to edit buffers.
All of these tasks are simple enough on their own,
but it's deceptively easy
to start tripping over ourselves
when we feel like it's necessary
to do all of these different things in one step.
So there are a ton of great string
manipulation tools for Emacs.
So that's checked off,
that's done, taken care of.
I'm still kind of daydreaming
about writing a wrapper around
some of the Emacs standard libraries
for directory traversal,
just to make it a little bit nicer to work with.
But the big thing
that really struck me as odd
is that there doesn't seem to be a great tooling
for destructuring Emacs buffers
beyond just chaining together
a bunch of editor commands.
Emacs is so buffer-oriented,
I feel like it really deserves a good library
for programmatic buffer destructuring.
I looked around for a bit,
but I couldn't really find anything.
So at the end of the day,
I could definitely just grit my teeth
and put my head down and just use tools
that feel cumbersome to work with if I wanted to.
I could write something
that's good enough
just for the purpose of my package
and then hide it deep inside the code base.
I could absolutely do that.
But I can't help but think about how
after I properly write the tooling I'm missing,
I'm really going to be thanking myself
in terms of reduced implementational complexity,
reduced bug hunting, real reusability,
and ultimately really just a deep sense of pride
in knowing that I took the time
to do something in a way that feels right.
This right here is the perfect time
to split off tapas.
Anytime that we find ourselves
reaching for a fictional dependency,
wishing that someone had written
a library like this...
We can take that opportunity
to remember that we are someone.
We can write that library ourselves,
and we deserve to write that library
because we deserve to get to use it.
So I'm going to briefly show
a book club buffer
for a program called Squint.
It's the buffer destructure
that I've been talking about, and it's real.
It's a wrapper around
Emacs's narrowing functionality
and regular expression search.
It's not totally done,
and we'll likely see some breaking changes.
But I really like where it is.
I'll be posting it in its current state
on some of the big source repository sites
relatively soon.
I think it has a good feature,
which is really quite exciting.
And it'll likely probably get split off
into its own Tapas.
We'll see. No matter what,
I do recommend being on the lookout for it
because I think it'll be
a really excellent demonstration
of some of the solid ideas
behind how to get rolling with Book Club Tapas.
So I have my background section
where I'm basically just sort of laying out,
you know, what the objective is for the program.
I have my vision where I'm doing
some daydreaming about, you know,
how this all ought to work.
I date stamped this.
As you can see, it's from a while ago,
but I still have the full context of, you know,
all the things that I've done working on this.
I listed out a bunch of ideas
for different forms for functions macros.
I did different pieces of research.
Yeah, I was trying to figure out
for the width restriction macro,
what types does it take?
And I did a whole bunch of tests
to try and ultimately figure it out.
Because it claims in the documentation,
I believe, that it will just take
any type for labels.
But in my testing, that's not
ultimately what I found.
The results of my tests
is that symbols, numbers, they work.
Strings do not.
I'm not sure why that is.
But for my purposes,
this is what I need to know.
I have my development focuses here.
So I have my assorted goals
for different directions
I want to take the program.
And then lastly, I have my functions, my macros.
And this right here
is the titular macro.
This is ultimately the big meat
of the program.
And it's all contained happily organized
inside my book club file.
I'm quite happy with it.
I think it looks really nice.
So what else does book club tapas do?
I don't know. It probably does a lot of stuff.
It does all sorts of stuff
that I don't know about yet,
but this is where you come in.
I'm really excited to see what people do
when they take these ideas
and run with them.
And if you have something really cool you're doing with it,
please email me and come talk to me about it.
I'd love to hear about it.
Again, my email is hello
at elephantergonomics.com.
So last, before we wrap up,
I want to go ahead and give
a quick plug for my services.
I am an independent software engineer
that has an emphasis in backend design
and general automation.
In particular, I have an emphasis
in that really cool new generative AI thing
that everyone's been talking about recently.
If you have a headache,
you have some sort of pain point
for your small or large business,
you wish you could just wiggle your nose and have disappear,
come talk to me.
I'll make it disappear. I love doing that.
Reach out to me at hello
at elephantergonomics.com.
If you think that book club tapas
would be a great fit for your team and your project,
I'd love to hop on and help you
get the ball rolling quickly.
Go ahead and email me at hello
at elephantergonomics.com.
Lastly, if you're a member
of the larger Lisp community
and you want to fund independent software development
for things that really excite you,
for passion projects
that make our ecosystem richer,
I'd love to look into accepting independent funding
so I can commit more hours
toward making that happen.
Some of the projects
that I want to work on
are a Python foreign function
interface for Guile Scheme,
A framework for rapidly creating simulation games
that feels just as simple as writing Emacs configurations.
I want to work on getting
a full graphical web browser
inside of Emacs.
And I want to finish programs like Swind.
These are just some of the projects
I want to work on,
but I need funding to do so.
If you want to see these things happen,
send me an email at hello
at elephantergonomics.com
with both your intention
to pledge a monthly contribution
as well as clarification,
a sort of vote on which project
you would like to see me prioritize.
I would love to have folks reach out
for any of these reasons.
I would just love to talk to you.
Thank you so much for watching.
I really hope that the talk was interesting,
and I'm really excited to see
your thoughts and questions
right now in the Q&A.
Thank you so much for watching.
Bye!