22. Modules: the shape of a wolf project
Chapter 18 was about work the compiler does before your program starts. The rest of this part is about work you do before the compiler starts: deciding where code lives, whose code you are willing to build, and what either decision costs later. It begins with the smallest of those questions, which wolf answers with a filesystem.
A module is a directory. Not a file, not a declaration, not a line at the top of the file announcing what the file is called. You make one by making a directory; you import it by naming the directory; and what the directory does with its files inside is nobody’s business but its own.
22.1 Directory = module
Here is a two-module project. The entry file:
use stats
fn main() -> !int {
var widths = List[int]()
(mut widths).push(4)
(mut widths).push(6)
(mut widths).push(8)
print("mean {stats.mean(widths)} of {stats.count(widths)}")
0
}
and the module it imports, which is the directory stats/ and happens
to hold one file:
/// Summary statistics over a list of column widths.
/// The arithmetic mean, truncated; zero for an empty list.
pub fn mean(xs: List[int]) -> int {
if xs.is_empty() { 0 } else { total(xs) / xs.len }
}
/// How many widths there are.
pub fn count(xs: List[int]) -> int {
xs.len
}
fn total(xs: List[int]) -> int {
var t = 0
for x in xs { t += x }
t
}
$ lupin main.lu
mean 6 of 3
Three things in that pair of files are decisions, and the third is the one this section is about.
use stats names the directory. There is no path and no extension to
keep in step with the disk: the import says which
module, and the module is the folder. A module’s name is where it sits.
pub is the whole visibility vocabulary you need here. mean and
count carry it and are visible to importers; total does not and is
not. Between them sits pub(pkg), which exports an item to the other
modules of your own package and to nobody outside it.
And total is inside the module, which is why the third decision is
invisible until you go looking for it. Move total into its own file:
fn total(xs: List[int]) -> int {
var t = 0
for x in xs { t += x }
t
}
Nothing else changes. The entry file, the import, and mean stand as
they were, and mean goes on calling total as if the two were still
typed one after
the other:
$ lupin main.lu
mean 6 of 3
That non-event is the design. File boundaries create no scopes and no
import edges: the module’s namespace is the union of
its files, and a reorganization inside a directory is not an event any
importer can observe. Languages that make the file the unit force you to
choose between a file you can navigate and an interface you can keep
stable. Wolf does not put that choice in front of you, and the price is
that you cannot use the file as a privacy boundary: total is private
to stats, not to summary.lu.
Two consequences follow immediately, and both are errors rather than conventions.
Because sibling files are one namespace, two of them cannot define the same name. It reads as a surprise the first time, because nothing imports anything:
fn describe() -> str { "from extra" }
with a describe already in the entry file, and the objection lands
where the second one sits:
$ lupin main.lu
main.lu: E0302: the name `describe` is defined twice in this module (defined again in `./main.lu`); file boundaries create no scopes (D32) — two separate programs sharing a directory each mark themselves `//! member: false` (D59) [mod.dup] at 1:4
And because pub is the export, asking for something private gets an
answer that respects your time:
$ lupin main.lu
main.lu: E0304: `total` exists in `vault`, but it is private; only `pub`/`pub(pkg)` items are visible across modules (D32) [mod.vis.private] at 4:19
Read that diagnostic twice. It does not say the name does not exist. A resolver that answered “no such item” would send you hunting a typo that is not there; this one tells you the item is real, the objection is visibility, and which two spellings would change the answer. The difference between those two messages is ten minutes of your life.
One more error belongs in this section because it surprises people arriving from anywhere else. An import you do not use is not a warning:
$ lupin main.lu
main.lu: E0305: the import `tools` is never used in `./main.lu`; an unused import is a hard error (D32), and deleting the line is machine-applicable [mod.use.unused] at 1:5
Wolf takes Go’s position here, with Go’s justification: an unused import is a dependency edge that slows every build and means nothing, and the fix is a deletion a machine can perform. A warning would be a request.
Exercise 22-1 (fingers · lupin). Build the two-module project
above from scratch: an entry file and a stats/ directory exporting
mean, with a private total the entry never sees. Run it. Then move
total into a second file inside stats/ and state what changes for the
entry file.
Exercise 22-2 (comprehension · lupin). vault/keys.lu defines
pub fn count(), pub fn loaded(), and private fn secrets() and
fn total(). The entry calls vault.total(). Predict the diagnostic
(including whether it says the name does not exist) and the exit code.
Exercise 22-3 (comprehension · lupin). twice/main.lu and its
sibling twice/extra.lu each define fn describe(). Neither file
imports the other. Predict the verdict, and say why “neither imports the
other” is a trap in the question.
Exercise 22-4 (comprehension · lupin). The entry imports tools
and never mentions it again. Predict: warning or error, and what the
diagnostic offers about the fix.
Exercise 22-11 (fingers · lupin). Two scratch programs, one
directory, on purpose: sum.lu totals three numbers, widest.lu
finds the longest of four words, and each is a whole program with its
own main. Mark each //! member: false and run both by name. Then
remove the marker from one of them and run the other. Why does the
diagnostic land on the program that kept its marker?
Exercise 22-12 (comprehension + extension · lupin). The labels
module is two files — upper.lu and banner.lu — and both define
pub fn title. Predict the diagnostic before running (22-3 is the
same law; what differs here?). Then fix it in a copy without deleting
either file, and say what the fix’s choices were.
22.2 No cycles
Two modules, each with a reason to import the other. store writes rows
and asks index to publish them:
/// The row store: writes rows, and asks `index` to publish each one.
use index
/// Store one row and publish it.
pub fn put(n: int) -> int {
index.note(n)
}
/// How many rows the store will accept at once.
pub fn batch() -> int { 64 }
and index publishes rows and asks store whether they are valid:
/// The row index: publishes rows, and asks `store` to validate them.
use store
/// Publish one row.
pub fn note(n: int) -> int {
if n < store.batch() { 0 } else { 1 }
}
/// The index's fan-out.
pub fn width() -> int { 8 }
Neither import is silly, and that is how real cycles are born: two modules each needing one true thing from the other. Wolf refuses anyway, and draws the loop it refused:
$ lupin main.lu
main.lu: E0303: this import completes a cycle: `store` → `index` → `store` (in `./index/index.lu`); imports between modules must form a DAG (D32) [mod.cycle] at 5:5
The arrows in that message are the whole refactor brief. Look at what
each side wanted: store wanted a publisher, index wanted the batch
rule. Only one of those is shared vocabulary: a fact about rows, not a
service. So it moves into a third module that imports
nothing:
/// What `store` and `index` needed from each other, and nothing else.
/// The largest batch either side will accept.
pub fn batch() -> int { 64 }
/// Whether a row number is inside a batch.
pub fn in_batch(n: int) -> bool {
n < batch()
}
index now consumes kinds instead of calling back into store:
/// The row index: publishes rows, and decides validity from `kinds`.
use kinds
/// Publish one row.
pub fn note(n: int) -> int {
if kinds.in_batch(n) { 0 } else { 1 }
}
/// The index's fan-out.
pub fn width() -> int { 8 }
and the arrows form a DAG:
$ lupin main.lu
stored 0
This is interface extraction, and it has one discipline that decides
whether it worked: the extracted module holds what both sides needed from
each other, and nothing else. If kinds starts importing things, the
tangle is reassembling under a new name.
Three things the rule buys, in the order you feel them.
Builds. A DAG gives every module a finish order, so compilation parallelizes and an incremental build has a frontier. A cycle collapses its members into one unit that recompiles together, forever, because nothing in it can be finished before the rest.
Comprehension. In a DAG, “what does this depend on” is a question that terminates. Inside a cycle everything depends on everything, and the forty files you would have to touch to break it were already one file wearing forty names.
Interfaces. E0303 forced us to name the shared vocabulary. kinds
exists because the compiler would not let the dependency stay implicit,
and a named seam is where documentation, tests, and ownership attach.
That is the rule paying for itself: the refactor it demanded is the one a
reviewer would have asked for.
There is a fourth, and it is mechanical. The thing importers depend on is a module’s exported surface, and the compiler will tell you exactly what that surface is and hash it:
$ wolf interface ./summary.lu
module pkg :: std.range
wolfi v0 · toolchain 0.2.29 · edition v1
export_hash d7c8e2cafbb418b680a777f06602e91128373f7ed3b341a20edccaa108e513bf
pkg_hash d7c8e2cafbb418b680a777f06602e91128373f7ed3b341a20edccaa108e513bf
deps: (none)
items:
[0] pub clamp_to — fn clamp_to(r: prelude.range[int], x: int) -> int ! {none} · regions (ρ1, -) -> -
[1] pub collect — fn collect[T](r: prelude.range[T]) -> prelude.List[T] · regions (ρ1) -> ρ_caller
[2] pub contains — fn contains(r: prelude.range[int], x: int) -> bool · regions (ρ1, -) -> -
[3] pub is_empty — fn is_empty(r: prelude.range[int]) -> bool · regions (ρ1) -> -
[4] pub len — fn len(r: prelude.range[int]) -> int · regions (ρ1) -> -
module pkg :: (root)
wolfi v0 · toolchain 0.2.29 · edition v1
export_hash c8e04b6da198d815eeb6be3fabd7eb6e3ca36b0cb85d505e4f29e9c33f0b78a6
pkg_hash c8e04b6da198d815eeb6be3fabd7eb6e3ca36b0cb85d505e4f29e9c33f0b78a6
deps:
std.range d7c8e2cafbb418b680a777f06602e91128373f7ed3b341a20edccaa108e513bf
items:
[0] pub count — fn count(xs: prelude.List[int]) -> int · regions (ρ1) -> -
[1] pub mean — fn mean(xs: prelude.List[int]) -> int · regions (ρ1) -> -
The listing opens with a module that is not yours. mean asks
xs.is_empty(), the standard library beside wolf answers that method
from std.range, and so std.range is listed with its own interface and
the root names it under deps:. Then the root: two pub items, and
total nowhere in the listing, because private surface is not part of
the interface and so cannot be part of the hash. The hash covers what the
file exports and what it uses, a std module reached through a method
call included, so the same file reads another number on a wolf with no
std/ beside it. That number is the honest answer to
“is this change visible to my importers”: if it does not move, they
cannot tell. Splitting a file does not move it. Renaming a private
helper does not move it. Chapter 25 spends that fact on semantic
versioning, where a hash that did not move is the difference between a
patch and a lie.
Read the line above the hash while you are here. toolchain 0.2.29 is
which compiler printed the page, and it is not part of what was
hashed. The distinction is worth a sentence because the alternative was
tried: while the compiler’s own version went into the digest, upgrading
your toolchain moved this number on a package nobody had edited, and a
hash that moves when nothing was published is a hash that cannot answer
the question it is for. The stamp is printed and the interface is
hashed.
Exercise 22-5 (comprehension + extension · lupin). In tangle/,
store imports index to publish entries and index imports store to
validate them; each import has a reason, which is how real cycles are
born. Predict the diagnostic. Then perform the interface-extraction
refactor in a copy: move the shared vocabulary into a third module
neither imports from, and run the result.
Exercise 22-6 (comprehension · prose). A library refactor splits
one 900-line module file into four files in the same directory, moves
nothing across module boundaries, and changes no pub markers. List
everything that changes for the library’s importers, then name the
artifact from this section that would prove your answer mechanically.
Exercise 22-13 (spelunking · wolf). 22-6 argued from the export
hash; now hold it in your hands. Run wolf interface on a two-module
project’s tokenizer, add a private helper to the file, and run it
again. Two things to explain from the output: why the hash did not
move, and what the W0313 warning beside it is asking for.
22.3 No life before main
Wolf has no code that runs before main; the static initializers,
module-level constructors, and linker-ordered registration hooks of
other languages do not exist. A program’s first act is its
first line. Module-level const, let and var exist, and they keep
the rule rather than bend it: the compiler evaluates every module
initializer while it builds the image ([mem.static.3]), so the value
is in the binary before the program starts and there is no order of
initialization to observe.
That rule deletes a whole genre of bug and, with it, a whole genre of
pattern. The genre is registration: a plugin system where each module
announces itself into a global table on the way up, so that by the time
main runs, the table is populated by everyone who linked in. It is a
good pattern with one soft spot, which is that it is a set of writes to
shared state in an order nobody chose.
Wolf’s replacement is a value:
struct Ingest { rows: int }
struct Report { rows: int }
struct Purge { rows: int }
comptime fn handlers(a: type, b: type, c: type) -> str {
"{typeinfo(a).name} {typeinfo(b).name} {typeinfo(c).name}"
}
comptime fn expect_three(a: type, b: type, c: type) -> bool {
assert(handlers(a, b, c).len == 19)
true
}
fn main() -> !int {
const HANDLERS = handlers(Ingest, Report, Purge)
const CHECKED = expect_three(Ingest, Report, Purge)
print("{HANDLERS}")
if CHECKED { 0 } else { 1 }
}
$ wolf run registry.lu
Ingest Report Purge
handlers is chapter 18’s tier doing chapter 18’s job: ordinary wolf,
evaluated during compilation, taking types as arguments and reflecting
their names. What lands in HANDLERS is a finished table. The running
program does not build it, does not lock anything to build it, and cannot
observe a moment when it was half-built, because there was no such
moment: the table was assembled on the compiler’s clock and the binary
carries the answer.
The ordering question was deleted, not answered. There is no “before main” in which two registrations race and no link order to depend on, and two builds of this source produce the same table byte for byte because comptime cannot read anything that differs between them.
The registry also gets something the init() version never had: a claim
the compiler settles. expect_three asserts the table’s shape at compile
time, so a fourth handler that nobody wired up is a build failure rather
than a mystery at run time. Break the claim and the build stops with the
assertion that failed:
struct Ingest { rows: int }
struct Report { rows: int }
struct Purge { rows: int }
comptime fn handlers(a: type, b: type, c: type) -> str {
"{typeinfo(a).name} {typeinfo(b).name} {typeinfo(c).name}"
}
comptime fn expect_three(a: type, b: type, c: type) -> bool {
assert(handlers(a, b, c).len == 26)
true
}
fn main() -> !int {
const HANDLERS = handlers(Ingest, Report, Purge)
const CHECKED = expect_three(Ingest, Report, Purge)
print("{HANDLERS}")
if CHECKED { 0 } else { 1 }
}
error[E0710]: this comptime assertion failed
--> ./s2.lu:13:5
|
13 | assert(handlers(a, b, c).len == 26)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluated to `false` at compile time
...
19 | const CHECKED = expect_three(Ingest, Report, Purge)
| ----------------------------------- while evaluating `expect_three`, entered here
| ----------------------------------- while evaluating `main`, entered here
|
= note: a failed comptime `assert` stops compilation — it is the witness mechanism for facts the
checker cannot see on its own.
What init() could do that a comptime registry cannot is exactly the set
of things this rule is glad to lose: arbitrary effects, at an unspecified
time, before anyone could have asked for them. Everything else (the
table, the dispatch, the check that the table is complete) is a value,
and values are the thing wolf is good at.
Exercise 22-7 (comprehension · wolf). The init() idiom this
section retires: a plugin system where each module’s init() registers a
handler into a global table at startup, in whatever order the linker felt
like. Write the comptime replacement for four handlers, with a witness
that fails the build if one goes missing, and say what became of the
ordering question.
Exercise 22-8 (design). Import cycles are errors (D32). A colleague argues the compiler should permit cycles and merely warn, citing a large codebase where breaking them means touching forty files. Argue wolf’s side using what the rule buys, then concede the strongest point on the other side and answer it.
Exercise 22-9 (extension · lupin). The word counter, split along
its seam: a tokens/ module that turns text into words and knows
nothing about counting, a tally/ module that counts a List[str]
and knows nothing about lines, and an entry that owns the text and the
report loop. Build it and run it. Then defend the seam: why does
tokens return a List[str] instead of taking tally as an import
and counting as it splits?
Exercise 22-10 (extension · lupin). A calculator whose
arithmetic lives behind a seam: an ops/ module exporting one
function, apply(op, a, b) -> int ! {BadOp, DivZero}, and an entry
that parses a b op lines and reports. Build it, with 9 0 / among
the inputs. The row crosses the module boundary — what does the entry
know about why a line was refused, and what would it take to start
caring?
Exercise 22-14 (design). 22-9’s tokens exports one function;
its counting sibling keeps index_of private; 22-10’s ops exports
apply and nothing else. State the rule these three choices follow,
then argue against the tempting alternative: why not export the
helpers too, since a future caller might want them? Name what every
pub costs its module under 22-6’s hash, and when the answer flips.