Sedulus

jint, an extensible semantic linter

Julian Cruciani ·

Source code and install instructions

Programming languages do two things at the same time: they instruct the computer and they convey intent to humans (and now, language models). This means that we have at least two different dimensions when looking at code, intent and instruction.

For validating and verifying machine instructions we test, compile, run static checks and benchmark. For communicating intent, we use naming conventions, comments and documentation. Unfortunately, it’s harder to measure and verify the latter. As opposed to metrics (time, resources) and values which measure quantifiable and deterministic properties, semantics measure qualitative properties.

Semantic dimensions

Because each dimension has its own characteristics, each one requires unique tooling and units of measurement. Consequently, a tool designed for verifying instruction should not measure intent, just as an intent-measuring tool should not evaluate performance. While a tool built for one dimension can technically function in another, a poor fit inevitably introduces friction. We must avoid forcing a form onto an incompatible entity.

As humans, we see data from different perspectives. Something that for a process is an input, is an output for another. When building software, this contrast appears when the perspective changes. In terms of a client-server architecture, a request holds the same payload in both systems, but they carry different meaning once they enter one system and leave another. In my opinion, this difference in perspective is what makes reading software hard. We might know the theory of what a server, a client, a protocol or a transport is but we do not know the hidden requirements and the moving parts of the thousands of implementations out there, particularly if we’re seeing it for the first time, something that’s specifically true in the agent era. To read software, you need context that is not within the code itself.

Each implementation contains its own nomenclature, its own understanding of its implementation, because it is tied to the context it exists in. This is why agents need guidance. A semantic linter is that guidance, just in time. A context injection mechanism that enforces semantic rules at a given point in the agent’s lifecycle, reminding the agent of what dimensions we care about, within all the dimensions it was trained on.

This is why I’m releasing jint. A tool that uses classifier models to keep the semantic dimension entropy in check, helping the agent measure its own work in the semantic realm as it performs coding tasks.

A function has thousands of dimensions

Take an ordinary Go function definition. It has a name, a receiver, a parameter list, a result list, parameter types (or their absence), error paths, concurrency behavior, performance characteristics, naming implications, among others. Each of those is a dimension you could think about, and each dimension has many possible values.

The dimensions of a Go function definitionA cloud of two thousand dim dots represents the thousands of dimensions of a function definition.parameter typesresponsibilityreceiverarityerror pathsresultsfunc (s *Store) GetUser(ctx context.Context, id string) (User, error)

A function definition holds thousands of semantic dimensions. We only care about a few, but the LLM doesn't know which ones matter to us.

Notice that two of the dimensions point at the same subset of words. Arity and type are orthogonal, different dimensions of the same subject: parameters. Arity tells us how many parameters exist while parameter types tell us what they are. Not only is the function polysemantic, their components are as well.

Of course, this is only a subset of possible dimensions. We must remember that overlooked dimensions do not cease to exist simply because they were forgotten or skipped. Flawed API designs, fragile abstractions, and autonomous agents acting unpredictably are all direct consequences of overlooking dimensions we, or the agent, didn’t consider fully.

I think of jint as a dimension reminder for the agent.

Results

For these files I instructed my agent (custom coding agent with GLM 5.3 Flash) to use jint with all the defaults that come today with it in a fresh context subagent. To properly test the CLI, I had to make sure the agent doing the work had no more context than what jint was giving it, in order to “isolate” and properly measure jint’s impact on the output.

The file rewrite took more than 10 minutes (of course, this varies per model and inference provider, and whether the model has something to pattern match in the codebase or not). The amount of time it took to rewrite it and the way the LLM behaved as it edited and “jinted” said something: the model understood good practices, but it had a hard time generating code that followed them. It tends to get lost in the dimensional space as it generates more output.

Now, it’s also important to say that best practices not stated in the rule became apparent, like naming effects or splitting big components into multiple components. The rewrite came from the model’s activations based on the wording used by the rules, more than the rule itself explicitly indicating the required shape to pass the rule. However, if this is the result you wanted, you could create your own custom rules for .tsx files and state those requirements explicitly. This would involve calibration, which is something I write about below.

How does jint work?

jint compares inputs like code blocks and files against a set of user-defined rules in a tight form.

It’s built as a CLI so an agent can invoke it over and over on the same file or lexical unit as it makes changes to it, and adjust based on the results. I think this is why jint is valuable: it’s a consistent semantic sanity check.

jint: rules and a raw diff in, clean code outA rules card and a before diff feed jint, a semantic linter in the middle. The before diff carries 3 lint errors; the after diff comes out with zero.BEFORE3 complexity errorsRULESnames state their purposetrim comments that mirror codekeep units small and cohesivejintAFTER0 complexity errors

jint receives a code block against a set of rules and fails until the rules are met

Mechanism

Using Tree-sitter, jint parses files and compares AST nodes against a set of user-defined rules. Tree-sitter turns a file into nodes and jint abstracts similar node types into a cross-language lexical unit: functions, methods, types, bindings, or comments bound to the node it claims to describe, if any. jint then feeds one node (or one explicit pair) to the model with the rules that apply to that node, defined in the configuration.

The binary ships with several Tree-sitter WASM grammars by default: tsx, ts, js, go, python. Each grammar makes the binary a few KB bigger on average, so the option to make all grammars optional is being considered.

Defaults

jint contains a set of default rules considered global across languages and codebases: complexity measurement, file naming, comment drifting, grepability, among others. These defaults can be overridden and new rules can be added, allowing granularity and customization through include and exclude rules. Think of a rule as a reminder of the dimensions you care about in the code.

export const will_comment_drift: Rule = {
  // the name of the rule to be used, verbosity is better for agent readability
  name: "will_comment_drift",
  // exactly like jev, you can set scores or nouls as the rules to enforce
  instructions:
    "Does any comment pin specific artifacts by name such that renaming or removing them silently invalidates the comment?",
  criteria: {
    true: "At least one comment pins specific artifacts by name - files, functions, parameters, constants, types, paths - such that renaming or removing that artifact silently invalidates the comment while the code keeps working. The claim is coupling between the comment's wording and identifiers, not comments already wrong today.",
    false:
      "No comment here is coupled to specific artifacts by name: every comment describes behavior, rationale, or constraints without pinning identifiers, so renaming or removing code can never silently invalidate it.",
  },
  // targets a specific AST node (definitions, bindings, comments, etc) and files
  target: "comments",
  // a declarative handler
  handleData: {
    warn: { greater_than: 0.55 },
    error: { greater_than: 0.7 },
  } as const,
  // or a callback-shaped handler
  handle: compileNoulHandler({
    warn: { greater_than: 0.55 },
    error: { greater_than: 0.7 },
  }),
  include: [],
  exclude: [],
};

jint also allows multiple configurations within the same project, as you can set the configuration to load through a CLI flag.

For now (will change this in the next versions) jint only works with your Typesafe AI key, picked up from your env.

At what stage should jint be triggered?

jint is workflow agnostic given that it is a CLI. You can run it in your pre-commit hooks or you could ask your agent to use it whenever it finishes editing something, as a post-edit hook. The problem with the former is that errors and warnings flagged by jint won’t be discovered until later. The problem with the latter is that it could annoy the agent as it’s executing its task, making the process tedious and making the agent ignore the results. I would advise using the former approach, so rules are not agent-specific, but repository-specific.

Why should I care about code dimensions?

A codebase that follows good practices, similar patterns and is agent-friendly is cheaper and easier to debug and maintain. Identifying the semantics that your agent misses over and over with linting rules helps maintain a codebase with the specific dimensions you care about, without you being in the loop. The agent will perform and behave better, as it won’t have co-located polluted context that skews its generation. It could still generate it, but less. A lower entropy codebase produces less entropy as it grows.

Think of it as automating model corrections within an agent’s generation path.

Calibration

One very interesting domain jint opens is that of the calibration of rules. As semantics are not precise, there’s variance across runs. Two identical inputs can yield results that differ by ±0.05, and changing a single word or the chosen primitive can change the run result by a wide margin. It’s imperative to iterate on the rules until you get the result you’re looking for.

Because of this, we need to make both the CLI output and SDK ergonomics friendly for agents.

Question strategy

All rules embed a question, a criterion Jev has to take into account when measuring something. This criterion is not easy to get right, because semantic meaning carried by code has to be interpreted in a way where rules are abstract enough to tackle the dimensions we want to measure and where rules are not overfitting a specific state.

Iterating on the sensitivity of different states

Arriving at the default set of rules that are shipped here took me a few days. Choosing the question shape, content and AST node granularity (finding whether comments should be coupled to their functions, whether to return statements, bindings, definitions, or all three) wasn’t simple. It took me hundreds of tests to measure abstract enough rules that could be applied to several open source repositories. Luckily, Jev is cheap.

This calibration and iteration process asked for a way that helped me visualize the input sensitivity of different question shapes.

UI

For us mere mortals, jint comes with a UI that renders the list of runs executed per directory, grouped by rule, file and request. This is useful to measure the variance of each rule applied to its state and see exactly the request made to the classification model. You can get the raw output of a run or you can compare it in the UI.

Request strategy

Given that source code has thousands of static elements and that Jev only has a 32k context, batching requests to the Jev API is a must.

jint requests are batched per element under analysis. After running lots of tests with different state shapes and rules, I reached the conclusion that the best results are obtained when state is isolated. A compound state whose parts were keyed with a unique identifier averaged question results, even if the questions had scoped state. I tried this compound approach and it did not work well. The only approach that worked is isolation: multiple questions against the same state do not pollute question results.

Goal

I think we’re going to stop reading code at one point, and software engineering will be harder, more abstract than what it is today. We’ll research and think more about abstractions, but we will write and read less syntax. I don’t think this will be a consequence of the models getting better only, but the tooling for models and humans getting better as well. The goal is for this to be one of those tools.

What’s next

System theory says that a system is more than the sum of all the parts. That emergent property, the delta that’s hard to visualize and measure, is where all the value of software is. This tool only reads parts, it doesn’t and cannot read the system as a whole. The question is how we build the relationships hidden in the code so we can measure them and form a cohesive system, easy to understand and evolve.

If you’re interested in this kind of work, send me an email at julian@sedulus.tech

References