RTK increases token usage instead of reducing it
As the industry shifts from TokenMaxxing to TokenMining, several approaches have appeared to offer significant reductions. One of these is Rust Token Killer (RTK), with advertised reductions varying from 60 to 90%!
To understand why it would provide savings, consider that during coding sessions your agent performs many tool calls. The output of said calls is made available to the agent, consuming more input tokens.
When we look at the output, made for humans, we can see that the actionable information could be represented with fewer characters.
That is the premise of how RTK works. It registers as a PreTool hook that rewrites the actual (eligible) commands before the agent sends them, with altered/more compact versions.
The problem with such bold claims usually is twofold:
It usually assumes all shell commands are sent using the Bash Tool call
It does not mention quality issues (as a result of unknowingly stripping valuable information)
Like many things, you can’t make informed decisions unless you measure it yourself or have independent benchmarks that look at the control group and the proposed solution.
JetBrains released an interesting study that did exactly that!
And surprisingly, the results showed the opposite outcome: RTK increased the cost instead of driving it down.
Their findings:
Out of the baseline transcripts, only 1 in 3 commands were actually executed using the bash tool. And out of those, the output that could be compressed by RTK was 20% of the total tool output.
The median cost increased by 7.6%. While no single cause was found, they found that even when the output of the tool call was compressed, the RTK-backed execution led to more steps, explaining the higher cost.
At higher reasoning, RTK did not lead to a cost increase, but also did not lead to savings
Using RTK did not impact the quality of the output
RTK’s methodology assumes unrealistic scenarios to count as savings (e.g counting input tokens at full value, instead of a cheaper cached version)
The point here is not to give RTK a hard time, but to remind us to be careful to measure the real gains in your context.
Discussing HTTP Integrity
HTTP is a popular protocol that enables communication on the Internet as we know it today. It is often perceived as a deceptively simple one, but that could not be further from the truth.
Want an example? Look at its capabilities to provide integrity controls for the content it transports. Integrity enables the receiver of the information to detect if it is the same as the sender intended it to be.
But how can HTTP deliver that? The solution evolved:
Content-MD5
The first option added a header to the transmitted content that used an MD5 digest of the message body. This is accomplished by taking the content and passing it through a hash function.
While it worked, it had some limitations with range requests, where we request a partial resource instead of all, and the collision profile of the MD5 hash function.
Digest
It defined two things that up until then were considered the same: the instance (the complete representation of a resource), and message-body (bytes actually transmitted in a particular HTTP message).
The digest is calculated over the complete representation. Confusing? Let’s look at the simple case.
We are requesting one resource, and we receive it all. The digest is calculated using the instance, which matches the message body in this case.
But when we use ranged queries, things are different: the digest still remains the same, reflecting the full resource.
The avid reader will notice that we can use different hash functions (and even include multiple different functions).
Repr-Digest and Content-Digest
So the previous two options approached the integrity at different levels and with limitations: Content-MD5 only supported MD5 and applied to what is currently being transmitted; Digest represented the entire resource, even if it was partially transmitted.
The latest approach deprecates both and creates two fields, one defined to represent the integrity of the bytes currently being sent (the message body) and another for the full resource.
And if we ask for a compressed version, we would get different digests.
With the latest version, we cover both cases and can better detect transmission issues on both the request and response sides.
When paired with TLS or QUIC we have both integrity and protection against corruption or malicious modification.
Curious to know more? Check this article that covers the above and more, including the use of signatures to improve the protection levels and a cool demo page.
Functional Event Sourcing Decider
Object-oriented programming surged in popularity and promised to solve all the problems we had with development. That did not happen, fueling the growth in popularity of functional programming alternatives.
While I believe there are benefits in combining both (see the functional core, imperative shell pattern), many solutions are presented only via object-oriented examples.
That is why I found this article very interesting, as it presents a functional approach to event sourcing using the decider pattern.
To understand it, we can think of our system as something that responds to inputs (be they actions or triggers), uses some sort of persistence to hold state, and has interactions with the outside world (outputs).
In this representation, the code in the subsystem should be composed solely of the system logic, aka the business logic. In the functional approach, the subsystem is then represented as a pure function.
This is important because we isolate the business logic from external dependencies, making it more predictable (given the same current state and action, expect the same new state and side effects to be performed externally.
With the decider pattern, we leverage this and separate further the decision making from the state mutation. In it, we have the command and a state with a decision process that, if valid, will produce an event.
So the Decider is a function that takes the command and a state and returns events but no longer mutates the state.
It depends only on the command and current state, without any side effects. We then have an evolver, which is responsible for taking the current state and an event, returning the new state.
When we put the two together, we have the basis of event sourcing in pure functional form.
From this point on, you would add an imperative shell containing the interactions with the outside world: calling external dependencies, saving or retrieving the events from the event store.
Since we are using event sourcing, the current state is obtained by obtaining the events from the store and using the evolver.
The article includes more details, including an F# implementation, but you can also implement it in other languages, such as TypeScript.