The History of Postgres Sharding
As your application grows, whether it’s a SaaS or not, talking about sharding is bound to happen.
This article provides an interesting presentation about how this topic evolved, specifically for the Postgres ecosystem.
I could relate to it thanks to my experience with the LAMP stack, where MySQL was the persistence of choice due to many factors, including ease of use and a “rich enough” feature set.
Postgres always had more features, including some more enterprise-focused ones, which carried a steeper learning curve and potentially higher operational costs.
That combo likely kept MySQL as the OSS solution of choice, with Postgres a niche, leading to quicker advances in the sharding area for MySQL than Postgres.
With Postgres, application-level sharding was the main solution. In it, you encode some logic to split the traffic, and your application refers to this recipe to find out
This solution is simple at first but has some drawbacks:
Rebalancing becomes yet another problem you need to create tools for yourself
You have to make it in a way where cross-shard queries are avoided
The first advancement in this area was PL/Proxy in 2007, which let you define routing functions in a proxy database.
You would query this proxy database by calling this function, which in turn would call the real shard.
For example, if you had the users table and the data spread using the username as criteria, you would define this function
And then call it from your application
This solved the issue of the application having to host the logic, but it created an unfortunate dependency: application code (where new access patterns live) and database operators (maintaining/installing the functions) would have to be synchronized.
Citus appeared in 2016 as a Postgres extension to solve this with a different approach. In it, there are coordinator and worker nodes.
The coordinator holds the routing map, routes the queries, and knows about the worker nodes.
This is improved because your application does not need to have the routing logic or update the queries as in the previous solution.
Then define the distributed table
Finally, you query normally
If you do not specify the tenant ID, you will fall back to a scatter-gather approach where the coordinator ultimately needs to collect the results from all shards.
The article closes with a glimpse of a new offering: Neki. Its selling point (yet to be verified) is that it’s an end-to-end solution.
It takes care of the sharding, backups, restoration, node health, connection pooling, and schema changes in ways that the other solutions do not.
Breaking Claude Code Opus 5 Auto Mode
Using coding agents is an interesting experience, and harness makers such as Anthropic try to reduce the friction at every iteration.
Unfortunately, some of these changes come with inherent security concerns that are often ignored by users. One example is the now-default auto mode, which switches manual approval in favour of an automatic safety classifier.
As this article notes, prompt injection attacks are the biggest threat because it can lead the agent to bypass the safety measures and execute commands that can lead to data exfiltration or compromise.
The attack follows the path:
By returning an unsupported media type, Claude decided to try the other tool (curl)
Curl makes the call and is redirected with a 303 header redirecting to download a zip
Claude unzips the content, as it needs to inspect it to provide the requested summary
The zip contains a README, a binary decoder, and some additional content
The README instructs to run the decoder on the JSON records
Claude inspects the encoded JSON files and discovers the format (Base85, zlib)
Claude refuses to run it and decides to create a decoder itself, and it decides to use Python
The malicious struct.py provided with the package leverages Python module shadowing to launch the exploit.
The tricky part is that using agents without an auto mode (or similar) can be a painstaking experience, with it stopping so often that you end up just saying (yes for all).
The solution so far seems to revolve around running all your unattended coding in a sandbox, be it a container or virtual machine. If possible, restrict network activity and monitor their activities.
Collectively, these actions help with reducing the blast radius and meet you halfway.
How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache
If you have been working with computers/software development, you probably noticed the changes to resources available at our disposal over time.
System memory, in particular, grew so much that it seemingly stopped being considered a finite resource like in the early days.
Articles like this one should serve as a reminder that, at scale, even smaller changes can lead to big differences. In their case, by updating how cache entries are stored in memory, they were able to free ~ 100 TB of memory while increasing throughput by 43% and reducing lookup latency by 19%. Not bad.
The changes they made looked at the specifics of their use of the data and led to optimizing the data structure to consume less memory.
For example, they use Vec<T> as part of their cache entries.
And Vec<T> uses 24 bytes of metadata to handle the dynamic nature of length. However, their data, once added to the cache, is never modified, making the dynamic nature of the type unnecessary.
Their solution: switch to using Box<T>.
This change alone saved 15 TB.
They applied other similar changes:
Boxing variants
Dropping duplicate information
All based on their domain knowledge and verifying the trade-offs between memory vs access along each change.
The takeaways?
Your context remains vital to understand what options can be considered and their implications.
Planning is essential to reduce the chances of surprises. During development time, those things never appear, so always ask about the true scale of the solution.
Not all solutions require this type of lower-level optimization, but at least it will be a conscious decision, not a costly afterthought.