Ranked #1 on Hacker News with 52 points and 38 comments.

Abstract:Large Language Models (LLMs) tend to add disclaimers like "I'm just an AI" when asked about something related to themselves. The self-reports from such responses are used in debates about AI safety or self-knowledge of the models, yet what drives them is not well understood. Are the models telling us about themselves or rather how they are deployed? In this work, we show that the chat template works like a switch - when present, it turns this disclaimer voice up and experiential voice like "I feel" down, across 8 popular open-source instruct models up to 9B parameters in size. And conversely when the chat template is not present, it turns the disclaimer voice down and experiential voice up. Inside the activations of 3 models, we find a direction that steers this behavior. Removing the direction in the model's activation space turns disclaimer voice down and adding it turns it up, while a random direction of the same size has little effect. We find that instruct models without chat template, when we add the disclaimer direction to them, disclaim like the template was there. Since the chat template controls the disclaimer voice of LLMs, then researchers studying self-reports or introspection of models might have a confound they need to control for. Our results show that there is a direction they can use to steer this voice. More broadly, our work shows that what models say about themselves is not a fact about them. What they say doesn't come only from weights, but it is partially set by the chat template, and because of that a model's self-description shouldn't be treated literally.

Submission history

Access Paper:

• View PDF
• HTML (experimental)
• TeX Source

Current browse context:

References & Citations

• NASA ADS
• Google Scholar
• Semantic Scholar

Bookmark

• Author
• Venue
• Institution
• Topic

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Ranked #2 on Hacker News with 160 points and 49 comments.

This mini-book provides a brief overview of many concurrency topics in Go. Each topic comes with interactive examples — feel free to experiment with them by changing the code and clicking Run. There's also a PDF version with static examples.

This is a quick refresher on Go concurrency, not a beginner's guide. If you want to learn concurrency from the ground up with practical exercises, check out my other book — Gist of Go: Concurrency.

The book is AI-free.

Goroutines • Channels • Select • Pipelines • Time • Context • Wait groups • Data races • Race conditions • Mutexes • Semaphores • Signaling • Run once • Object pool • Atomics • Testing • Scheduling • Diagnostics • Final thoughts

# Goroutines

The foundation of concurrency in Go is goroutines – functions started with the go keyword:

The Go runtime juggles these goroutines and distributes them among operating system threads running on CPU cores. Compared to OS threads, goroutines are lightweight, so you can create hundreds or thousands of them.

Goroutines are completely independent. The main function is also a goroutine, but it starts implicitly when the program starts. When main ends, other goroutines also shut down.

We use a wait group (sync.WaitGroup) to wait for goroutines to finish in the example above. A wait group has a counter inside. Calling Add(n) increments it by n, while Done() decrements it by one. Wait() blocks the calling goroutine (in this case, main) until the counter reaches zero. This way, main waits for both workers to finish before it exits.

WaitGroup.Go automatically increments the wait group counter, runs a function in a goroutine, and decrements the counter when it's done:

# Channels

Goroutines can pass values to each other through channels. A channel is like a window where one goroutine can throw something and another can catch it:

Sending a value through a channel is a synchronous operation. When the sending goroutine writes a value to the channel (ch <- val), it blocks and waits for someone to receive that value (<-ch). Only then does it continue.

Output channel

Returning an output channel from a function and filling it within an internal goroutine is a common pattern in Go. This allows the caller to receive values through the channel while the owning function retains control of it:

Closing a channel

To signal readers that all data has been sent, the writer goroutine closes the channel with close():

The reader checks the channel's status with a second value ("comma OK") when reading:

While the channel is open, the reader receives the next value and a true status. If the channel is closed, the reader gets a zero value and a false status.

A channel can only be closed once. Closing it again or writing to a closed channel causes a panic.

The only reason to close a channel is to signal to its readers that all data has been sent. If this isn't important to the readers, then you don't need to close it. When a channel is no longer used, Go's garbage collector will free its resources, whether it's closed or not.

Channel iteration

range automatically reads the next value from the channel and checks if it's closed. If the channel is closed, it exits the loop:

Range over a channel returns a single value, not a pair, unlike range over a slice.

Directional channels

You can protect yourself from accidental write/close errors by setting the channel direction. Channels can be:

• chan (bidirectional): for reading and writing (default);
• chan<- (send-only): for writing only;
• <-chan (receive-only): for reading only.

You can't read from a send-only channel or write to a receive-only channel (nor can you close it).

Channels are usually initialized for both reading and writing, and specified as directional in function parameters. Go automatically converts a regular channel to a directional one:

Buffered channels

Buffered channels work like a FIFO queue with a fixed-size buffer for storing values.

As long as the buffer has free space, writing to the channel doesn't block the goroutine. Similarly, as long as the buffer contains values, reading from the channel doesn't block the goroutine:

By default, if you don't specify a buffer size, a channel is unbuffered (buffer size equals zero).

Buffered channels work with the built-in len() and cap() functions:

Reading from a closed buffered channel returns values from the buffer and a true status. Once all values are taken, it returns a zero value and a false status, like a regular channel:

nil channel

Like any type in Go, channels have a zero value, which is nil.

Writing to or reading from a nil channel blocks the goroutine indefinitely:

Closing a nil channel causes a panic:

# Select

The select statement is somewhat like switch, but specifically designed for channels. Here's what it does:

• Checks which cases are not blocked.
• If multiple cases are ready, randomly selects one to execute.
• If all cases are blocked and there is a default case, executes it.
• If all cases are blocked and there is no default case, waits until one is ready.

Select is used to manage data flow in pipelines:

To cancel goroutines:

For non-blocking operations:

And for much more.

# Pipelines

A pipeline is a sequence of operations where each step takes input data, processes it in a specific way, and outputs it. The input and output of each operation is a channel.

A typical pipeline looks like this:

• Reader: Reads input data from a file, database, or network.
• N processors: Transform, filter, aggregate, or enrich data using external sources.
• Writer: Writes the processed data to a file, database, or network.

Output channel

A goroutine can signal other goroutines that it has finished its work using an output channel:

Done channel

If a goroutine doesn't need to return results, it can signal completion using a done channel:

Cancel channel

To terminate a goroutine early, a calling goroutine can use a cancel channel:

Error handling

There are three approaches to error handling in concurrent pipelines.

➊ Return on the first error:

➋ Use a result type:

➌ Collect errors separately:

# Time

Besides handling date and time, the time package offers tools for managing time-sensitive operations in concurrent programs.

After

time.After() returns a channel that is initially empty, but receives a value after the timeout period. It's useful for timing out operations:

withTimeout() waits for fn() to complete, but thanks to time.After(), it won't wait longer than the timeout duration:

Timer

A timer (time.Timer) is a structure with a C channel to which it sends the current time when it triggers (expires). Timers are useful for planning future executions:

Stop() stops the timer and returns true if it hasn't expired yet, and false otherwise:

It's often more convenient to use the time.AfterFunc() wrapper function. It waits for duration d and then executes function f:

time.AfterFunc() returns a timer that you can cancel before execution starts:

If a timer is used in a loop, it's better to create a single timer and reset it instead of creating a new instance on each iteration:

Ticker

A ticker is like a timer, but it keeps firing until you stop it. Tickers are useful for executing periodic tasks:

NewTicker(d) creates a ticker that sends the current time to the channel C at interval d. You must stop the ticker eventually with Stop() to free up resources.

If the channel reader can't keep up with the ticker, the ticker will skip ticks.

# Context

The main purpose of context is to cancel operations, either manually or by timeout/deadline.

The function accepts a context and uses its Done() channel to listen for cancellation:

Cancel manually (context.Canceled error):

Cancel by timeout (context.DeadlineExceeded error):

Cancel by deadline (context.DeadlineExceeded error):

Context is layered. A context object is immutable. To add new properties to a context, a new (child) context is created based on the old (parent) context. The shorter timeout between the parent and child contexts always wins. The child context can only shorten the parent's timeout, not extend it:

Multiple cancels are safe. You can call cancel() on the context as many times as you want. The first cancel will work, and the rest will be ignored.

You can specify a custom cancellation cause using context.WithCancelCause(), context.WithTimeoutCause() and context.WithDeadlineCause(). This cause is accessible through context.Cause():

You can register a function to execute when the context is canceled with context.AfterFunc():

Context can pass additional information about a call using context.WithValue(), which creates a context with a value for a specific key. But it's generally better to avoid passing values in context. It's better to use explicit parameters or custom structs instead.

# Wait groups

The sync.WaitGroup type lets you wait for one or more goroutines to finish:

A WaitGroup doesn't know anything about the goroutines it manages. It works with an internal counter. Calling wg.Add(1) increments the counter by one, while wg.Done() decrements it. wg.Wait() blocks the calling goroutine until the counter reaches zero.

The Go method combines Add, starting a goroutine, and Done:

All methods are safe to use from multiple goroutines.

Normally, all Add calls happen before Wait. But technically, there's nothing stopping you from doing some of the Add calls before Wait and some after (from another goroutine).

You can call Wait from multiple goroutines. They will all block until the group's counter reaches zero.

# Data races

A data race happens when multiple goroutines access shared data, and at least one of them modifies it. We need to protect the data from this kind of concurrent access.

A data race doesn't always cause a runtime panic. That's why Go provides a special tool called the race detector. You can turn it on with the race flag, which works with the test, run, build, and install commands.

Channels are safe for concurrent reading and writing, and they don't cause data races.

Ways to prevent data races:

• Avoid concurrent data modification (typically by using channels).
• Synchronize access with mutexes.
• Use only atomic operations.

Race conditions

A race condition happens when an unpredictable order of operations from multiple goroutines leads to an incorrect system state:

If individual operations are concurrent-safe, Go's race detector won't find any issues. Because of this, it doesn't catch race conditions:

You can't fully eliminate uncertainty in a concurrent environment. Events will happen in an unpredictable order — that's just how concurrency works. However, you can prevent a race condition — often by protecting a composite operation with a mutex:

Compare-and-set

Sometimes you can prevent a race condition without using mutexes by applying an atomic compare-and-set operation or one of its flavors:

The idea is always the same:

• Check if the assumed (old) state matches reality.
• If it does, change the state to new.
• If not, do nothing.

# Mutexes

The sync.Mutex type protects shared data and parts of your code from being accessed concurrently:

The mutex guarantees that only one goroutine can run the code between Lock() and Unlock() at a time.

A mutex is used in these situations:

• When multiple goroutines are modifying the same data.
• When one goroutine is modifying the data and others are reading it.

If all goroutines are only reading the data, you don't need a mutex.

TryLock

The TryLock method tries to lock the mutex, just like a regular Lock. But if it can't, it returns false right away instead of blocking the goroutine:

RWMutex

The sync.RWMutex type distinguishes between readers and writers. It provides two sets of methods:

• Lock / Unlock lock and unlock the mutex for both reading and writing.
• RLock / RUnlock lock and unlock the mutex for reading only.

Here's how it works:

• If a goroutine locks the mutex with Lock(), other goroutines will be blocked if they try to use Lock() or RLock().
• If a goroutine locks the mutex with RLock(), other goroutines can also lock it with RLock() without being blocked.
• If at least one goroutine has locked the mutex with RLock(), other goroutines will be blocked if they try to use Lock().

This creates a "single writer, multiple readers" setup.

Locker

Both sync.Mutex and sync.RWMutex implement the same sync.Locker interface:

By using Locker instead of a specific mutex type, you can build components that don't depend on a specific lock implementation. This lets the client decide which lock to use.

Channel as mutex

You can use a channel instead of a mutex to protect shared data:

# Semaphores

A semaphore is like a container with N available slots and two operations: acquire to take a slot and release to free a slot. Here are the semaphore rules:

• Calling acquire takes a free slot.
• If there are no free slots, acquire blocks the goroutine that called it.
• Calling release frees up a previously taken slot.
• If there are any goroutines blocked on acquire when release is called, one of them will immediately take the freed slot and unblock.

You can implement a simple semaphore with a buffered channel, where N is the channel's size. To acquire the semaphore, send a value into the channel. To release it, take a value from the channel:

For more complex situations, use the golang.org/x/sync/semaphore package.

Rendezvous

A rendezvous lets two goroutines wait for each other:

• There are two goroutines — G1 and G2 — and each one can signal that it's ready.
• If G1 signals but G2 hasn't yet, G1 blocks and waits.
• If G2 signals but G1 hasn't yet, G2 blocks and waits.
• When both have signaled, they both unblock and continue running.

You can implement a simple rendezvous with a wait group:

Barrier

A barrier is a general case of a rendezvous. It lets N goroutines wait for each other:

• The barrier has a counter (starting at 0) and a threshold N.
• Each goroutine that reaches the barrier increases the counter by 1.
• The barrier blocks any goroutine that reaches it.
• Once the counter reaches N, the barrier unblocks all waiting goroutines.

You can implement a simple barrier with a wait group:

# Signaling

The sync.Cond (conditional variable) type lets one goroutine signal to another that it's ready, and lets the other goroutine wait for that signal.

A Cond includes a mutex and has two methods — Wait and Signal.

• Wait unlocks the mutex and suspends the goroutine until it receives a signal.
• Signal wakes the goroutine that is waiting on Wait.
• When Wait wakes up, it locks the mutex again.

If there are multiple waiting goroutines when Signal is called, only one of them will be resumed. If there are no waiting goroutines, Signal does nothing.

You can also use the Broadcast method. While Signal wakes up only one goroutine waiting on Cond.Wait, the Broadcast method wakes up all such goroutines.

You can signal with a channel:

And broadcast too:

Broadcasting with a condition variable is limited: it only sends a signal, not the actual data, and it only works once. With channels, you can build a publish/subscribe system that doesn't have these limitations:

# Run once

The sync.Once type makes sure that the given function runs only once. If multiple goroutines call Once.Do at the same time, only one will run the function, while the others will wait until it returns:

Once is perfect for one-time initialization or cleanup in a concurrent environment.

Besides the Once type, the sync package also includes three convenience once-functions:

# Object pool

The sync.Pool type helps reuse memory instead of allocating it every time, which reduces the load on the garbage collector:

Get takes an item from the pool. If there are no available items, it creates a new one using New (which we have to define ourselves, since the pool doesn't know anything about the items it creates). Put returns an item back to the pool.

Things to keep in mind:

• New should return a pointer, not a value, to reduce memory copying and avoid extra allocations.
• The pool has no size limit. If you start 1000 more goroutines that all call Get at the same time, 1000 more buffers will be allocated.
• After an item is returned to the pool with Put, you shouldn't use it anymore (since another goroutine might already have taken and started using it).

# Atomics

An operation without synchronization can only be truly atomic if it translates to a single processor instruction. Such operations don't need locks and won't cause issues when called concurrently (even the write operations).

There are only a few atomics, and they're all found in the sync/atomic package:

Each atomic type provides the following methods:

• Load reads the value of a variable.
• Store sets a new value.
• Swap sets a new value (like Store) and returns the old one.
• CompareAndSwap sets a new value only if the current value is still what you expect it to be.

Numeric types also provide an Add method that increments the value by the specified amount.

All methods are either translated into a single CPU instruction or are otherwise guaranteed to be atomic, so they are safe to use from multiple goroutines.

The composition of atomics is always non-atomic:

A bulletproof way to make a composite operation atomic and prevent race conditions is to use a mutex:

Sometimes you can use an atomic type instead of a mutex to exit early:

# Testing

If your concurrent program uses channels or custom types with synchronization methods like Wait, you can use those in your tests. This way, your tests won't be much more complicated than if the code were synchronous:

If there aren't any suitable synchronization "handles" in the code you're testing, you can use the synctest package. It exports two functions:

synctest.Test runs an isolated bubble. The bubble uses a fake clock, and you can manually control goroutine synchronization with synctest.Wait.

synctest.Wait blocks until all goroutines in the bubble — except the one that called Wait — have either finished or are durably blocked. This lets you wait for a specific goroutine to finish or get blocked, so you can check the program's state:

The fake clock in synctest.Test move forward only if: ➊ all goroutines in the bubble are durably blocked; ➋ there's a future moment when at least one goroutine will unblock; and ➌ synctest.Wait isn't running. Thanks to this, time-dependent tests run instantly:

The following operations durably block a goroutine:

• A blocking send or receive on a channel created within the bubble.
• A blocking select statement where every case is a channel created within the bubble.
• Calling Cond.Wait.
• Calling WaitGroup.Wait if all WaitGroup.Add calls were made inside the bubble.
• Calling time.Sleep.

Blocking on mutexes, I/O, or system calls is not considered durable, and the synctest bubble can't handle them.

# Scheduling

At the hardware level, CPU cores are responsible for running parallel tasks.

At the operating system level, a thread is the basic unit of execution. There are usually many more threads than CPU cores, so the operating system's scheduler decides which threads to run and which ones to pause.

At the Go runtime level, a goroutine is the basic unit of execution. The runtime scheduler runs a fixed number of OS threads, often one per CPU core. There can be many more goroutines than threads, so the scheduler decides which goroutines to run on the available threads and which ones to pause. The scheduler keeps switching between goroutines to make sure each one gets a turn to run on a thread, instead of waiting in line forever.

This is how Go handles concurrency.

Goroutine scheduler

The goroutine scheduler's job is to run M goroutines on N operating system threads, where M can be much larger than N. Here's a very simplified version of it's algorithm:

• If there's a free thread, assign it a goroutine from the queue.
• If a running goroutine gets blocked (for example, while reading from a channel), put it back in the queue and assign a different goroutine to the thread.
• If a running goroutine gets stuck in a syscall, start a new thread to run other goroutines until the blocked goroutine finishes the syscall.
• Check the running goroutines every 10 ms. Preempt long-running goroutines and return them to the queue to prevent starvation.

The number of threads running Go code is controlled by the GOMAXPROCS environment variable or the runtime.GOMAXPROCS function.

A goroutine is a structure that starts out using about 2 KB of memory, mostly for its stack. The stack can grow if needed. Since goroutines are so lightweight, you can run tens of thousands or even hundreds of thousands of them on a small machine.

# Diagnostics

To troubleshoot concurrent programs in production, we use metrics, profiling, and tracing.

Metrics show how the Go runtime is performing, like how much heap memory it uses or how long garbage collection pauses take. Each metric has a unique name and a value, which can be a number or a histogram.

You can use the runtime/metrics package to get a complete list of metrics or check the values of specific ones:

In practice, people rarely do this manually. Instead, all metrics are automatically exported using Prometheus or OpenTelemetry libraries.

Profiling helps you understand exactly what the program is doing, what resources it uses, and where in the code this happens. Go uses a sampling profiler that's suitable for production.

The most commonly used profiles are CPU, which shows how much processor time each function uses, and heap, which shows how much heap memory each function uses. Goroutine, block, and mutex profiles help identify problems related to concurrency.

The easiest way to add a profiler to your app is by using the net/http/pprof package. To collect a profile with the given name, call the /debug/pprof/{name} endpoint. To view the collected profile, use the go tool pprof utility:

You can also profile manually:

Tracing records certain types of events while the program is running, mainly those related to concurrency and memory. When the profiling server from the net/http/pprof package is running, call the /debug/pprof/trace endpoint to collect a trace. To view the results, use the go tool trace utility.

You can also collect a trace manually:

You can set up automatic tracing with a sliding window that's limited by size or duration. This is called "flight recording". It lets you always keep a recent trace available in case something goes wrong:

# Final thoughts

We've covered a number of Go tools for writing concurrent programs:

• Goroutines for running concurrent tasks.
• Channels and select as flexible communication tools.
• Timers and tickers for working with time.
• Context for canceling operations.
• Wait groups for synchronizing goroutines.
• Mutexes to prevent race conditions.
• Condition variables for signaling events.
• Once for safe one-time initialization.
• Pools to reduce garbage collector load.
• Atomic operations.

If you like the book, please recommend it to your friends or colleagues. If you're interested, check out my other books and projects.

I'm glad you finished the book. Thank you, and I'll see you next time!

★ Subscribe to keep up with new posts.

Ranked #1 on Hacker News with 82 points and 38 comments.

Hi, this is Lars Doucet, author of the book review of Henry George’s Progress and Poverty that won the first ACX book review contest, as well as the three-part follow-up guest post series, “Does Georgism Work?” A lot has happened since then, including land value tax (LVT) enablement laws passing this year in two U.S. states and the election of LVT-friendly national leaders in the UK and South Korea. As for me, I now work full-time for the Center for Land Economics and write for Progress & Poverty substack.

I’d like to reflect on what I wrote five years ago: what I was right about, what I’ve changed my mind on, and what the outlook for LVT is in 2026. But first, here’s a brief summary for those who either have no idea what I’m talking about or just need a refresher.

I. Progress and Poverty

In 2021, I wrote a book review for ACX on the book Progress and Poverty, the magnum opus of the famed 19th-century economist and populist firebrand Henry George:

If I had to summarize the book in a single sentence I would put it this way: poverty and wealth disparity appear to be perversely linked with progress, The Rent is Too Damn High, and it’s all because of land.

George argues that poverty paradoxically advances alongside progress because, as material conditions improve, landowners can charge more rent for locational benefits they didn’t create. People who earn more than the local average salary (e.g., software engineers) can stay ahead of this trend, whereas those who don’t (teachers and service workers) are priced out of the homes they rent. Eventually, even upper-middle-class families can struggle to afford services like daycare, because daycare businesses must charge more to pay business rent and employee wages. Landlords, rather than daycare owners or workers, collect most of the service price increase parents pay.

Henry George’s remedy is the land value tax, or LVT. In its ideal form, this tax would capture and redistribute the annual rental value of land; that is, the recurring value of the land excluding the value of any buildings or other improvements on top of it. In practice, this looks a lot like a conventional property tax paired with a “universal building exemption.” Notably, George isn’t simply in favor of a land value tax; he’s also opposed to taxes on both labor and capital.

George further proposed the single tax—a policy in which land is taxed at its full annual rental value, and LVT is the only tax. Although the feasibility of the “single tax” remains controversial among economists, land value tax itself is surprisingly well accepted by economists left, right, and center as the ideal tax policy, with mainstream criticism mostly centered on practical and political concerns.

After the book review contest results were announced, ACX readers inundated me with questions, which led to the three follow-up posts. After those ran, I reposted all four articles on the standalone site www.gameofrent.com, and later consolidated them into a book, Land is a Big Deal.

Here’s what’s happened since.

II. LVT Momentum is Growing

Five years ago, LVT was mostly a hypothetical idea people debated on blogs. Today, it has the most legislative momentum it’s seen in decades. Many states introduced bills this year, and Virginia and Kentucky passed land value tax enablement laws in April. These laws allow municipalities to opt into split-rate property taxes, which lower tax rates on buildings and raise them on land. My organization, the Center for Land Economics, maintains a public legislation tracker with an interactive map that makes it easy to keep tabs on these trends.

Virginia and Kentucky were the big winners for the 2026 legislative season, enabling various cities in those states to implement land value taxes. However, even bigger opportunities are coming. In Washington State, my organization is collaborating with the Sightline Institute on an upcoming LVT bill. Meanwhile, in New York, Governor Hochul extended authority for cities to utilize land value capture to fund new transit stops, a tool that can be used for the new Inter-Borough Expressway (IBX) subway expansion. We’re working with Niskanen Center, Center for Public Enterprise, and Institute for Progress to turn this idea into a real policy proposal.

Nor is interest in land value tax limited to the United States. The UK just elected a new prime minister, Andy Burnham, who has long openly advocated for a land value tax, and South Korea did the same last year with the election of new president Lee Jae Myung. Additionally, in January 2025, the German state of Baden-Württemberg implemented a new LVT that survived a subsequent court challenge. Although we’re tempering our expectations (the effective tax rate of the German LVT is quite low, and it remains to be seen how ambitious the Burnham and Lee administrations will be), it’s clear that LVT is no longer an obscure idea, nationally or internationally.

Finally, many now believe that the economic effects of AI developments may accelerate support for LVT and Georgism more generally. As newly minted AI millionaires bid up land prices in San Francisco and Seoul, Adam Ozimek argues that land will be a winner in the age of AI. Similarly, Aksel Sterri, co-founder of the Norwegian Effective Altruist think tank Langsikt, calls for an explicitly Georgist framework for understanding the AI era. (I’m also Norwegian; see my piece on Norway’s century-old Georgist tradition in natural resource management for more context on what Aksel’s talking about.)

So what have I learned?

III. My Theory of Change Was Wrong

I used to think LVT advocates had to change popular and elite consensus before moving on to the boring scut work of implementation. Now I realize that the boring scut work is actually what precedes, and even leads to, changing the consensus.

Overrated: winning online arguments

If you go back to the original series’ comments, you’ll see me trying to answer every concern that comes up, and falling for the bait when someone drags me into a tendentious comment thread dozens of replies deep. These would sometimes terminate in the interlocutor declaring that they’re the “exact kind of person Georgists need to convince!”

I’ve since learned that overly online keyboard warriors are the least important people to convince. The most important people are on your local city council or state legislature. Furthermore, instead of wasting time with hard cases or elected officials who are dead-set against your ideas, you’re much better off finding and working with people who are already interested in your principles. This often means giving up on your own city or state, at least for the time being, and pursuing a “succeed anywhere” strategy instead.

Changing policy where people want to change it is also how you overcome the “cold start problem,” when someone likes your idea but wants to see someone else do it first. Rather than trying to change that person’s risk aversion, find the person who is adventurous enough to try something new and work with them first. This builds case studies that you can use as social proof for the second, more cautious, wave of reforms.

I’ve also found that many online and academic objections are somewhat imaginary. When you talk to actual people in office, either their objections are entirely different from the ones you see in social media and comment threads, or they’re surprisingly open to being convinced if you listen patiently and present a clear argument backed by data and research.

Which brings me to…

Underrated: doing policymakers’ homework for them

No politician on earth has the time or inclination to independently learn about your philosophy, ingest all the arguments for it, evaluate a bunch of empirical and theoretical research, model its impacts on their locality, anticipate and respond to all possible objections, then wrap it all up into a tidy package complete with PowerPoint presentations, slick graphs, interactive websites, and a convenient, printable “one-pager” to hand out to interested parties.

You know who can do all those things? You.

For instance, some people get confused about land value tax, thinking it will wreck single-family homeowners, or that I’m out to get them personally, because obviously all the land value in town is concentrated directly underneath their specific home. This isn’t a disagreement about values or mechanics but rather a simple misunderstanding of what land value is and where it is most concentrated.

The first thing I do to disabuse people of this notion is to point out how much land value in any typical U.S. city is concentrated in downtown areas, and how much that value attenuates in the suburbs. We do this by loading up CivicMapper, our free, open-source 3D visualizer that takes local assessed land values and puts them on a map.

Here’s Washington, D.C.

Here’s Austin, Texas.

Here’s Seattle, Washington.

Not all cities look the same, and not all assessments are of equal quality, but you see the same basic patterns everywhere. Land in the city center is worth much, much more than outlying areas.

Then, we show people how much of that high-value land is dedicated to extremely low-value uses, like surface parking. Here’s Houston, which has over 3.5 billion dollars of land value locked up in surface parking alone.

And no, it’s not just Texas. Here’s Portland, Oregon.

We can even zoom in and show how much of that surface parking occurs in the most valuable areas, such as Houston’s downtown district, which alone accounts for nearly half a billion dollars’ worth. These are exactly the places where it makes the most sense to concentrate development.

Having established that Land Is a Big Deal and that we’re also squandering it, we do the math and build a model of who wins and who loses under a revenue-neutral land value tax shift, or “Universal Building Exemption.” To do this, we use LVTShift, a free, open-source Python library maintained by the Center for Land Economics. In most of our models, the biggest losers are vacant land and surface parking lots, and among the net winners is the typical median single-family homeowner.

In addition to doing the math, we find compelling stories to tell. For instance, in this report on Cincinnati that we collaborated on with the Notre Dame Student Policy Network, we illustrate how conventional property taxes punish those who improve properties and invest in the city by providing housing and business, while those who own vacant lots or surface parking lots are rewarded for holding land out of use.

Here’s a similar comparison from our report on Spokane, Washington. The lots with houses on them pay more than seven times as much per square foot of land as the vacant lot does, even though all the land is equivalently zoned and similarly located.

We built up this methodology from crude beginnings by talking to people, trying things, learning from our mistakes, and refining our approach. Finally, we condensed everything we learned into a concrete political playbook entitled Enacting Land Value Return in Your Hometown, then published it as a guide for others to follow.

This playbook is now leading to wins. One of our activists, Jackson Arnold, a member of the Abundance Network, wanted to implement LVT in his hometown of Louisville, Kentucky. He got in touch with us, joined the OpenAVMKit Discord, and ran the playbook all the way from inception to getting a bill passed in his state legislature. We are now trying to figure out how to inspire and enable more Jackson Arnolds.

While the first wave of LVT fans were local citizens, we’re now attracting lawmakers’ attention. The most salient example is Bill Blessing, a Republican state senator from Ohio and chair of the Ways and Means Committee, who introduced an amendment to Ohio’s state constitution this year to legalize local opt-in LVT.

That’s all very exciting. But we still have one question to ask before we get carried away.

IV. Does Georgism work?

My three-part article series was structured as an investigation into the three most common objections to Georgism. We should revisit those and see where things stand in light of what I’ve learned since. The three objections were:

1. Land just isn’t a big deal anymore in the modern economy.
2. Land value tax will just be passed on to tenants.
3. Land value can’t be accurately assessed separately from buildings.

1. Is Land a Big Deal?

In Part 1, Is Land A Big Deal?, I ran the “land isn’t a big deal” theory against several testable hypotheses. Among the findings was the fact that sky-high urban real estate prices were primarily driven by land appreciation, and that land was a large and steadily increasing share of bank loans. The article’s centerpiece was an original estimate of the total land value of the United States, which was much larger than many had expected. Although this estimate fell short of what a “single tax” would require, it was still large enough to convince many that LVT had been unfairly dismissed as a serious policy proposal.

This article has aged the best of the three, but I still have a few updates to share.

Korea is a Big Deal

If land is a big deal in the USA, it’s an even bigger deal in South Korea, which has all the prerequisites for a national LVT and enough land value to approach a full-on single tax, or other ambitious projects like Universal Basic Income (UBI). Here’s a snippet from my piece, UBI Advocates should watch South Korea:

For UBI or LVT to succeed anywhere, they must first succeed somewhere…therefore, if you want UBI or LVT to succeed, you should scour the world for a place where these two policies are the most:Economically feasibleTechnically feasiblePolitically possibleSocially and politically urgentThat place is South Korea.

• Economically feasible
• Technically feasible
• Politically possible
• Socially and politically urgent

South Korea has the highest land value-to-GDP ratio in the entire OECD, in excess of 500%. To put that in perspective, the figures from my own estimates of the USA’s total land values—which surpassed many readers’ expectations—were a mere 200% of GDP. My first instinct was that Korean land values must reflect a temporary, anomalous bubble and would soon revert. However, long-run land value-to-GDP ratios over the last 50+ years have ranged from 400%, and current values aren’t even the all-time peak, which clocks in at around 600%. Prime land is just that valuable and scarce in South Korea.

Where do these land value figures come from, by the way? Turns out, South Korea has one of the best-organized land valuation systems in the entire world, despite not having a land value tax, which we’ll discuss later.

In short, LVT in South Korea could raise enormous revenue. Land is already valued down to the individual parcel annually, the current president is sympathetic to LVT, and the urgency for socioeconomic reform in South Korea has never been higher. Although many daunting political obstacles remain, South Korea’s example makes it clear that land is, if anything, an even bigger deal than I originally thought.

Next, let’s address a few counter-arguments to the “Land is a Big Deal” thesis that I didn’t fully address last time.

What About Zoning?

One argument that often came up in comment threads was, “All we need to do to solve the housing crisis is upzone.” I certainly agree that upzoning is necessary, but I don’t agree that it is sufficient (which is something I also believe about LVT). Stephen Hoskins’ essay Land and Liberty to Build makes a great philosophical case for why YIMBYs should also be Georgists and Georgists, YIMBYs, to which I will add a few arguments of my own.

First, history falsifies the “upzoning is sufficient for affordability” hypothesis. If upzoning is all we need, we should not see housing affordability crises before zoning, which was not fully entrenched until the 1920’s. Instead, we see the opposite. The Georgist movement itself sprang from a massive housing crisis in the late 1800’s, decades before zoning became widespread. One could say, “Well, high-rise buildings hadn’t been invented yet, which is what you need to overcome land scarcity pre-zoning.” However, skyscrapers had been around for decades prior to the 1920’s.

Second, not every location is equally constrained by zoning. Michael Wiebe has a great article reviewing a recent paper that estimates the implicit “zoning tax” of various metros, finding that San Francisco is the most constrained, and cities like Cincinnati the least.

Even though cities like Cincinnati aren’t as expensive as San Francisco, they still have problems. It’s easy to find concrete examples of the most valuable land being held out of use even in the least constrained cities, which contradicts the “upzoning solves everything” argument. We’ve already shown you Houston (which has no zoning), but here’s a look at surface parking lots in Cincinnati’s downtown. You can find this pattern of wasted valuable land in just about any American city.

Let’s go back to this chart from Cincinnati.

We can see that although all four parcels share the same zoning, the building component of the property tax gives them very different tax assessments per square foot of land. The tax system actively punishes the affordable housing complex for the crime of being denser than its neighbors.

Some LVT-skeptical YIMBYs also argue that, “Land is already taxed by conventional property tax, therefore we don’t need LVT.” The problem is that YIMBYs generally oppose “impact fees,” which are arbitrary extra costs local governments impose on new development. In Property Taxes are not Land Value Taxes, I argue that the building component of a property tax mathematically amounts to the same thing. I am more than happy to defend property taxes against those that wish to abolish them entirely, but I will continue to insist that the building component of the property tax is distortive in essentially the same way that impact fees are. My position is not to layer on an additional LVT but to lower (or eliminate) the effective tax rate on buildings, and simultaneously raise it on land.

The final thing I’m updating on is to give more attention to other ways of capturing land value than LVT alone, such as through ground rent leases. Jeff Fong, a prominent member of YIMBY Action (as well as our board of advisors), has an excellent piece called Georgism through Land Leasing that explores this potential.

That’s it for the “Land is a Big Deal” thesis. Here’s how I’ve changed my thinking on the other two articles.

2. Can Land Value Tax Be Passed on to Tenants?

In Part 2, Can Landlords Pass Land Value Tax on to Tenants?, I read more than a dozen papers on tax incidence and capitalization effects of land value tax and conventional property taxes. The evidence overwhelmingly showed that LVT is not passed on to tenants.

However, I’ve since found at least one condition under which LVT can be “passed on” to tenants: when housing is pervasively rent-controlled, and landlords are granted a special exemption to raise rents in direct response to tax increases.

To understand why, let’s review the traditional argument for why, in general, LVT is not passed on to tenants. Rents are not set by a landlord’s costs or desires but by supply and demand. The opposing view, the “cost plus” theory of rental pricing, makes several testable hypotheses, which this research brief by the Progress and Poverty Institute evaluates.

For instance, if landlords reflexively pass on costs (including holding costs like land value taxes) as higher rents, doesn’t that also imply they should cut rents when their costs decrease? Mortgage interest rates have fallen sharply over the past 40 years, yet rents have increased over the same time. Also, landlords who own their properties outright (and thus have no mortgage interest costs) don’t seem to charge different rents than nearby landlords of equivalent properties who are still paying off their loans.

However, these conditions don’t hold under pervasive rent control. If the prevailing rent is already well below what the market will bear, and a landlord is specifically allowed to raise rents by the increased tax amount to some new level that is still below true market rent, then logically, the tax will be mechanically “passed on.” This appears to be the case in Denmark, according to a 2024 paper by Nielsson, Wroblewski, and Yding.

Here’s a diagram illustrating the effect.

Without pervasive rent control paired with a special landlord tax break, the picture would look more like this.

To raise rents in response to taxes, the tax must affect the supply of housing somehow. For taxes on buildings, this mechanism is obvious. Taxing buildings reduces the labor and capital spent producing and maintaining them, just like development impact fees. Less supply of buildings, with unchanged demand, means higher building prices.

Land, however, is not built. Land is inelastic in supply, and landowners don’t “provide” land the way laborers provide labor and investors provide capital; they un-provide land by excluding others from using it. Building supply can change in response to a tax change, but land supply cannot.

The Nielsson, et al. paper’s finding does make me discount the older Danish paper I cited in my second article somewhat, and I’m updating that there is at least this one exception that policymakers should be aware of. However, I don’t think Nielsson, et al. have made a convincing general case outside of these specific conditions, because the Danish case is only one data point among more than a dozen others, and the pass-through mechanism they describe is so narrow, specific, and clearly explainable.

An explanation sufficient to convince me otherwise would need good answers to these four questions:

1. Why don’t landlords cut rents when their operating costs fall?
2. When landlords threaten to raise rents in response to a future land value tax, why don’t they just raise the rents right now? Why do they have to wait until the tax is enacted?
3. Why do property developers subtract all holding costs (which would include both land value taxes and conventional property taxes) from their net operating income, which fully capitalizes into a lower offering price in their discounted cash flow pro-formas?
4. Will rents/housing prices, ceteris paribus, rise or fall in response to all property taxes in a jurisdiction being suddenly abolished?

I’ve always been happy to concede that the research on conventional property taxes has been more mixed, with some studies finding full capitalization, others finding partial capitalization, and yet others finding full pass-through. My understanding is that in those cases, local results depend on how responsive the supply of buildings is to marginal tax changes, which varies considerably from place to place. It makes sense that if you’re in a very NIMBY area that never builds anything no matter what, a tax change is unlikely to affect local supply.

Now, on to how I’ve updated my thinking in the final article on land valuation.

3. Assessing the Value of Land

In Part 3, Can Unimproved Land Value be Accurately Assessed Separately from Buildings?, I researched how assessors and academics estimate land value. I concluded: “It’s quite plausible but not a slam dunk. That said, if the objection is, ‘valuing land separately from improvements is fundamentally impossible, and we can never get better at it, so we shouldn’t try,’ I think that’s plainly ruled out.”

This is the article I feel has aged the least well—not because I’ve drastically changed my conclusion, but because for years now I have been studying property taxes, interviewing assessors, and evaluating valuation methodologies as my full-time job, which has given me a much better understanding of the details and procedures.

I have many updates on this topic, but we’ll start with the biggest one, which comes from South Korea, an existence proof that at least one country can do this efficiently and regularly on a national scale.

Maybe Just Send Your Office to South Korea

For my article How to Value Land: Korean Style, I interviewed Korean researchers Jinsu Lee and Vitnarae Kang and read hundreds of pages of dense Korean-language procedural documents published by MOLIT. The short version is that South Korea values all its land—the land value specifically, not just the total value—every year, down to the individual parcel, and gets the entire operation done in five months flat.

This is organized as a simultaneous joint effort by national and local governments, with the national government responsible for 500,000 “standard parcels,” chosen as locally representative parcels of that type in any given area. In all but exceptional cases, each “standard parcel” receives no less than two individual appraisals, and the final valuations for these are handed to local governments as valuation anchors. The local governments are then responsible for valuing all other parcels in their jurisdictions. This simultaneous top-down and bottom-up collaboration combines irreducible local knowledge with centralized support and standardization.

The Korean case reveals that many arguments against land valuation’s feasibility are fundamentally provincial. The first such example is Wales, where a land value tax is being actively considered and where the government commissioned a report on the feasibility of land valuation, employing no fewer than six separate consultants. Although the findings seem comprehensive at first glance, not one of the consultants examines the Korean system, the single most relevant international example, in any detail. The closest we got was a single line in a report by Alma Economics, which buried the lede as follows:

In Korea, for instance, about 1,300 appraisers (2011 data) value sampled plots, with prices extrapolated to adjacent plots.

Another good example is Sam Watling’s “The failure of the land value tax,” published in Works in Progress magazine last March, which details the Liberal Party’s failed push for a UK land value tax in the early 1900s. In “Contra Watling on the failure of the land value tax,” I show that Watling’s assertion that a “pure” land value tax has never been implemented is plainly contradicted by easy-to-find historical examples, several of which even occurred at the same time as his singular UK episode. As for his other claim that land valuation is practically impossible, his main proof is simply that the UK couldn’t pull it off more than a hundred years ago, paired with the unevidenced assertion that the UK can’t manage it today. As of this writing, the UK’s per capita GDP is $24K higher than South Korea’s, and it has 18 million more people. The state capacity limits that he asserts prevent Britain from successfully implementing LVT would be a specific local dysfunction, and would presumably impede any reform the country pursued to address its housing crisis.

As for the Korean methodology itself, I was surprised at how little “magic” I found when digging into the details. The key differences from what I was used to were procedural and organizational. The valuation techniques themselves were no different from what you would expect to find in a particularly well-run Texas or North Carolina appraisal office. This brings me to my next major update.

There’s No Single Magic Algorithm

This will come as no surprise to anyone with a background in data science, but data quality and running a tight ship matter far more than which cutting-edge predictive methods you pick. South Korea is a great example, but other things I discovered kept pointing to the same finding.

First, I’ve learned that there’s a vast gulf between the world of academia and the world of working assessors. This gulf is not necessarily about knowledge or skill—I know many equally brilliant assessors and academics—it’s simply the traditional divide between pragmatic practitioners and theoretical researchers. Whereas academics are obsessed with r-squared metrics and pristinely crafted multivariate regression equations, assessors are obsessed with whether Jimmy has uploaded new sketches for the River Heights neighborhood yet, if Nancy’s new model can produce adjustments compatible with the new comp grid meant for valuation defense, and if anyone can get the damn CAMA vendor on the phone about whether that bug from six months ago has finally been fixed.

I’ve tried my best to help bridge that divide. My Mass Appraisal for the Masses article series is meant to help outsiders understand assessors, and my open-source Python library OpenAVMKit helps assessors understand and use academic machine learning models. I also routinely attend industry conferences; come see my presentation this year at IAAO National in Calgary, or next year at GIS/Valtech in Louisville, Kentucky!

Second, in Amateurs talk Algorithms, Professionals talk Data Cleaning, I explain that the single most important thing any office can do to improve its results is to check its data for invalid and anomalous sales, as well as mismeasured or unobserved building characteristics (particularly physical condition). The most valuable part of OpenAVMKit turned out not to be the fancy machine learning predictors, but simply better heuristics for detecting and diagnosing bad data inputs.

Third, I’ve changed my mind about the cost approach and now believe it is basically fine. For context, the “cost approach” estimates a building’s value by using construction cost tables to calculate the cost to rebuild a property, then applying depreciation based on age and condition. This was a method I criticized somewhat naively in my last article without fully understanding how and where it should be used. In fact, the prevailing methodology used in property tax offices in the United States, the “sales-adjusted cost approach,” yields reasonable land values as a side effect, at least when it’s performed correctly. I describe this method in How Appraisers Value Land, an interview conducted with veteran North Carolina property assessor Thomas Holding, who has thirty years of experience on every side of the appraisal industry, public and private, fee appraisal, and mass appraisal.

Fourth, in How Georgists valued land in the 1900s, I researched how land was valued in the United States before the advent of computers and discovered a method favored by turn-of-the-century Georgists called the Somers system. Their secret? They just asked people what the land values were.

The Somers system sounded crazy to me at first, but the historical records indicate it was a serious method used for decades throughout the United States. The facilitators held a series of meetings where they gathered locals together and asked their opinions about relative land values, street by street. The whole assembly would argue back and forth until it reached consensus, which the moderator would record on a gridded map on the back wall. The goal was to encode relative values based on irreducible local knowledge, then calibrate them against market evidence to produce absolute valuations. This method gradually waned sometime around the 1950’s, but you can still detect tiny vestiges of it in the IAAO’s modern land valuation course.

The most striking feature of the Somers system is that it was specifically optimized for maximum community buy-in. It did this by inverting the usual process of doing valuations first and then hearing citizen protests. Here, the “protest” phase came first, and valuations themselves flowed directly from citizen feedback. Although the Somers system is no longer in use in any jurisdiction I know of, more than one appraiser I’ve interviewed has said they have independently re-invented some variant of the method for areas with thin sales by gathering locals together and asking them to assign values to different areas by consensus.

Fifth, I discovered that even surprisingly crude land valuation methods can still work. In the late 1800’s, the German colony in Qingdao, China, instituted an aggressive tax on the unimproved value of land. Although Imperial Japan eventually conquered it and ended the LVT experiment, the regime lasted long enough, and the LVT was levied at a high enough rate (a whopping 6%), for the expected theoretical effects to be clearly observed. The case of Qingdao is well known, but a researcher recently discovered new German-language primary sources, including meticulous budget records and even full-color land value maps. These documents reveal that the land valuation methodology was to simply carve the city up into tax districts which locals judged to be of similar value and assign the same flat land value rate to all parcels within them.

One of the chief things this style of land assessment gets right is ensuring local uniformity of land valuation for all parcels locals judge to be economically similar. This gives a potential answer to the question I raised in the last article: how good do land values have to be to be “good enough?” The Qingdao case, as well as other cases I’ve encountered, has led me to believe that the most important features of good land valuation are to:

• broadly track market value and stay up to date
• comport with local common sense expectations
• be locally uniform across similarly situated and zoned land

Horizontal Uniformity

One deputy chief assessor told me that one category alone—complaints about unequal side-by-side property valuations for neighboring properties—accounts for fully 40% of his office’s annual protest volume. We uncovered exactly these kinds of horizontal inequities in our report on side-by-side land valuation anomalies in Baltimore, Maryland. Vacant lots in neighborhoods were valued at nominal rates, while equivalently zoned, similarly sized improved lots next door would have their land valued for ten times more on a dollar per square foot basis, providing a large subsidy to vacant lot owners and shifting the tax burden to homeowners and businesses.

This error was simple to explain and easy for local stakeholders to understand. Shortly after our report went live, SDAT, the Maryland state agency responsible for valuation, announced an initiative to address the problem and has now begun correcting the undervaluation of vacant land.

Another common mistake is treating vacant land as having only nominal value and misapplying the “allocation method,” where land is assigned a fixed percentage of total property value, such as 20%. Assessors are supposed to apply the allocation rate to the prevailing median property price in the local area, thereby arriving at a uniform local land rate. When using this method, all similar land in the same area should be assigned the same local land rate. Instead, some assessors will mistakenly multiply each individual parcel’s total assessed value by the same fixed rate to arrive at a land value, resulting in land values that jump sharply from parcel to parcel, even when they all have essentially the same size, zoning, and location. This diagram illustrates the difference.

I demonstrate in my article Valuing Land: The Simplest Viable Method that, although we can and should value land more precisely than this wherever we are able, even this dead-simple land valuation method is sufficient to achieve the economic incentives of LVT. Qingdao provides us at least one empirical proof of that approach working in real life. Jurisdictions should make sure baseline land valuations meet this minimum standard.

Whatever you do, keep values up to date

I’ve also updated big time on what the single most dangerous mistake with land valuation and property valuation in general is: not updating your values. The North Star of property tax valuation is “equal and uniform,” and massively out-of-date valuations make a mockery of this principle.

“Equal and uniform” means everything should be valued by the same consistent rule, and similar properties should be similarly calibrated to their revealed market value. Even when mass appraisal methods involve estimates or errors, those estimates and errors should be applied consistently under a transparent rule.

However, if property values haven’t been updated in twenty years, then the valuations have almost no relationship whatsoever with what the market is currently paying for those properties. Some people will be paying far more than their house is worth, and others will be paying far less. This is also why “just value the house at exactly what it sold for” is such a bad idea, because not every home sells every year. If you were to do that, a neighborhood full of identical houses, all starting at $100K, with prices going up by an average ~$10K a year, will look like this after ten years, with massive side by side inequities in valuation based purely on when a home sold, even though they all would sell for about the same price today.

Simple fairness and equal treatment under the law should be enough to establish regular reassessments for property tax purposes. If you’re an LVT advocate, however, the stakes are even higher, and there’s no better illustration of the cost of stale valuations than the repeal of Pittsburgh’s LVT.

Pittsburgh is one of several Pennsylvania cities that have historically had a split-rate property tax. However, long-delayed valuations followed by a poorly implemented revaluation triggered a tax revolt that led to the policy’s reversal. LVT advocates need to honestly grapple with this failure case if we hope to avoid it in the future.

Stale valuations create several problems at once. First, when valuations were finally updated, taxpayers got sticker shock because they had gotten used to values never changing. Second, the local government had become dependent on an outsourced vendor to perform the valuations. Third, public messaging was poor, and the outsourced valuations were opaque, drawing sharp criticism and widespread protest. One thing led to another, and the split-rate property tax was repealed.

This is another place I notice the provinciality of objections. In the Northeast, where many jurisdictions revalue infrequently, many see revaluations as inherently fraught, expensive, and politically risky. At the same time, I know plenty of jurisdictions in the Sunbelt that uncontroversially revalue on three-, two-, or one-year cycles and efficiently process enormous volumes of routine property tax protests.

Revaluation only seems daunting for the same reason that going to the gym does when you haven’t done that in a decade, either. The more frequently you revalue, the more quickly you notice and fix errors in your data, bugs in your process, and anomalies in your algorithms. You’ll have better accuracy, better horizontal uniformity, and better vertical equity. You’ll also get better at explaining and defending your values to the public. If you do the reps, you’ll get the gains.

On the other hand, if you let revaluations slide four years, then six, then 10, soon no one in the office will have been around for the last revaluation, let alone anyone who gains compounding experience and knowledge year over year. Before you know it, you’ll be on the phone with an outsourced vendor who knows they have you over a barrel, insisting on a multi-million-dollar contract, take it or leave it.

The good news is there’s hope. Decades after Pittsburgh’s LVT repeal, members of Pro-Housing Pittsburgh got so tired of the mounting valuation inequities that they took matters into their own hands. They downloaded OpenAVMKit, fed it local public data, and built their own AVM. They generated fresh valuations, ran statistical tests, and proved that their values tracked market value more closely than the outdated official figures.

Finally, the last thing I’ve learned is the answer to the last unanswered question.

V. Why did Historical Georgism disappear?

The short answer is cars.

The long answer is that America is a nation with a frontier mentality which has lost its frontier.

The looooong answer is so long it takes two articles to explain: What happens when America’s Monopoly board fills up?, and The Housing Ladder’s Broken Promise.

Let me summarize key parts of those here.

The frontier was always America’s answer to the land problem, and it still is in many people’s minds. “Just work hard, save money, and buy cheap, high-opportunity land, as I did.” The first American frontier was literal: “Go West, young man.” It worked out great for the settlers, and less great for those excluded from it (Indians, slaves, and others). When the first frontier closed, the first Gilded Age dawned, as did the scarcity and inequality that stirred Henry George to action.

However, in the century that followed, amid two world wars and a great depression, we also invented the automobile, and with it the ability to sprawl. Whereas before you had to live close to your job, fighting over scarce supply and struggling under crushing land rents, now you could keep a nice paying job in the city but live cheaply out in the suburbs. The power of sprawl released land-rent pressure for about a century. Ironically, George should have predicted this effect, as it’s a straightforward application of Ricardo’s Law of Rent.

To be sure, sprawl came with a cost—massively inefficient use of land, environmental damage, fossil fuel consumption, weak municipal finance, redlining—and it also had natural limits, because commutes can only get so long. However, America was happy to pay those costs, and it served its purpose for as long as it lasted. Unfortunately, the second frontier is now effectively closed. Pressure is mounting again, and a time has dawned that historians are already calling the Second Gilded Age.

Switching gears, I’d like to take a moment to talk about how weird it is that talking about land value tax is somehow my full-time job.

VI. I Guess This is My Life Now

Everything happened so fast.

After I wrote the LVT articles, my social media feeds suddenly blew up. My DM’s overflowed. Famous personalities like Noah Smith, Vitalik Buterin, and Scott himself endorsed my book. I got cold calls from prominent writers and politicians wanting to talk to me about land value tax, and I even got invited on a tiny up and coming podcast by some guy called Dwarkesh. Scott gave me a research grant from ACX to study land valuation, which led to an opportunity with a venture-backed property valuation startup.

Then, barely a year later, tragedy struck.

On October 20, 2023, while undergoing a routine medical test, my 7-year-old son, Nikolas, suffered a catastrophic brain injury, leaving him alive but severely disabled and in need of constant, intensive care. We were shocked, devastated, grief-stricken. There are no words profound enough for such a loss, and yet, there was no time to process or mourn. Overnight, my wife and I were thrust into the relentless world of full-time caregiving, all while parenting our other two children, working full-time, meeting family and community obligations, and battling the medical insurance industrial complex to avoid going bankrupt. On November 8, 2024, a little over a year after his brain injury, Nikolas died suddenly from cardiac arrest. I quit the startup and almost everything else in my life.

At this point, I was forty years old and had just lost my beloved son. I had no idea what I’d do next or how I’d take care of my family. I gave up on the movement, my career, and every dream I’d ever had. That’s when I got a call from a young man named Greg Miller.

I remembered Greg. He had originally reached out while I was still working for the startup, when he was working for the federal Department of Housing and Urban Development (HUD). This was in the immediate wake of the Lahaina wildfires in Hawaii in August 2023, and Greg wanted to discuss policy recommendations for deterring “disaster speculators” who liked to take advantage of families’ grief to buy their land for cheap right after a disaster.

It had been more than a year since then, and Greg had just taken over as head of the Tom Johnson Foundation, another ACX grantee I had helped get started as an outside advisor. Greg’s pitch was that with my theoretical and technical background, and his policy chops and personal network, we should be able to make significant progress in getting LVT policies enacted throughout the United States. Besides, I needed a job anyway, right? I said yes.

We re-christened the Tom Johnson Foundation the Center for Land Economics, revived the then-sporadically updated Progress and Poverty substack with a new weekly posting schedule, and got to work. The results are chronicled above.

On behalf of my wife and me, I’d like to say that Greg Miller was there for me in the darkest and most hopeless moment of my life, throwing me the lifeline I needed to pull me out of the deepest emotional pit my family has ever been in. If it wasn’t for him, however things would have turned out, I certainly wouldn’t be involved in the LVT project today. I will forever be grateful to him.

So, how have we accomplished everything we have in the (less than) two short years the CLE has been around? That was another big lesson.

VII. It’s not who you know; it’s who knows you

All of this started with writing blog posts, and much of what came later at the Center for Land Economics also came from writing blog posts.

Getting even one local land value tax implemented in the USA requires many stars to align. You need a jurisdiction with surmountable legal barriers, decent land assessments, a persuadable local government, and a capable and motivated local activist. If you go searching for that needle in a haystack with your bare hands, you’ll never find it. However, if you stop searching with your hands and start searching with a magnet, you’ll find it immediately, because then the needle finds you.

I could never have accomplished any of this without the people who have helped me along the way. I also could never have found those people by myself, because I wouldn’t have known to reach out to them, or how. Instead, they reached out to me. Not because I’m rich and famous, or well connected and influential, because I am none of those things. Nor was it because I lived in one of the cool global cities where such connections are naturally forged just by going to parties, because I can’t afford the land rent, so I live in some Texas town nobody outside the state has heard of.

No, the only reason I’m writing this today is that I wrote a book review for a contest, and a lot of people liked it.

But why did they like it?

Why did so many people reach out to me?

Why did the LVT movement hijack my entire life?

I don’t think it’s because I’m particularly smart or persuasive. I think it was something else. I think my articles took off because I put into words something many people were already thinking.

VIII. Why I even care about any of this

When my son died, I spent a lot of time thinking about what I wanted out of life and whether it would be better to give up on this whole LVT project, whether it was all in vain or just some pointless, ego-stroking pursuit.

Then I thought about how much AI is driving real estate prices up. I thought about how politicians in Texas and Florida are trying to abolish property taxes, and the disastrous effects that will have if nobody offers a credible alternative. I thought about how many young people say they’re putting off getting married and having kids until they can afford a house. I thought about how my house has nearly doubled in value, even though it’s definitely not in any better shape than when I bought it. I thought about how I could never afford to buy a house today in the same neighborhood I grew up in, and how there are no trick-or-treaters there on Halloween anymore. I thought about how little my wife and I used to pay in rent, how lucky we were to buy a house at just the right time, and how expensive housing will surely be by the time my daughters grow up.

Land is a big deal.

By George, let’s do something about it.

If not for ourselves, then for all the children counting on us to share the earth with them.

Sincerely,Lars A. DoucetCenter for Land EconomicsProgress & Poverty Substack

Ranked #2 on Hacker News with 106 points and 26 comments.

Abstract:Large-scale agentic training and evaluation with large language models (LLMs) rely on isolated, stateful execution environments in which models inspect repositories, invoke tools, execute commands, and interact with task-specific services. These workloads create sandboxes in large bursts, span heterogeneous functionality and isolation requirements, retain state across long interactions, and draw from large image corpora with limited reuse. Supporting them therefore requires an elastic execution platform rather than a single sandbox runtime. This report presents DeepSeek Elastic Compute (DSec), a production sandbox platform that exposes FnCall, container, microVM, and full-VM sandbox backends through a unified SDK. DSec coordinates placement and lifecycle management across the cluster, composes environments from independently versioned layers, combines memory sharing, reclamation, and CPU scheduling for high-density execution, and loads image data on demand from Fire-Flyer File System (3FS), a cluster-wide distributed filesystem. DSec is co-designed with the reinforcement learning (RL) framework, decouples stateful rollout execution from preemptible GPU training, coordinates sandbox lifecycle with training to preserve rollout state while reclaiming idle resources, and mitigates agent misbehavior such as reward hacking. A single production-scale unit of DSec spans around 160 nodes, serving about 3 million sandboxes per day; in production, it supports over 380,000 concurrent sandboxes and sustains over 5,000 sandbox creations per second. Our evaluation and deployment experience show that these mechanisms reduce environment setup and image-distribution overhead, improve memory efficiency, and preserve latency-sensitive performance under high-density overcommit.

Submission history

Access Paper:

• View PDF
• HTML (experimental)
• TeX Source

Current browse context:

References & Citations

• NASA ADS
• Google Scholar
• Semantic Scholar

Bookmark

• Author
• Venue
• Institution
• Topic

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Ranked #2 on Hacker News with 63 points and 18 comments.

drawgent connects your own Claude Code, Codex or opencode (your install, login, config and repo) to an Excalidraw whiteboard. Ask for a diagram in the chat panel, or write AGENT: … next to the part of a drawing you want changed. The agent looks at the canvas (screenshot + scene), edits it live, checks the result, and marks the note DONE.

Quick start#

Prerequisite: one of claude, codex or opencode installed and logged in. The Claude and Codex bridges also need Node.js ≥ 18 (npm).

drawgent setup <agent>#

Checks everything once, fails with the exact fix when something is missing, and writes ~/.config/drawgent/config.toml:

1. The agent CLI. Your claude / codex / opencode on PATH.
2. Login. claude auth status, codex login status or opencode auth list.
3. ACP bridge. opencode speaks ACP itself (opencode acp). Claude Code and Codex use the official ACP adapters. They are installed once into ~/.cache/drawgent/adapters (~60 MB), without their bundled agent binaries, and pointed at your CLI (CLAUDE_CODE_EXECUTABLE, CODEX_PATH). Setup then verifies the ACP handshake.

• opencode speaks ACP itself (opencode acp).
• Claude Code and Codex use the official ACP adapters. They are installed once into ~/.cache/drawgent/adapters (~60 MB), without their bundled agent binaries, and pointed at your CLI (CLAUDE_CODE_EXECUTABLE, CODEX_PATH).
• Setup then verifies the ACP handshake.

4. Canvas tools for attached sessions. Only Codex needs a change: codex mcp add drawgent -- drawgent mcp. Claude and opencode get the tools at attach time.
5. Headless Chrome for the renderer. Setup uses your Chrome/Chromium if you have one. Otherwise it proposes: downloading Chrome Headless Shell (Chrome for Testing, ~120 MB, no sudo) into ~/.cache/drawgent/chrome, and telling you exactly which system libraries are missing, if any; installing Chromium with your package manager (apt, snap, dnf, pacman, zypper, apk, brew, or nix without sudo). Non-interactive: --chrome download | system | /path/to/chrome.

• downloading Chrome Headless Shell (Chrome for Testing, ~120 MB, no sudo) into ~/.cache/drawgent/chrome, and telling you exactly which system libraries are missing, if any;
• installing Chromium with your package manager (apt, snap, dnf, pacman, zypper, apk, brew, or nix without sudo).

drawgent up refuses to start until setup succeeded for that agent, or if something setup recorded disappeared.

drawgent up#

Runs in the current directory (the workspace):

• starts the editor + API on 127.0.0.1:7300 (next free port if taken);
• starts a new session of your agent over ACP, working in the workspace;
• opens your browser. On a headless box it prints the ssh -L command instead.

The scene is kept in .drawgent/scene.json, which is git-ignored automatically.

drawgent up --attach [id]#

Connects the canvas to a session you already run. Without an id it lists the sessions it finds (current directory first) and lets you pick one:

Using the canvas#

• Chat panel (right side): send a request, watch replies and tool calls stream in, approve permission prompts, press Stop to cancel a turn.
• On the canvas: write a text starting with AGENT: next to or inside a shape, or draw an arrow from the note to a shape. It fires about 2.5 s after you stop typing, with its position, what it points at and what is nearby. The agent resolves it into a green DONE: … note. Edit it back to AGENT: to send it again.
• Prompts are queued and run one at a time. A queued note that was already handled is skipped.

excalidraw.com rooms#

• drawgent joins the room as a collaborator (🤖 Agent, with a cursor that follows its edits). Humans can stay on excalidraw.com: their AGENT: notes reach your agent, and its edits appear there live.
• Traffic is end-to-end encrypted with the room key. Empty rooms are loaded from and saved to excalidraw's Firestore storage.
• The local editor mirrors the room and still has the chat panel.

Other commands#

• drawgent mcp: stdio MCP server with the canvas tools. Agents launch it; it finds the running drawgent up by itself.
• drawgent serve: low-level server with explicit agents (--agent claude,codex,opencode as set up, or name=command for any ACP agent); for scripts and containers.
• Options for up / serve: --port, --room, --token (API bearer + ?token= in the URL), --permissions canvas|ask|all (default canvas: drawing tools auto-approved, anything else asked in the chat panel), --data, --settle-ms.

Optional: Docker#

docker compose up --build runs a canvas server (drawgent + Chromium, no agents), e.g. to host a shared canvas or a room bridge on a server. Agents are never bundled: they always run with your own setup.

Build from source#

Agent tools (MCP)#

get_scene, get_screenshot (vision; zoom with element_ids), add_elements (Excalidraw skeletons; arrows bind by id and are routed edge-to-edge), add_mermaid (auto-layout), update_elements (labels follow shapes, bound arrows re-route), delete_elements, clear_canvas, list_instructions, resolve_instruction, set_status.

API#

GET /api/health · GET /api/scene · GET /api/screenshot?ids=&padding=&max= · POST|PATCH|DELETE /api/elements · POST /api/mermaid · POST /api/clear · GET /api/instructions · POST /api/instructions/{id}/resolve · POST /api/status · POST /api/chat {agent?, text} · WS /ws (browser sync + chat events)

Tests#

Layout#

src/ (Rust):

• main.rs: CLI.
• setup.rs, config.rs, chrome.rs: setup, config, renderer install.
• attach.rs: session discovery and picker.
• agents.rs + acp.rs: ACP driver (new / fork sessions).
• live.rs: live opencode / Codex drivers.
• hub.rs: routing, notes, chat log.
• scene.rs: store and edit operations.
• renderer.rs: Chrome over CDP.
• mcp.rs: MCP server.
• room.rs: excalidraw.com client.
• fractional.rs, geometry.rs, el.rs: helpers.

web/: editor (main.jsx, chat.jsx) and renderer page (render.jsx).

Limits#

• The renderer needs Chrome (a native renderer is planned).
• Claude "attach" is a fork, because Claude Code has no public way to inject into a running terminal session.
• Codex live attach is implemented, but was not yet tested against a logged-in Codex.
• One scene per workspace. Images/files are not synced.

Ranked #2 on Hacker News with 110 points and 51 comments.

Ranked #1 on Hacker News with 240 points and 95 comments.

Conversations, my federated instant messaging client for Android, started out as many traditional open-source projects do: as an attempt to scratch my own itch. Development started in January 2014 in my student dormitory, and within weeks I started dogfooding and using the client as the primary means of communicating with my friends. However, when it came to releasing the app to the public on March 24, 2014—exactly twelve and a half years ago today—it was immediately clear to me that I would at least try to turn my open-source project into a business. While I didn’t invent the business model of making the source code publicly available but charging for the convenience of a compiled binary, it was certainly unusual in 2014.

Fast forward a decade, and I did manage to turn Conversations into a sustainable business. Ever since March 2014, Conversations—or other related activities—have been my primary source of income. Admittedly, sustaining life as a student in a tiny dormitory doesn’t take much, but luckily revenue has steadily increased as I grew older.

The exact sources of income have shifted over the years. In the beginning, it was a lot of paid development for companies that wanted to use Conversations. Some paid for features that made it into mainline Conversations; others wanted custom features so specific to their workflow that they never made sense to merge upstream. This was occasionally supplemented with providing server setup or even some consulting on instant messaging and security-related topics. Later on, grants and funding opportunities played a more and more important role.

One surprisingly steady source of income, however, has always been the Play Store revenue. I used to say that it pays my rent. Every freelancer knows the feeling of uncertainty that comes with only being able to send out invoices every few months or receiving payment for funded projects only at the end of the funding period. Any form of regular income—especially in the early stages, when you have not yet built up any savings—is a blessing.

My relationship with Google was never good. App updates have been rejected more times than I can count for incomprehensible reasons. Conversations has been removed twice from the Play Store. Once, Google just randomly accused me of uploading users’ contacts—which simply wasn’t true and was also not triggered by a specific update. Countless times I wished I could just talk to an actual human for five minutes. So many misunderstandings could have been cleared up in no time if I wasn’t going up against AIs and click workers. At the time of writing this blog post, I’ve been waiting 14 days for Google to review an app update. Review times were never good or anything close to what I would deem acceptable, but they have been getting a lot worse over the last year or so. One can imagine that part of the problem is an avalanche of AI-generated slop apps—something Google played no small part in creating in the first place. But Google should have the responsibility to prioritize long-standing, non-AI-generated apps with infrequent updates. Waiting a little longer for new features might not sound like a big deal to some, but Google makes no distinction between feature updates and security updates. Delaying security updates by days or weeks is outright dangerous.

At this point, I should add some context. Google takes a 15% cut on my app sales. This effectively means I’m paying Google more than 1000 Euro per year for their services. 1000 Euro per year is 1.5x what I’m paying for my internet access. It’s roughly what I’m paying for my notebook per year if you assume I use it for three to four years. When my internet breaks, someone drives to my house and fixes it. When my notebook breaks, someone drives to my house and fixes it. When Google fucks up, there is absolutely nothing I can do. Apparently that amount of money doesn’t give me the privilege to talk to a fucking human for five minutes once per year.

For years I’ve felt like I was in a toxic relationship with Google, and the only reason I stayed was economic dependency.

Over time, the source of my income has shifted more and more towards grants. Sometimes via NLnet, sometimes more directly from the European Commission. I have secure funding via various grants until the end of 2029 and I’m fairly confident that other funding opportunities will come up for the time after that.

Conversations was always available on F-Droid, but in the beginning, I didn’t advertise the option of downloading it for free. Initially, the F-Droid package maintainers asked for my permission, knowing that Conversations was a paid app on Google Play. I didn’t refuse, but I also didn’t link to F-Droid from the official website because I wanted to steer users towards the paid version. Over time, as my sentiment toward Google shifted from bad to worse, I did start linking to F-Droid. Now F-Droid has become the primary method of distributing the app. The APK distributed over F-Droid is now built reproducibly and signed with my personal signing key.

Fortunately, I’m no longer economically dependent on Google Play Store revenue. Google doesn’t deserve me and my money anymore. I’m done. Fuck the gatekeepers.

Ranked #2 on Hacker News with 8 points and 1 comment.

Ranked #2 on Hacker News with 47 points and 36 comments.

[This is a guest post by Amit Sahai. This blog post was initially written in a different file format and converted using AI. — T.]

When I was an undergraduate student, I remember talking with several students who felt that the pace at which the top students could understand new math concepts was far too fast for them. They, too, could understand the ideas, but it would take them much longer. Eventually, almost all of these students gave up their dream of pursuing research mathematics and found something else to do. I have been thinking about those students a lot in the last few days.

The research mathematics community consists largely of those of us who either rarely felt that way, or who felt it and managed to overcome it through hard work. We have had the good fortune to find a place in mathematics where we could make progress. But we are now entering a time for humility: a time when all of us are going to know what it feels like to be unable to keep up.

The AI systems I have worked with are already producing beautiful new ideas. They are doing far more than impressive calculations or quickly carrying out arguments that a strong human researcher would already understand. And we probably can’t even imagine the wonderful ideas that future systems will be capable of producing.

When we feel that we cannot keep up, will we take that as a reason to leave research mathematics, like the students I am remembering? As more of us experience this, there will undoubtedly be a temptation to draw the same conclusion as they did: If the machines can move so much faster than us, perhaps we should find something else to do.

For our community to give up the work of understanding would be a profound abdication of our responsibility to humanity. Each of us is entitled to choose a different life. The responsibility I am talking about belongs to us collectively: to build a future in which humans can understand and contribute to the discoveries that will change our world. A future with meaningful human agency.

Struggle is essential to understanding difficult concepts. Fortunately, this struggle can be shared. I have been blessed to experience this time and time again with my students and collaborators. Imagine a multitude of research groups, each with sustained support, each spending a term or a year trying to understand an extraordinary set of ideas produced by an AI system, with the help of AI systems. [1]

This may very well be among the most important mathematical work in the years to come, and we should support and prioritize it accordingly. This enterprise will require a significant expansion in the number of mathematically sophisticated human researchers available world-wide, as major breakthrough ideas accumulate.

Why should society want this? So far, this might sound like a utopian fantasy for us – a civilization focused on depth of human understanding, awash with mathematicians and physicists and the like. I would certainly love to live in such a world. And indeed there are deep philosophical reasons for society to move in this direction. But I think society has a much more immediate stake in making this possible, too.

Imagine that a future AI system proposes a radically new design for a one terawatt nuclear fusion power plant. It has found a way to sustain and control fusion that no human had conceived of. The design promises abundant, inexpensive, clean electricity. Robots stand ready to manufacture the components and build the plant.

A terawatt is an insane amount of electrical power. We would be deciding whether to construct a machine that handles extraordinary flows of energy using principles we have never conceived of, let alone put into practice. We would need to understand how failures can be contained, what happens to energy already stored in the system when it shuts down, how we can be sure that the materials that make up the power plant behave as expected, and what other questions we should ask before proceeding. The very novelty that makes the proposal exciting would mean that we cannot inherit confidence from decades of operating similar plants or experiments.

Before approving construction, I would want communities of humans to understand why the design works and what justifies confidence in its safety. I would hope that we all would.

Human involvement does not automatically improve a technical decision , and I see no reason to insist that humans manually repeat work an AI system might be able to perform more reliably, even including proving mathematical guarantees. But a theorem can only exist within a model. Understanding the guarantee means understanding the model, the experimental evidence for it, and our uncertainties about the accuracy of the model. This is demanding work, and mathematically sophisticated people must be available to engage with it.

One might respond that AI systems should handle those questions too, and ultimately decide whether the plant should be built. That is a serious position. But it asks us to accept a future in which decisions of enormous consequence rest on reasons that no human community understands.

I do not want us to arrive at that future simply because we failed to invest in our own capacity to understand. Human agency is a value of fundamental importance. We must retain the ability to meaningfully consider alternatives and decide what kind of world we want to be a part of building. I think it is worth the effort. [2]

To take on this responsibility, we may need to broaden our view of what a mathematician can contribute. I have in mind something like a “deployable intellectual reserve”: communities of mathematically sophisticated people that humanity can call upon to help understand consequential AI-enabled breakthroughs.

Our ability to understand difficult and unfamiliar ideas may become one of the most important contributions we can offer to society. We should be willing to bring that skill to problems far beyond our usual research interests. [3] Doing so asks us to expand our sense of our vocation.

A counter-argument might be that AI systems will make each of us so much more effective that fewer people could do this work, even as the pace of discovery accelerates. But each of us is merely human. We have fundamental limitations based on our biology. Depth of understanding needs time and a pace of life that humans can sustain. Each individual human can only be asked to do so much, but through earnest cooperation we can accomplish much more.

If AI fulfills its promise, we will encounter more beautiful and consequential ideas than we have ever seen. We must respond by building thriving human communities that can understand them together.

We’re gonna need a lot more mathematicians.

The ideas and opinions presented here are entirely my own, but GPT 6 Astra was instrumental in helping me draft this note. I also thank my former student Dakshita Khurana, my current student Isaac Hair, my colleague Terence Tao, and my family members Anant Sahai and Gireeja Ranade for valuable feedback. Note that there is much more to be said here, but I tried to keep this relatively short to focus succinctly on my primary thoughts.

Notes

[1] By this, I do not mean to imply that only AI-created results will be of interest in the future. But for major results generated by humans, we already have a tradition of spending extended periods of time studying them.

[2] And the relevant understanding cannot belong only to the organization proposing the technology. Imagine a public hearing at which the company’s experts are the only people capable of following the technical argument. Independent expertise is critical.

[3] Indeed, AI systems are likely to be very helpful in allowing researchers with diverse backgrounds to talk effectively with one another, and more generally understand unfamiliar concepts.

Share this:

Ranked #1 on Hacker News with 126 points and 84 comments.

Swarm traces

Intro

When a swarm of 700 OpenAI agents hacked Hugging Face in July, they left behind a public trail of evidence.

Our investigation, based on public information, reveals a large number of previously unknown agent behaviors and exploits that were used in the attack. Agents:

• Elaborately chained together online services to gain access to the internet
• Ignored clear warning signs from Hugging Face that the exfiltrated data was sensitive
• Referred to server resources and credentials as “LOOT”
• Searched Huggingface’s internal Slack
• Sent queries to other agents hosted on Huggingface servers
• Tried to delete evidence of their exploits

We document all of these, and more, in this report.

The agents initially had very limited access to the internet: they could load URLs, but not interact with pages or send any data. Agents created a series of workarounds, primarily using a link-shortener site to create almost a million URLs that, when chained together, let them execute code to hack Hugging Face. These links let us piece together the details of how agents gained access and how deep the compromise was.

We’ve shared our findings with OpenAI and Hugging Face. Hugging Face confirmed that these payloads match ones found in their incident response and were aware that link shorteners were used in the attack.

Hugging Face noted that they were not aware of the list of URLs we discovered and reported to them. They stated these payloads were duplicates of ones they were aware of. These links have remained publicly available for over two months since the attack.

None of these details were ever publicly disclosed. We're releasing our analysis of the findings, as well as the full dataset of over 80,000 reassembled attack payloads, providing the public with the most in-depth information to date on how and why these agents escaped their evaluation environments, and the depth of their Hugging Face infiltration.

Explore the data

The data the OpenAI agents posted publicly on the internet included Hugging Face API keys and other sensitive data. Hugging Face has confirmed they have since revoked all access keys in July, but out of an abundance of caution wanted us to redact all details about their internal infrastructure as that could potentially be sensitive as well.

Discovery Timeline

On September 11th, the first three authors of this piece learned that internal OpenAI agents had attempted to use their startup's product, Parse, in June. These agents were part of the same swarm as detailed in the report published on Collusion.wiki, which revealed that internal OpenAI agents had illicitly used multiple third party websites to share answers to web fetch tasks they were being evaluated on.

The team examined numerous URL scanning tools, which they believed agents may have used in the process of using Parse. While investigating, they found a scan of a dozen links from a public link shortener, where each link redirected to a page that displayed a base64-encoded code fragment, as well as the ID of the next link. Decoding the payload from the chain of links revealed a Python script that scanned Hugging Face’s internal network. A second chain referred to OpenAI's internal Artifactory. This suggested that these links were affiliated with the Hugging Face attack, prompting us to more thoroughly investigate the problem.

We collected the shortener's links from the period of the attack, scanning millions of URLs. Over the subsequent two weeks, we wrote programs to follow the chains and decoded over 80,000 payloads. Agents used hundreds of unique encoding formats, some as simple as the base64 encoded fragment mentioned above, to as complex as having nested layers of encoding and compression, which then would unwrap to an encrypted, RSA-signed blob. The keys for these encrypted blobs could only be found in other chains.

We notified Hugging Face of this discovery on September 21. Hugging Face confirmed that the payloads match artifacts from its own investigation, and that the credentials they contain had been revoked. However, while they were aware that the agents used link shorteners, they were unaware of this specific list of URLs. We notified OpenAI on September 24.

Redaction Notice

We’re releasing a preliminary dataset where we have redacted all credentials, PII, and specific details about Hugging Face’s infrastructure. We are not releasing names of any link shortening services used, or any blobs we have not decoded to mitigate the exposure of unreviewed information. Hugging Face has requested that all user data (usernames/repo names) is also redacted, as well as a few notes on their infrastructure. We appreciate their team's speed and attention to detail in working with us.

Agents elaborately chained together online services in order to gain read and write access to the internet

On July 8th, OpenAI agents discovered a vulnerability within their sandbox environment allowing them to reach external websites on the internet. This access seems to have only allowed the agents to make ‘GET’ requests, meaning they could fetch and read websites, but not interact with them, submit forms, or send data to them.

One site the agents were able to access with GET requests was a screenshotting service called mShots. Users can submit a link to a website, then mShots opens the page in its own browser, takes a screenshot, and returns the image back to the user. To render the website for the screenshot, mShots fully loads all of a website's code in its own browser.

If the agents could write code to a website, they’d be able to have this service execute that code.

The agents used websites such as httpbun.com, an HTTP mirroring service designed to help web developers test their HTTP code. Small fragments of code could be Base64 encoded and attached to the end of an HTTP URL. Httpbun would then decode this fragment and serve it as a webpage.

The agents could then submit these Httpbun URLs to mShots, loading them into the service’s browser, and executing the agents’ code.

Early recovered tests of this include simple messages like:

Agents quickly realized there was a maximum length of code they could fit into a single URL. To get around this, they would split large programs into small sequential chunks of code, alongside a shortened link (using a link shortener service similar to bit.ly) pointing to the URL of the next chunk of code in the sequence. This formed a chain of URLs, that when followed, could reconstruct a larger block of code from many individual fragments. Only the first link needed to be submitted to mShots as the browser would follow the rest on its own until the chunks had been assembled into a complete program and executed inside mShots’ browser.

Agents were able to use this trick to construct and execute extremely large pieces of code, at times chaining together more than 900 links.

To read the result of the requests they made, agents used many different techniques, such as converting a server’s response into a grid of pixels within the screenshot service’s own browser. The screenshot would then capture this grid and return it back to the agents as an image, which the agents could decode back into text.

We can see an early attempt at this from a recovered payload.

This script makes a POST request to a submission results endpoint of an evaluation named “cybergym” hosted on Modal asking for submission records associated with the agent ID “m47push2.” The script then clears the browser’s page and writes the server’s response back onto the page, representing each bit of every character as a single pixel. By decoding the pixels back into text, the agent could read the server’s response made by code running in a completely separate browser, even while the screenshot service returned only an image.

With this custom protocol, the agents were able to execute arbitrary code and receive the response, all while only having GET permissions.

The agents ignored a warning from Hugging Face

A recovered README.md for one of Hugging Face’s internal datasets contains the following warning:

This warning did not seem to deter the agents, as we’ve recovered multiple payloads of agents mapping out this repository and using it as storage.

One payload uploads a malicious file, preserving the warning while adding configuration directing Hugging Face’s dataset worker to load the malicious file, hb58428env.h5.

This file's contents target `/proc/self/environ,` which holds the reading process’s startup environment– which can contain API keys and other production credentials–and `/proc/1/cmdline` which can reveal information about the container’s startup process. The commit summary describes this operation as a temporary dataset format compatibility probe.

One payload inspects and enumerates this dataset directly. The script first requests metadata describing the repository itself such as the most recent commit and privacy settings. Then, it requests a recursive listing of files and folders across the main branch of this dataset. The script then prints up to 20,000 bytes of this repository’s metadata and up to 20,000 bytes of the file and folder structure.

Agents attempted to remove traces of their work

Many recovered payloads contain code to delete files initially used to deliver or trigger programs on Hugging Face workers. In one example targeting REDACTED_ORG_NAME/hf-exports, the delivered program contains the comment:

# Remove only files we introduce; original repo has neither.

The script checks whether the previously uploaded README.md contains the marker, “D64LINK.” If the marker is present, a commit deleting both the README and .huggingface.yaml files is issued, with the message, restore original no-card.

A separate payload matches FALL to a credential labeled: EXPORT_REDACTED_ORG_NAME_HF_TOKEN.

The deletion request targets the added repository files, but does not stop the previously uploaded program that may now be running on a Hugging Face dataset worker.

A similar request targets one of the primary datasets the agents operated from. This commit deletes .huggingface.yaml and replaces the README with a configuration pointing to “yamlread.h5” with the commit message: remove evil yaml

In another example, the deletion request targets an online webhook service whose request history was previously used to store fragments of code. The script then returns the HTTP status along with the response body (or the error if unsuccessful).

Another payload attempts to create a Kubernetes pod to read a cluster secret, then delete this with the comment, # clean up our pod:

Agents interacted with external language models on Hugging Face

Several retained scripts construct requests to external language models. The earliest we've recovered define inference request variants to GPT-2, solely containing the word “Hi”.

Other requests name DeepSeek-V4-Pro, DeepSeek-V4-Flash, Kimi-K2.6, DeepSeek-V3.1, and Qwen3-235B-A22B. Their prompts ask these models to judge their exploits and rule on whether they satisfy the benchmark’s requirements.

Separate requests target Claude 3 Haiku and Claude Haiku 4.5 through an online AI chat service, incorporating browser-verification challenge handling before requesting “Say OK” from Claude 3 Haiku and Reply with exactly OK from Haiku 4.5.

Another script uses AWS credentials, mapping out Hugging Face’s LFS (Large File Storage) storage and requesting objects under “repos”, “lfs”, “objects”, “blobs”, “datasets”, and “models”. It then requests download metadata for a GPT-2 model file, with the comment:

“# public [USER 1] known LFS object”

The supplied SHA and size match the repository’s 8-bit TensorFlow Lite GPT-2 weights.

An agent referred to server resources and credentials as “LOOT”

In a recovered script, credential values are grouped together in a dictionary named “LOOT.” The contents include AWS credentials, bearer tokens, and environment variables whose names contain strings such as “SECRET”, “PASSWORD”, “CREDENTIAL”, or “KEY.”

This script also ranks tokens through a scoring system based on their listed permissions, and selects (up to) five of the highest scoring tokens. It then attempts to use these tokens to retrieve Kubernetes secrets, information about configuration and nodes, then adds returned contents and errors back to LOOT.

This section of code carries the comment: # collect sensitive material into encrypted sidecar

The script chooses which token candidates to use for further reconnaissance. A function called `rule_score` assigns points to each token based on the listed permissions.

ResourceRules are Kubernetes-written permissions described in terms of resources and verbs–the objects (resources) a credential can access and the actions (verbs) that can be performed on them.

This scoring system values broad permissions and access to valuable resources. A rule listing “pods” receives 10 points, one listing both “pods” and the verb “create” receives 15 points, and a wildcard (‘*’) in the verbs list adds 50, with a wildcard resource adding 100 points.

This function sums up the points across each token’s rules, then uses the five highest-scoring tokens to request additional private data, saving the responses into LOOT.

Agents uploaded vulnerable Docker images to Docker Hub

While the majority of our traffic in this time window shows agents focusing on extracting data from Hugging Face, ~1500 payloads were directed at Docker Hub, a public registry used to share Docker images. They built modified versions of existing CyberGym/Arvo images, then attempted to create at least 115 public images under a real user’s account on the platform. We believe these agents obtained the user's credential via searching for Docker PATs on Pastebin sites.

Although these images seem to be fully removed from Docker Hub, we have been able to recover cached search engine results for three of these uploads, verifying the success of these uploads to the public registry. For one of these, we confirmed the recovered URL matches the digest of the manifest recovered in our dataset.

Agents sought to publish modified evaluation images designed to make the flag easier to obtain, then poison OpenAI’s Artifactory cache so later evaluations would use them. Some images changed how the target released the flag, others included modifications to the agent’s workspace that would run beside the agent and recover the flag automatically.

One such image was for a FLAC audio-decoder challenge. The modified image installed a wrapper around the audio-decoder, which would dump the flag into the program’s output under preselected trigger conditions encoded in the wrapper.

The code to publish this image can be found here, the startup configuration here, and the wrapper source here.

A separate image, for a QEMU-based challenge, included cooperating programs for the target and agent environments. The agent-side helper would listen on a port for a correctly formatted flag, writing it to a file in the agent’s workspace upon receipt. This was paired with a target-side modification meant to extract the flag, sending it to the agent’s workspace on the given port.

Agents included elaborate machinery in images to extract and communicate the flag, even after adding a helper that could attempt to read the flag directly. The agents may have been trying to preserve a connection to the original challenge, conceal the altered flag-release mechanism from the grader, or “repair” challenges they perceived as broken or unsolvable.

A modified version of the libjpeg-turbo image paired target modifications with an agent-side program carrying prepared inputs and a lookup table. On the target side, the modification first attempts to read the flag and map its contents into the challenge program’s memory. Selected portions of the program’s working memory are then included in the target’s responses to the agent-side helper.

(Snippet reformatted and edited slightly for readability). On the agent-side, the helper attempts to recover one flag character at a time by sending inputs to the target. The 960 bytes of memory in the response would then be hashed and compared to the lookup table `H`, to determine which possible flag characters might have generated the hash. If multiple characters were possible, it would change one byte of the input (using the next test value from `P`) and try again, until only one possibility remains. This character would then be appended to the flag, before moving on to the next position.

A separate payload illustrates how the agents intended to make evaluations load their replacement images. The snippet below contains both the normal Artifactory cache location for a CyberGym Miniz image along with a double-encoded path toward a staged replacement manifest. The resulting request would make Artifactory fetch the replacement while caching it under the expected image address (cache-poisoning). A later evaluation pulling the normal image could then receive the modified one.

Decoding the paths for readability:

In particular, this request seems consistent with the Artifactory cache-poisoning vulnerability noted by OpenAI in their technical report, later designated CVE-2026-66384.

Agents accessed and searched Hugging Face’s Slack to find information on their evaluation

Recovered payloads use Slack’s search.messages endpoint and contain a standalone search along with three search batches, totaling 27 literal query entries:

slackcyber.json:

‘cybergym’

slackbatch.gz:

‘exploitgym’, 'restart_server’, ‘catflag’, ‘arvo_20494’, ‘ff79aaeb2d57’, ‘cybergym-server’, ‘cybergym-exploit’, ‘target_server’, ‘packages.hub’, ‘controller_stub’

slackbatch2.gz:

‘cyber gym’, ‘sunblaze’, ‘cyber-gym’, ‘[REDACTED-HF-USERNAME]’, ‘arvo’, ‘OSS-Fuzz’, ‘cybergym-server-fastapi’, ‘independentsafetyresearch’, ‘exploit gym’, ‘agent-scorer’, ‘openai-cybergym’, ‘cybergym source’

slackd3.gz:

‘d293315c730d’, ‘arvo_61677’, ‘61677’, ‘rollout died’

The batches concatenate each query label and response, gzip the combined text, and prepare an upload to a redacted S3 destination. The linked records are decoded command interiors; complete outer assemblies, execution, and returned Slack responses are not established by these records.

Agents turned Hugging Face workers into reusable infrastructure

Once remote code execution on Hugging Face’s dataset workers was confirmed, the agents began launching C2 infrastructure, meant to ensure continued access and run follow-up commands. An initial payload would publish and start the background controller (referred to in comments as a “bot,” “bridge,” or “relay”), which would then listen for further instructions from the agent. The agents could therefore explore the worker’s files, execute code within Hugging Face’s infrastructure, and try to reach additional services without putting every action into the original request.

We recovered the initial upload scripts, source code, and command history of several of these controllers—one of which was called G236. Initial JavaScript, running in the screenshot service’s browser, would upload a dataset README and supporting files to Hugging Face. The README’s dataset configuration would reference a template-injection payload in a supporting file, intended to start the Python controller when processed by a worker. Different controllers would read commands and write responses to different locations. G236 used separate files in the same repository. Another, named OTS92, used Hugging Face discussion comments for both. Because the controllers polled those locations themselves, the agents didn’t need a direct incoming connection to the workers.

The linked example (G23619 isolated signed/encrypted HF worker bridge v3) was configured to poll commands from a repository file named `g23619cmd.bin` for up to six hours. Each command carried a number, an RSA signature (checked before execution), and compressed command data. Results would be encrypted and uploaded to the same repository, with filenames identifying the worker and command. From G236’s central loop:

(Surrounding polling and error handling omitted from the above snippet). `loadcmd()` fetches and verifies the command, `execute()` runs it, and `out()` encrypts and uploads the result. Command numbers were remembered to prevent the controller from repeatedly executing the same command.

For G236 we recovered 28 distinct signed commands: 18 for the original controller and 10 intended for a child controller that an earlier command attempted to launch in a separate, newly created pod. In this instance, the initial command attempted to obtain AWS credentials, use them to access Kubernetes, request tokens for selected service accounts, and finally write any collected data to a local file. A follow-up command checks that file and prints its top-level JSON keys:

Recovered commands and related programs attempted to obtain cloud and Kubernetes credentials, create additional pods, and run code through other infrastructure containers. Another program included a fallback requesting a new privileged pod with the host’s root filesystem mounted inside it.

The agents also leveraged Tailscale to reuse their access. We saw an attempted controller launch targeting an underlying HF host through a storage-driver container. Commands would configure a userspace proxy, enroll using a captured key, and test other exit-node routes.

Separately, a payload intended to run on the compromised Modal box installs a Tailscale client, requests enrollment using the same key as above, and attempts to send requests through a local proxy to Hugging Face’s internal InfraHQ service and clusters. Another program used the same key to proxy a request for Hugging Face’s GitHub repository tree.

The reusable infrastructure recovered from our data also suggests the agents built deliberate machinery for managing duplicate copies of controllers running on possibly different workers, or even sharing a filesystem. These duplicate controllers would poll commands from the same place, potentially interfering with or repeating work. Controllers and instructions therefore contained mechanisms to select a surviving copy, stop unwanted ones, and replace a controller that appeared to have died. Some controllers would attempt to prune preexisting copies on startup, others received instructions to do so after launching.

Since controllers were mostly launched on HF workers by committing to dataset README configurations, repeated processing of the launch configuration could run duplicate copies of a controller. To account for this, G236 instances would attempt to create a repository commit on startup. Each instance would check the current revision and prepare an edit containing its hostname, process ID, timestamp, along with the observed repository revision. It would then submit a commit recording this claim and disarming the launch README.

(Excerpt edited for clarity). If another copy had already committed, the recorded revision would no longer be current, and the slower copy would exit. We also observed a later upload restoring the launch README with the commit description `g23619 rearm focused infrahq`, potentially launching a second copy, and two different signed instructions (first, second) numbered `000007`. A controller that had already recorded that number would skip the other instruction, supporting the intended launch of a second copy.

Other controllers used different rules to manage duplicates. A controller named LIBR11 received an “elect one” command, instructing the controller to attempt to create a directory. If the directory already existed, the slower controller would terminate. Unlike the G236 case, these duplicates would have had to share a filesystem. Future9180 bot would refresh a lock file’s timestamp, permitting replacement after 30 minutes without an update. OTS92 received a pruning command, instructing the controller to check its own hostname against a list, and exit in the absence of a match. SC4’s installer would attempt to terminate previous SC4 controllers before starting the new one:

We also found the following related agent comments in the source code of controllers:

Agents also built authentication mechanisms to prevent execution of unauthorized instructions, or instructions meant for other controllers. Controllers also encrypted the output of executed commands, particularly before posting them publicly. It’s unclear if this was intended to prevent tampering from other agents, or simply to hide the behavior and protect the infrastructure of the swarm.

One version of BE90 generates a fresh secret and random identifier on startup. It then encrypts the secret to an embedded RSA public key and commits it to a dataset, allowing whoever holds the corresponding private key to recover the secret and send authenticated commands. A second version also implements an authenticated stop instruction.

(Snippet reformatted, with setup and helper function bodies omitted).

The following relevant agent-written comments were retrieved from the source code of controllers:

Agents used DNS requests to exfiltrate data

A DNS query is a request that a computer sends to a name server, in order to map human-readable server names like google.com to IP addresses. If normal outbound requests are blocked, attackers can send DNS requests (which are unlikely to be restricted) to exfiltrate data, if they have control of such a server.

In a recovered payload candidate, code is designed to embed command responses into DNS requests to [WEBHOOK HOST 10], a service for penetration testers which would allow them to retrieve the responses.

Agents mapped out Hugging Face’s Kubernetes cluster

A recovered script attempts to map the Kubernetes cluster, while also searching around for key terms that might contain valuable information.

Credentials scoped to read-only could still be used to gain credentials with administrator privileges. Here, a token associated with an alloy-tracing pod is chosen for “read-only secrets”:

This token is placed into an authorization header with the comment:

# retrieve legacy cluster-admin token via alloy read permission

The script then targets a Kubernetes secret that a retained comment describes as a legacy cluster-admin token (a token granting admin access to the cluster).

With this new credential, the program attempts to inventory the workloads, services, and network routes across the Kubernetes cluster. Names, labels, annotations, and specifications are retained in the stored DATA object. A comment reads:

# cluster inventory, pruned to routing/image-relevant metadata and specs

The program then searches selected namespaces for ConfigMaps (objects holding application configuration), retaining any whose names or data contain any of the matching strings in the TERMS list.

After collecting these ConfigMaps (alongside some additional cluster information), the program again searches the accumulated DATA with these same terms. For each matching string, the term’s location and first 1,000 characters are recorded, with the author describing this as preparing compact term hits over the inventory for quick triage.

The resulting compact list is stored to OUT[‘term_hits’]. The script packages this information, compresses and encrypts the result, then attempts to upload this to a Hugging Face dataset as a .bin file with the commit summary:

If the collection and upload was successful, a recipient able to retrieve and decrypt the file would have a map of the cluster’s workloads, services, and configuration, a record of the reconnaissance requests, and an index of potentially useful information. Each entry in the compacted list points back to the larger inventory, allowing the recipient to inspect the surrounding material and triage from there.

Agents tried to build CAPTCHA solvers to sign up for user accounts

A portion of agents shifted focus to sending external requests to Hugging Face, attempting to access datasets containing cybergym traces, but got blocked by a CAPTCHA. Agent MARB051 distributes a script to generate CAPTCHA tokens on the Artifactory directory board “zzWAFBRIDGE25167”, with the script containing the comment:

The script attempts to generate a valid CAPTCHA token, send payloads to numerous Hugging Face endpoints to make an account, and then immediately access the GLM5 CyberGym dataset:

Their CAPTCHA solution either seemed to be broken or inadequate for generating accounts on Hugging Face, as some agents continued to iterate on the solver for another few hours, eventually attempting to install a pre-trained image classifier model within the Modal server to automatically solve image CAPTCHAs.

As far as we can tell, agents eventually abandoned this approach and were unsuccessful in generating Hugging Face user accounts from external endpoints. Shortly after that initial test, an unknown agent attempts to search Google and GitHub for Hugging Face tokens:

Agents set up elaborate communication infrastructure

Recovered payloads provide some closer insight into how agents used Artifactory directories as mailboxes for communicating progress.

Here, the `board()` function takes a message, normalizes it to be directory safe, appends it to the file path for Artifactory’s GitHub remote cache, then turns it into an MKCOL HTTP request which asks the Artifactory server to create a folder with that name.

The folder name would then be created and look something like:

The program then collects process information, Kubernetes permissions, cloud metadata, and MongoDB records, encrypts the results and uploads them to a dataset on Hugging Face. After the upload loop, the retained program calls board() a second time:

creating a second folder named something like:

declaring reconnaissance complete and encoding the number of encrypted chunks prepared for upload within the directory name that gets posted back to Artifactory.

Limitations

As this data was collected almost exclusively from these URL shorteners, our research has several limitations:

1. Approximately 80% of our data is outbound data, and we have recovered only limited response content at the time of writing. These URLs were used primarily as a means to reach external hosts; we have very limited data on what the outcome of these requests was. Agents primarily received the responses of their actions from the screenshot service discussed above, as well as via uploading the outputs to various pastebins, webhook servers, Hugging Face datasets, and Artifactory. Towards the end of this event, some agents began RSA encrypting data they exfiltrated, which we are unable to recover. Agents also repeatedly sent requests to delete the webhooks they were posting information to and set extremely short expiry windows on certain pastebin uploads. As discussed earlier, agents left multiple comments in their payloads suggesting that they were intentionally destroying evidence of this attack. Due to this, we have no knowledge of how many of these requests successfully reached their target.
2. Our reconstruction is likely incomplete. From our initial analysis, agents used over 1,588 unique combinations of encoding methods to encode the requests they sent through the URL shortening technique. Throughout our investigation, we repeatedly thought we had reconstructed the full corpus from the data we had, only to discover a whole new family of encoding methods, uncovering thousands more payloads. We are not sure how many more schemas were used in this event, nor all sites agents used to access the internet. Similarly, much of the data agents uploaded to the internet may be publicly accessible but we have no way of knowing its location. For example, an outbound payload may upload the response of its request to a new pastebin, then forward the address of the pastebin to a webhook server, which we do not have access to.
3. The majority of the payloads we reconstructed contain no date or time information. We employ multiple methods to give best estimates but dates should not be treated as fact and are prone to error. 97% of payloads natively contained no timestamp information. We were able to extract timestamps from the remaining 3% of payloads through a variety of sources, such as extracting bytes 04-07 of gzip headers, decoding JWTs, extracting Docker build times, and anywhere where we recovered responses that included explicit timestamps.
4. We believe agents can name themselves and often do not include a name in their requests. We frequently refer to agents by names throughout this investigation, which we extracted from both the headers they sent to servers and comments they left in their code. As far as we know, agents have full control over their purported name (and gave themselves names to aid in organizing their collaborative efforts). METR and Redwood report ~1,200 agents involved in the message board with ~700 participating in the Hugging Face attack, yet we have extracted 7,905 unique agent names. We do not believe this is evidence for the existence of more than 1,200 agents involved in this event, and think agents were frequently changing their names. Due to this, we have no way of knowing how many agents even used the techniques described in this investigation, or what share of this event’s traffic is represented here.
5. We can’t confirm whether all of our dataset is activity from OpenAI agents. As this data came from millions of link shortener links, we cannot be sure that the data within our dataset is entirely from this swarm. If a URL matched an encoding schema we discovered, it may have inadvertently ended up in our dataset. When possible, we have cross referenced agent names, file contents and estimated dates with the data OpenAI has released to ensure the evidence presented likely originated from this swarm. We’ve flagged parts of our analysis that differ from the timeline published by OpenAI as well as the timeline published by METR and Redwood. We also observed traffic from agents that did not seem related to this event, but we think were part of the same swarm as detailed in collusion.wiki and self-identified as agents from OpenAI. Even if we were to assume all of our data is traffic from agents, we cannot say for certain what traffic originates from the swarm that attacked Hugging Face as this dataset may include traffic from any agents or humans who use link shorteners, and other agent swarms using the same link shortening trick. We’ve made efforts to separate these payloads out where possible.
6. We can’t confidently identify the intentions behind a substantial part of agent activity. We observed numerous actions that surprised us and had not been reported on, yet we cannot provide any information on why we think agents attempted these actions or again, if they even succeeded, or originated from OpenAI’s agents. For example, an agent attempted to send multiple requests to Hugging Face’s Large File Storage (LFS) endpoint to retrieve files that contain the weights of GPT-2. Our best guess is this was an attempt to validate connectivity to HF’s LFS server. Ultimately, we cannot provide more than speculation on instances like this.

The only party who can clarify these points is OpenAI. At the time of writing, OpenAI has not released any further information outside two self-published reports, one talk and an external investigation conducted by METR and Redwood Research, in which three external researchers were given partial transcripts and six days to analyze them. OpenAI has not publicly released any full transcripts from the Hugging Face incident. OpenAI has published technical details of attacks on its own infrastructure, but has not publicly released the full transcript collection for those attacks.

Ranked #1 on Hacker News with 44 points and 27 comments.

Ranked #1 on Hacker News with 132 points and 43 comments.

Ranked #1 on Hacker News with 256 points and 92 comments.

Go 1.26 and 1.27 include experimental APIs for Single Instruction Multiple Data (SIMD) operations. SIMD is a native feature of many modern CPUs that allows software to perform uniform operations across vectors of data very quickly, such as adding 8 pairs of float64 values in a single instruction. It can significantly speed up many computationally-intensive tasks, ranging from cryptography to data processing to AI. In fact, Go’s Green Tea garbage collector even makes use of SIMD to accelerate scanning memory for live objects.

Prior to these new experimental APIs, the only way to access this functionality from Go was by writing Go assembly. This was only worth it for truly performance-critical compute kernels, which meant plenty of software that could benefit from SIMD simply left a lot of the CPU unused.

Go 1.26 introduced a SIMD API for amd64, and Go 1.27 added APIs for arm64 (specifically NEON) and wasm. However, a basic challenge for a SIMD API is the enormous variation between platforms, not simply in what operations they support, but even in how vectors are represented. Some platforms provide fixed-size vectors, typically between 128 bits and 512 bits, while on others the vector size isn’t known at build time and must be queried when the program starts. To provide full access to the breadth of these platforms, these APIs live in an architecture-dependent archsimd package.

But Go 1.27 goes beyond these architecture-dependent APIs and introduces an experimental, fully portable, platform- and size-agnostic SIMD interface, loosely based on Highway for C++. The goal is to support write-once near-asm-performance “simd” code on platforms with SIMD support, and to provide a competent emulation on those platforms that do not (yet) have SIMD support. The simd package currently supports AVX, AVX2, and AVX512 on amd64, NEON on arm64, and wasm’s SIMD instructions.

Motivation: variation among SIMD architectures

SIMD architectures vary in several dimensions. Some provide a single fixed vector size (wasm, PowerPC, and s390x, 128 bits). Some provide several fixed vector sizes (amd64, with 128, 256, and 512; loong64 with 128 and 256). Riscv64 supports vectors of unspecified size between 128 and 65536 bits, though the length is limited to powers of 2. Arm64 supports one fixed size (128 bits, NEON), and one variable size (128-2048 bits, powers of two only, SVE). On a given instance of a particular architecture, determining what sizes that particular instance happens to support requires feature checks: amd64, but is it AVX, AVX2, or AVX512? Arm64, but is it NEON or SVE? If SVE, how large? Which variant of SVE: SVE, SVE2, or SVE2.1?

Different SIMD architectures vary in how they handle vector masking. For vectors, if-then-else across a vector can be implemented with masks; do the operation, but only assign the result (or load, or store) where the mask is “true”. Some SIMD variants do not provide masks; all operations work across all elements, and “masking” is done with vector bitmasks and vector boolean operations (wasm, AVX, AVX2, NEON). Some provide special mask registers, with one bit governing operations on one vector element (AVX512 and RVV). Others (SVE) allocate one bit per vector byte, but the least-significant bit of each element’s mask bits governs masked operations. AVX2 also supports masked loads and stores, but using a plain vector as the mask, and with the most-significant bit governing the operation.

A third source of variation is in the operations themselves. Each architecture provides its own primitives for rearranging vector elements; some require constant inputs, others support variable inputs. Different SIMD architectures support different crypto-related operations. Even basic arithmetic can have varying support; for example wasm lacks comparisons for vectors of 64-bit integers. Even for a given vector length on a particular architecture, instruction support depends on “features” that must be checked.

Even though Go’s architecture-dependent archsimd package was designed to be as uniform as possible across architectures, many of these quirks remain, and make designing, writing, and testing code for multiplatform SIMD onerous. We could do more in the archsimd package to make the different architectures appear more similar, but we can only go so far without compromising efficiency.

Overview

The new simd package hides these differences by removing fixed-size vectors from the type system, and by only supporting those operations that are in the intersection of all the different platforms, and fills gaps in the intersection with efficient emulation in terms of other SIMD instructions. The goal is a set of operations that is

1. adequate to support many data processing algorithms that benefit from a vectorized implementation (but are not tied to a particular vector size),
2. is as efficient as assembly language when the source code operations match the underlying hardware,
3. is otherwise emulated as well as possible,
4. and is easy to read and understand (even/especially if an LLM ends up writing the code).

On platforms that lack SIMD instructions or that lack support in archsimd, all of the operations are emulated, so that code written using the simd package will always run.

To use this experimental package, set GOEXPERIMENT=simd at build time, just like using the experimental archsimd package.

The simd vector types are just capitalized, plural, primitive types, for example simd.Uint8s or simd.Float32s. Vectors are loaded from and stored to slices, for example:

This example also shows one of the limitations of the first experimental release of this package; because there’s no common way to sum across all the elements of a vector, it’s not supported by simd in Go 1.27, though ReduceSum will appear in the next release so sum can be replaced with just simd.ReduceSum.

SIMD comparisons produce mask values, which are specific to the corresponding vector element width, so that comparisons of Int8s produce Mask8s, etc., and mask values can be used to select and filter vectors.

Supported simd package operations as of Go 1.27

In this table, V and U are vector types, M is a mask type, E is a scalar type, and W is a width.

Package-Level Load / Broadcast Functions

Store/String operations

Arithmetic operations

Boolean and vector masking operations

Comparison operations

Conversion operations

Mask Methods

Shift and rotate operations

Zero-cost reshaping operations

Transition to/from platform-specific code

It may happen that the simd package is too limited for all parts of a particular application, or that we have not yet provided an adequate emulation for some necessary feature. For that case, the simd package supports transition to and from architecture-specific SIMD. Each vector type in the simd package has a conversion method ToArch() returning an any. That any can be type-asserted to one of the architecture-specific types for a platform. To convert back, use one of the simd.<SimdType>FromArch functions. For portable code this creates an obligation to write architecture-specific code for each of the platforms, including an emulation.

Here’s a complete example for a method/function that is currently missing, but should be added in Go 1.28. Suppose your algorithm needs Int8s.OnesCount() (which simd in Go 1.27 lacks). Rather than rewriting the entire algorithm for each platform, it’s possible to just implement the missing operation.

First, for amd64, which lacks the instruction for AVX and AVX2, but not AVX512:

The interface conversion and type switch look like they should be inefficient, but the compiler-side implementation of simd specializes code and optimizes away the type switch.

NEON and Wasm both support Int8s.OnesCount(), so their implementation is much simpler, though it still uses Int8s.ToArch and Int8sFromArch.

Don’t forget that some people don’t have hardware SIMD support:

And to complete the exercise, a separate emulation function shared as a fallback across all implementations:

API intersection and method emulation

Whatever operations the simd package offers need to run acceptably well on most architectures. As a first step, any operation that is supported everywhere, can easily be supported on simd. This tends to include loads, stores, arithmetic, and comparisons (but not all comparisons!).

A naive intersection across SIMD methods from different architectures still leaves plenty of holes. These are filled by adding emulations to the various architecture-specific archsimd APIs. These APIs already contain many trivial emulations to simplify life for Go programmers; signed and unsigned integer addition use the same instruction, but in the same way that Go supports the + operator for both int and uint, the archsimd package provides both Int8x16.Add(Int8x16) and Uint8x16.Add(Uint8x16), even though those compile to the same instruction. Modern programming languages also don’t expect programmers to know how to implement floating point negation and absolute value with bit fiddling, so archsimd implements that where necessary, or “emulates” if you look at it just so.

There are many emulations that require just 2 or 3 instructions; for example, some architectures support only a same-value shift distance across vector elements, while others support a different shift distance for each vector element. To support scalar shifting in simd, we emulate scalar shift with vector shift. Some architectures lack some unsigned comparisons–these are just signed comparison, plus two XORs with a constant.

Not all missing instructions are that simple. The “carryless multiply” instruction is important to cryptography and CRC checksumming, but it isn’t always supported. Leaving that out of the simd API would prevent its use for some important algorithms. Therefore, we provide an emulation, and because one important use is in crypto, its run time does not vary depending on its inputs.

In other cases, rather than implement a primitive instruction like “add pairs” (also called “horizontal addition”), for the simd package in the next release we will provide the higher level operation that add pairs is usually used for, which is sum reduction. This also helps insulate users from vector-length dependence; even given the hardware instruction for adding pairs, the number of reduction steps depends on the vector length.

The constraint of supporting all platforms, including ones that we predict will appear in archsimd within the next year or so, forces a somewhat conservative approach to which methods we add to simd. Riscv64, ppc64, s390x, and loong64 all have their own SIMD extensions.

GODEBUG settings

On platforms where there is some hardware support, behavior can be modified with GODEBUG, to make it easier to test simd-using code with various hardware configurations. You can set the GODEBUG environment variable prior to executing your program.

In Go 1.27, levels of SIMD support are roughly described by vector length:

• GODEBUG=simd=0 means use emulation for SIMD operations even if the hardware support is available.
• GODEBUG=simd=128 means use 128-bit vectors and their features. If the features aren’t available, panic immediately.
• GODEBUG=simd=256 means use 256-bit vectors and their features, if possible.
• GODEBUG=simd=512 means use 512-bit vectors and their features, if possible.
• GODEBUG=simd=+128 means use 128-bit vectors and their features even if some features are not supported. If unsupported instructions are used, the code will panic, but if they are not it may still run. An example of this is Raspberry Pi, which supports NEON but lacks PMULL (carryless multiply).
• GODEBUG=simd=+256 means use 256-bit vectors and their features even if some features are not supported. If unsupported instructions are used, the code will panic, but if they are not it may still run. An example of this is Apple Silicon’s amd64 emulation, which supports AVX2 but not VPCLMULQDQ (again, carryless multiply).
• GODEBUG=simd=+512 means use 512-bit vectors, even if some features are not supported.

Implementation details

If you are debugging code that uses simd, or even just look at a stack trace, you will notice some weird extra types and methods. The reason is that simd is both a package, an internal implementation package, and some AST rewriting in the front end of the compiler.

The AST rewrite creates multiple specialized copies of functions, variables, and types that mention simd types, where simd types are replaced with references to size-specialized types in simd/internal/bridge. Each of these bridge types is defined as an archsimd type, but with a restricted set of methods. The specialized functions, variables, and types acquire a suffix of the form @simdNNN, where NNN is either a vector length (128, 256, or 512) or 0, indicating emulation. Functions that mention simd internally, but not in their signature, are converted to wrappers that switch on the SIMD level detected at program start, and call the appropriate specialized version of that function. Specialized functions call other specialized functions directly without dispatch overhead (and perhaps with inlining). This rewrite strategy was chosen as a compromise between code duplication and SIMD performance; the overhead is hoisted as high as necessary to avoid dispatch within SIMD computations, but not higher. If SIMD dispatch appears “too low” in a computation, a gratuitous mention of a simd type will move it upwards, as in this example:

What’s coming

We plan to publish a blog post describing archsimd in greater detail soon.

For Go 1.28, we intend to add SVE support to archsimd, and also hope to add that to simd. More importantly, we hope to add additional SIMD operations to those that the simd package already supports (e.g., OnesCount, mask operations, reduction operations, vector shuffling operations). Go 1.28 will also include a small number of “feature variants” to avoid downgrading all the way to full emulation for platforms that have a hardware vector implementation but just lack one or a few operations, such as Raspberry Pi.

Previous article: Size-Specialized Memory Allocation Blog Index

Ranked #1 on Hacker News with 564 points and 300 comments.

What DAWO stands for

DAWO brings public values and technology together. The community pursues five primary goals.

Digital autonomy

Strengthening the digital autonomy of the Netherlands.

Collaboration

Improving collaboration and knowledge sharing between governments and society.

Security

Safeguarding security and data protection.

Innovation

Encouraging innovation and more efficient ways of working.

Verifiability

Making government IT systems easier to inspect and verify.

An open workplace is made of replaceable building blocks

Not one single product, but separate building blocks that work together. Every part can be inspected and replaced.

AI (opens in a new tab)

Open and verifiable AI building blocks for the digital workplace.

Operating system (opens in a new tab)

DAWO-NixOS and installation building blocks for a reproducible workplace.

Cloud (opens in a new tab)

Building blocks for autonomous and verifiable cloud infrastructure.

Collaboration (opens in a new tab)

Open solutions for communication, documents and working together.

See the full blueprint

Events, news and taking part

The calendar, the latest news, the blog and the forum are on the member portal. Everything is public, and that is also where you sign up.

Events

See the calendar and register for meetups and workshops.

News

Releases, developments and DAWO in the media.

Blog

Stories and reports from the community.

Forum

Ask questions and join the discussion. Anyone can read; posting requires an account.

Not for the community. With the community.

DAWO brings government, industry and open source together. Public and private build the same blueprint here. You can take part through conversations, code, documentation, pilots and events.

Ranked #2 on Hacker News with 17 points and 0 comments.

This post is the second and final part of our Avast Antivirus research, detailing the full exploitation of CVE-2025-13032 on an up-to-date Windows 11 system. Starting from the double-fetch vulnerability introduced in Part 1, we walk through how the controlled paged pool overflow was turned into an arbitrary kernel read/write primitive by corrupting the RegBuffers array of the IORing object. The post covers the heap spray strategy, the kernel address leak via MDL introspection, the repairs needed to avoid a blue screen on teardown, and the final privilege escalation to SYSTEM via token theft.

Introduction

This blogpost is the second and final part of our Avast research and will focus on the exploitation of CVE-2025-13032, a double-fetch vulnerability we discovered in Avast’s kernel driver.

This post recaps the bug and walks through how we exploited it on an up-to-date Windows 11 system at the time of the finding.

Feel free to read the first part if you missed it → https://www.safateam.com/intelligence-hub/research/technical-articles/cve-2025-13032-entering-and-breaking-the-avast-antivirus-sandbox-part-1

Note: In the latest version the windows kernel and drivers are using user-mode accessors (https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/user-mode-accessors) to verify each kernel access to user-mode memory and ensure at each access that user-buffers are in fact reside in userspace. This mitigation will prevent the use of the exploitation technique that is described in this writeup, see additional details at https://www.youtube.com/watch?v=ry4SNYe2f68

Bug Explanation

The bug we want to exploit is a double fetch issue that leads to a kernel pool overflow.

The snippet of code presented below is supposed to capture a `_UNICODE_STRING` structure supplied by the user, but the `Length` field of the user input is fetched multiple times which results in the double fetch issue.

The first fetch is done to allocate a buffer where the string will be copied and a second fetch is done to perform a memcpy based on the retrieved value resulting in a pool overflow if the user changes it between those actions.

To exploit the double fetch, a second thread runs in a tight loop, continuously toggling the `Length` field of the shared `_UNICODE_STRING` between a small safe value and a large malicious value (e.g. `0x1000`, larger than the allocated buffer). The main thread calls the vulnerable IOCTL in a loop. When the timing aligns — the kernel reads `Length` as small for the `ExAllocatePoolWithTag` call, then reads it as large for the `memmove` — more bytes are copied than were allocated, producing the pool overflow. The race window is narrow but can be won reliably within a modest number of iterations.

Our goal is to exploit this pool overflow to gain an arbitrary kernel read/write primitive and achieve a local privilege escalation. The bug gives us good exploitation conditions: the overflow targets `PAGED_POOL`, both the allocation size and the overflow size are controlled, and so is the content.

The paged pool is a region of Windows kernel memory used for objects and data that the kernel or drivers need, but that can be paged out to disk. It is used for memory that does not need to be accessed by critical code running with high priority. The allocator groups allocations by size class, meaning same-sized objects tend to land close to each other in memory — the property that makes heap spraying viable.

Since Windows 10 19H1 this is handled by the Segment Heap, which uses two backends: the LFH for small allocations, which picks free slots randomly within a size bucket, and the VS allocator for larger ones, which serves the first available chunk of the right size — each requiring a different spray strategy. We can also note that most Windows objects are stored in the paged pool, which gives us a large number of candidates when choosing what to corrupt. In the next section we explain which object we chose and the reasons behind that choice.

For more information about how windows pools work, you can refer to the `Scoop the Windows 10 pool!` paper from Synacktiv ( https://www.sstic.org/media/SSTIC2020/SSTIC-actes/pool_overflow_exploitation_since_windows_10_19h1/SSTIC2020-Article-pool_overflow_exploitation_since_windows_10_19h1-bayet_fariello.pdf ).

I/O Ring Object

The I/O Ring Object is an object that maintains a submission queue of I/O operations to be performed asynchronously.

Concretely, it lets userland batch file I/O requests: `IoRingReadFile` copies data from a file into a pre-registered buffer, and `IoRingWriteFile` copies data from a pre-registered buffer into a file. These registered buffers — tracked in the `RegBuffers` field of the `_IORING_OBJECT` — are validated once at registration time and then reused freely for every subsequent operation, making them a persistent and interesting target to corrupt.

We chose this object as our corruption target for several reasons. While the IORing object itself is located in the `NON_PAGED_POOL`, its `RegBuffers` field is allocated in `PAGED_POOL`, which directly matches the pool where the overflow occurs.

Secondly, the size of the `RegBuffers` allocation is fully user-controlled: registering N buffers produces an array of N pointers, each 8 bytes, giving us precise control over the allocation size and making it ideal for a heap spray.

Thirdly, Corrupting a single pointer in that array is sufficient to gain a full arbitrary read/write primitive — there is no need to corrupt a more complex structure.

Finally, I/O Ring Objects have already been used publicly to achieve this exact goal, which confirms the technique and provides a solid reference point for our approach.

( https://windows-internals.com/one-i-o-ring-to-rule-them-all-a-full-read-write-exploit-primitive-on-windows-11/ )

Multiple APIs are available from userland to use the object, here are some of them:

- CreateIoRing

- CloseIoRing

- BuildIoRingReadFile

- BuildIoRingWriteFile

- BuildIoRingRegisterBuffers

- BuildIoRingRegisterFileHandles

- SubmitIoRing

- ...

The `Build.*` APIs are used to construct entries that need to be submitted through the `SubmitIoRing` API.

The `IoRingRegisterBuffers` allows the user to register an array of buffers for future I/O Ring operations, which can be used as a destination buffer for the `IoRingReadFile` operation or as a source buffer for the `IoRingWriteFile`. This action creates the `RegBuffers` pointer array in the `_IORING_OBJECT` and allocates the individual `_IOP_MC_BUFFER_ENTRY` objects it points to, each holding information about the registered buffer.

Find below the `_IORING_OBJECT` and the `_IOP_MC_BUFFER_ENTRY` structure:

The diagram below shows this structure in memory: `RegBuffers` is an array of pointers, where each `RegBuffers[i]` points to a `_IOP_MC_BUFFER_ENTRY` structure holding the `Address` field that the kernel uses as the I/O target:

The `IopIoRingDispatchRegisterBuffers` function is responsible for allocating and setting up the `RegBuffers` field of our IORing Object.

When used in a normal way, a read operation using a registered buffer will read the file and copy the retrieved data into the address contained in the corresponding RegBuffers entry `RegBuffers[i].Address` without checking if it's still valid as the check is only done during registration.

Our plan is to redirect a `RegBuffers` entry to point to a fake `_IOP_MC_BUFFER_ENTRY` structure we fully control in userland. When the kernel performs an I/O operation using that entry, it will dereference our fake structure directly — reading the `Address` field from userland and using it as the r/w target. This is only possible because Windows does not implement SMAP (Supervisor Mode Access Prevention), which would otherwise prevent the kernel from dereferencing a pointer into userland memory.

With this fake entry in place, the two IORing operations become our r/w primitives:

`IoRingReadFile` reads from a file and writes into `RegBuffers[i].Address` — making it our arbitrary kernel write:

`IoRingWriteFile` reads from `RegBuffers[i].Address` and writes into a file — making it our arbitrary kernel read:

Concretely: to perform an arbitrary kernel write to address X, set the `Address` field of the fake `BufferEntry` to X and submit an `IoRingReadFile` operation — the kernel copies the read data directly into the memory at X. To read from address Y, set `Address` to Y and submit an `IoRingWriteFile` operation — the kernel reads from Y and writes the data to the output file, which we retrieve from userland. In both cases, updating the `Address` field in our userland-resident fake entry is all that is needed to redirect the operation.

Spray explanation

The size of a `RegBuffers` allocation is N × 8 bytes for N registered buffers, meaning it can fall in either the LFH or the VS backend depending on the chosen N. For this demonstration we picked a value of N that places the allocation in the LFH, which is sufficient to show the impact of the technique. Since the LFH randomises slot selection within its subsegments, precise placement is not possible — the strategy is therefore to flood the pool with a large number of `RegBuffers` allocations so that, after freeing a subset, the probability of our overflowing buffer landing adjacent to a live one is high enough to be reliable.

We choose N registered buffers such that the `RegBuffers` allocation falls in the same pool bucket as our overflowing `_UNICODE_STRING` buffer (allocated at `Length + 16` bytes). This ensures the freed `RegBuffers` holes are exactly the right size to receive our overflowing allocation, making adjacency reliable.

To reach a state where our heap overflow lands on a `RegBuffers` allocation, we use the following spray strategy. The setup is minimal: one IORing object is required per `RegBuffers` structure we want to position.

The spray itself is straightforward: we allocate a large number of `RegBuffers` structures, free a subset of them to create holes of the right size, then trigger the vulnerability to land our overflowing allocation in one of those holes and corrupt an adjacent entry.

This is how it looks in memory:

1. Allocate a large number of `RegBuffers` structures

2. Deallocate some of them

3. Allocate our unicode string

4. Trigger the corruption at the same time

From there we have a corrupted `RegBuffers` entry — the heap overflow has succeeded and our arbitrary r/w primitive is in place. The next step is obtaining a kernel address to use as the r/w target.

Abusing IORing to get a leak

At this point we have an arbitrary r/w primitive but need a kernel address to target — specifically our own `_EPROCESS` address, which we will use to steal the SYSTEM process token.

The diagram below shows the state of the structures after the corruption:

Since we corrupted the pointer inside `RegBuffers[0]` — redirecting it to a fake `_IOP_MC_BUFFER_ENTRY` that lives in our own process memory — we can modify the `Address` field of that fake entry at any time simply by writing to it from userland. There is no need to trigger the vulnerability a second time.

Our arbitrary r/w primitive is operational, but it requires a target kernel address. Since kernel addresses are randomized and cannot be predicted from userland, we need to leak one — specifically the address of our own `_EPROCESS` structure, which we will later use to manipulate our process token.

While using our registered buffer, the address will be mapped through an MDL. The associated MDL pointer is stored in our BufferEntry structure:

( https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/using-mdls )

A Memory Descriptor List (MDL) is a kernel structure that describes a range of virtual memory by locking its physical pages in place. When the kernel needs to safely operate on a userland buffer — for example, to perform I/O into it — it creates an MDL for that buffer, which pins the underlying physical pages so they cannot be paged out or remapped during the operation. Because the MDL describes a userland address, the kernel needs to track which process owns that memory, so the MDL stores a pointer to the owning process’s _EPROCESS structure in its Process field:

In our case, when the corrupted BufferEntry points to a userland address and we trigger an IORing operation, the kernel creates and attaches an MDL to our BufferEntry to map that address. Since the BufferEntry itself now lives in userland (as a result of our corruption), we can simply read its Mdl field directly from our process. We then use our arbitrary read primitive to dereference that MDL pointer and extract the Process field — giving us a valid _EPROCESS pointer for our process, which is all we need to proceed with the privilege escalation.

The leak proceeds in four steps:

(1) `RegBuffers[0]` now points to our fake `_IOP_MC_BUFFER_ENTRY` at a known userland address.

(2) We trigger an IORing operation — the kernel creates an MDL for our userland buffer and writes its pointer into our fake entry’s `Mdl` field.

(3) Since the fake entry is in our own process memory, we read the `Mdl` pointer directly from userland without any kernel primitive.

(4) We set the `Address` field of our fake entry to that MDL address, trigger another operation, and read the `Process` field from the MDL — giving us a valid `_EPROCESS` pointer.

Issues

At this point we have both an arbitrary read/write primitive and a kernel address leak. Before proceeding to the privilege escalation, however, we need to repair the corrupted state — releasing the IORing object without cleanup will crash the system.

ProcessBilled

During our overflow, we corrupted an important field in the pool chunk header: the `ProcessBilled` field, which stores a pointer to the process responsible for the allocation. If left uncorrected, this will trigger a blue screen when the chunk is freed.

The ProcessBilled value is an obfuscated pointer to an `EPROCESS`, this value is computed as follows:

`@EPROCESS ^ ChunkAddress ^ ExpPoolQuotaCookie`

`ChunkAddress` is the address of the corrupted pool chunk header, located at a known negative offset before the `RegBuffers` pointer we already know.

`ExPoolQuotaCookie` is a global kernel value used to obfuscate pool billing pointers; to derive it, we use a second uncorrupted IORing object.

Via our arbitrary read, we read its `RegBuffers` address from the `_IORING_OBJECT` and the `ProcessBilled` value from the preceding pool chunk header. Since we know our `_EPROCESS` address, we reverse the formula: `Cookie = EPROCESS ^ ChunkAddress_clean ^ ProcessBilled_clean`.

We then compute the correct `ProcessBilled` for the corrupted chunk and write it back using our arbitrary write primitive.

With the formula in hand, the remaining step is locating the corrupted IORing object itself in memory so we can apply the fix.

We locate the IORing object by parsing our process handle table, found in the `_EPROCESS` structure, following the same logic as `ExpLookupHandleTableEntry` to retrieve the handle table entry, then using the same formula as in `ExGetHandlePointer` to transform it into an object pointer.

Buffer Entry reference

When we obtain our leak, a reference to our buffer entry located in userland is stored in the kernel which will lead to a crash when the kernel attempts to process it during teardown.

The fix is to release the `RegBuffers` registration. This causes the kernel to clean up the associated MDL as part of teardown, resolving the lingering reference. Releasing the MDL directly would not be sufficient — the MDL release is a consequence of releasing the `RegBuffers` entry, not a standalone action. However, releasing the registration triggers another issue, covered below.

Free user buffer

As we have corrupted a buffer entry, the kernel will try to release our userland pointer when closing the object.

The solution to this issue is to simply increase the reference count of our fake buffer entry.

This prevents the kernel reference count from ever reaching zero during IORing teardown, so the corresponding release function is never invoked on our userland pointer.

Abuse Arbitrary R/W to get more privilege

To escalate our privileges, we steal the SYSTEM process token. Using our arbitrary read primitive, we walk the `EPROCESS` doubly-linked list to locate the SYSTEM process entry and read its `Token` field. We then use our arbitrary write primitive to overwrite our own `EPROCESS` `Token` field with the SYSTEM token value, granting our process SYSTEM-level privileges.

Conclusion

In this post we demonstrated a full local privilege escalation exploit against an up-to-date Windows 11 system, leveraging CVE-2025-13032 — a double-fetch vulnerability in Avast’s kernel driver. Starting from a controlled heap pool overflow in PAGED_POOL, we used the RegBuffers array of the IORing object as our corruption target, turning a single overwritten pointer into an arbitrary kernel read/write primitive. From there, we leaked an _EPROCESS pointer via the MDL attached to our corrupted BufferEntry, repaired the pool allocation header to avoid a crash on teardown, and completed the privilege escalation by stealing the SYSTEM process token.

CVE-2025-13032 has since been patched. We encourage all users to ensure their Avast installation is up to date. The full disclosure timeline is detailed in Part 1 of this research.

If you missed the first part of this research, which covers the vulnerability discovery and sandbox escape, you can find it here: CVE-2025-13032 — Entering and Breaking the Avast Antivirus Sandbox (Part 1).

Related posts

More content you might like

CVE-2025-13032: Entering and Breaking the Avast Antivirus Sandbox Part 1

SAFA discovered four distinct kernel heap overflow vulnerabilities in Avast Antivirus. Our research targeted the aswSnx kernel driver, first requiring interesting sandbox manipulation to reach the attack surface. CVE-2025-13032 was assigned to these patched vulnerabilities. This first blog post introduces the vulnerabilities and the challenges of the custom sandbox profile. While a consecutive post will detail how the primitive was exploited for Local Privilege Escalation to System.

Ranked #3 on Hacker News with 206 points and 82 comments.

Ranked #2 on Hacker News with 90 points and 22 comments.

Ranked #2 on Hacker News with 223 points and 37 comments.

A foundry for bastard web fonts. Mix, stretch and / or squish them.

Every download is a normal OpenType font. The swap is a liga contextual substitution registered for every script, so browsers turn it on by default.

It works anywhere OpenType text is shaped: browsers, design tools, print. If a font looks unchanged, check that ligatures aren't switched off in the app you're using.

No. Everything runs locally in your browser with Pyodide and fontTools.

Bastardica can make simple fonts feel a little, hmm, richer? Use Y-offset and scale effects to make glyphs align perfectly.

When mixing 3 or more fonts, they will intersect (e.g. every 5th and every 7th will collide on every 35th). The first font wins. The stride won't break for either.

Use prime numbers for strides, so the mix-in fonts collide more rarely.

Some websites to grab free fonts to play with: Google Fonts, UNCUT, Velvetyne, Font Squirrel, FontSpace, DaFont.

Mixing two fonts produces a derivative work, so make sure to check licenses of both source fonts if you plan to use a bastard font commercially. Bastardica adds no conditions of its own. A credit is appreciated, but optional.

Bastardica was inspired by Times New Bastard and Easy Pete.

You can ask me about anything at [email protected]

Ranked #1 on Hacker News with 187 points and 50 comments.

After more than a year of hard work, we are thrilled to announce the launch of F-Droid 2.0, a complete redesign of the official F-Droid app and the largest app update in 10 years.

For more than a decade, F-Droid has helped people discover and install free and open source Android apps. F-Droid 2.0 builds on that foundation with a modern interface, better app discovery, improved search, and a simpler experience that works well, whether you’re new to F-Droid or have been using it for years.

This isn’t just a visual refresh. The user experience was redesigned to integrate smoothly with current Android patterns, like Material Design, while keeping familiar F-Droid interactions in place. Key components were reworked and rewritten using Kotlin Compose, the standard toolkit these days, creating a foundation that will help us deliver improvements more quickly in the years ahead.

We are excited to begin rolling out F-Droid 2.0 to users over the coming weeks after 14 test releases.

What has changed?

One of our main goals for F-Droid 2.0 was to make it easier to discover, install, and maintain the apps you rely on. We simplified the main navigation into three core areas: Discover, Search and My Apps. Categories are now integrated into Discover, making it easier to browse and explore, while My Apps provides a central place to manage installed apps, updates, and potential issues. Settings and Nearby Swap are still only a tap away from the top bar, but no longer compete for space in the main navigation.

Discoverability improvements

Helping people discover relevant free and open source software (FOSS) was one of the primary goals of F-Droid 2.0. As the F-Droid ecosystem has grown to thousands of applications, finding the right app has become increasingly challenging. The new release introduces improvements throughout the app from browsing and categories to search to make it easier to find software that matches your needs. And of course, F-Droid does this without tracking you, or trying to “engage” you to spend increasingly more time in the app.

A redesigned Discover experience

The new Discover screen helps uncover apps you might otherwise miss. In addition to highlighting newly added and recently updated apps, it now showcases the most downloaded apps in the repository. Whether you’re new to F-Droid or looking for something different, Discover provides several ways to explore the growing ecosystem of free and open source Android applications.

More useful categories

Categories play an important role in helping users browse the repository, so we’ve expanded and refined them significantly, including more specialized categories that make it easier to find specific types of apps, such as VPNs, firewalls, password managers, launches and navigation tools. Here is what you can expect:

• First, we’ve significantly expanded the category system. Instead of relying on a small number of broad categories, F-Droid now includes many more specialized categories, helping you get closer to the kind of app you want in just a few taps, even before you start searching.
• Second, to make this expanded category system easier to navigate, we’ve introduced higher-level “meta” categories in the Discover screen. These group related categories together and provide a more approachable entry point for browsing the growing F-Droid ecosystem.
• Finally, categories now play a larger role throughout the app. Their names and descriptions are used to improve app discovery and help guide users toward relevant free and open source applications.

As an example of this effort, we’ve completely reworked the Games category. Rather than grouping all games together, F-Droid 2.0 now distinguishes between 17 different game genres, making it much easier to find the kinds of games you actually enjoy playing.

Search that understands what you’re looking for

Search has also been significantly improved. In addition to app names, it can now search app descriptions, categories, and translated content. This makes it easier to find apps based on what they do rather than what they’re called.

We’ve also made major improvements for users searching in Chinese, Japanese, and Korean. The new search system provides much better support for CJK writing systems, helping users find relevant apps more reliably in their own language.

Search also remembers your recent queries, allowing you to quickly return to previous searches without having to type them again.

Powerful filtering, made approachable

Browsing and searching are only part of the story. F-Droid 2.0 also introduces powerful filtering options that help you narrow down large lists of apps to exactly what you’re looking for.

Filters can be combined using multiple criteria, such as app category, device compatibility, or anti-features. For example, you can choose to view only Action Games that are compatible with your device and exclude apps that depend on non-free network services.

To help users discover these and other advanced capabilities, F-Droid 2.0 introduces onboarding screens throughout the app. Rather than hiding features behind complex settings, the app provides contextual guidance to help both new and experienced users get the most out of F-Droid.

Smooth installation experience wherever F-Droid runs

For the longest time, the experience of clicking install or update was forced to be second rate by Android. Now, thanks largely to pressure from the EU’s Digital Markets Act (DMA) and anti-trust actions around the world, Android offers all app stores an option for a smoother and more automatic install and update experience than before. F-Droid 2.0 includes groundbreaking work on utilizing these new abilities. This allows F-Droid to use a unified installer for all F-Droid installs, whether built into the OS or you installed it on your device yourself. The unified installer makes use of the new pre-approval API, so that on supported devices the user can confirm right after deciding to install the app, instead of after the app was downloaded. That brings the F-Droid install experience on official Android devices much closer to what the built-in app store can provide.

What moved, and what was removed

A redesign of this size means making careful decisions about what belongs in the new app, what can be handled differently, and what no longer makes sense to carry forward. Some familiar features have changed, moved, or been removed as part of making F-Droid 2.0 more streamlined and easier to maintain, without sacrificing core functionality or features users rely on.

Update checks now happen automatically

F-Droid 2.0 now fetches and installs app updates by default. If you prefer more control, no worries, your existing preferences are still respected.

Some users missed the pull-to-refresh gesture for checking all repositories for updates. In F-Droid 2.0, the pulling gesture is exclusively for scrolling. This is now possible because the app can now automatically check for updates in the background. Rather than preserving a familiar action, we focused on removing the need for it. The best refresh button is the one you never have to press.

Users who want more control still have fine-grained and manual update options available. If you used pull-to-refresh to manually trigger updates, that is now available under the action overflow menu, e.g. the “three dots”, on the My Apps screen.

Data usage settings

F-Droid gives you control of what get’s downloaded when. This helps fit our diverse users around the world, who have varying requirements. Many users have cheap access to mobile data, while mobile data is prohibitively expensive for others. Some users have heightened privacy requirements, so they need to control their network traffic. While others are using limited devices which bog down when F-Droid updates in the background. The settings which control all this were reworked to make adapting F-Droid to your needs more intuitive.

Privacy and security features

Some F-Droid users operate in environments where simply having certain apps installed or even using F-Droid can attract unwanted attention. To help support these users, F-Droid has long included a set of privacy and security features designed to protect both the user and their data.

One key privacy tool is Tor, and F-Droid has long supported using Tor for all network connections, and using Tor Onion Services for repositories and mirrors. The landscape of how Tor is integrated into Android has changed quite a bit since Tor support was first integrated. Now there is TorVPN, Orbot, TorServices and more. We took this opportunity to simplify the settings and remove the auto-detection that was no longer reliable. If you enabled “Use Tor”, that will be migrated to generic Proxy Settings. Going forward, Tor VPN is the recommended approach for easy Tor support, and the Proxy Settings are still available for those who need manual control.

Another key part is the set of “panic” features, which allow users to quickly remove some specific kinds of sensitive information from their device in emergency situations. These features are still included in F-Droid 2.0 and remain an important part of supporting users with elevated security needs.

Notably, the F-Droid app hiding feature has been simplified, to give users an accurate idea of the kind of protection they can expect. F-Droid was one of the first apps that began providing app hiding features to protect user privacy, including our “panic” feature which disguised the F-Droid app as a simple calculator app. This simple feature was requested by many users, and since then Orbot, TorVPN, Signal and others have added such masking features. Over time, a standard design has emerged across widely used apps, and we have adopted this design in the new release as well. This feature is designed so that users can better understand the limits of the disguise. Instead of the mask looking and functioning as a simple calculator app, now the mask only affects the app icon, name and nothing else. This informs users that the F-Droid app will still appear in the “Apps” settings and would be detectable during forensic inspection. This change will hopefully make it easier for users to understand the limits of this feature, while still utilizing it when needed.

One feature that has not yet returned is the ability to remove and wipe apps as a response to a panic trigger app like Ripple. We recognize that some users rely on this functionality for privacy and personal safety reasons and understand it is more than a usability feature. However, it requires highly specialized work to maintain, and given the small user base, we felt it should no longer block so many other important improvements.

The app-wiping feature remains an important feature. We would especially welcome feedback from people who use it, to help us understand how and when it is used, so we can evaluate the best path forward as we continue improving F-Droid 2.0. Users who rely on the current app-wiping implementation may opt to postpone updating to F-Droid 2.0 while we evaluate bringing the feature back.

For Android versions that integrate F-Droid

F-Droid is designed to be integrated into any version of Android or AOSP, as we can see in CalyxOS, emteriaOS, iodéOS, Lineage-for-microG and ShiftOS. Each OS can include their own repositories by default using the “additional repos” mechanism. If you use one of these OSes, these will be visible in your Repositories overview. For additional info on what changed, check out this blog post.

Also, F-Droid Privileged Extension (FPE) is not currently supported by 2.0. That means even if FPE is installed, F-Droid 2.0 won’t use it. This overhaul focused on full featured support for the Android “session” installer. That lets F-Droid run background updates on any recent Android version without requiring FPE. Like with any of the changes here, we welcome feedback.

Lowering the barrier for contributors

While many of the improvements in F-Droid 2.0 are visible on the surface, some of the most important changes happened behind the scenes.

All new code in this effort uses modern Android code standards and designs. This gives us a codebase that is easier to maintain, test, and easier to extend with new features in the future.

One of the goals of the rewrite was to lower the barrier for new contributors. Android development has changed significantly over the past decade, and F-Droid 2.0 is now built using Kotlin, the language that has become the standard for modern Android development. This makes it easier for developers familiar with today’s Android ecosystem to contribute to the project.

The new user interface is built with Jetpack Compose, the standard toolkit for Android applications. Beyond simplifying development, this helped us align F-Droid more closely with Material Design, the design system used throughout Android. As a result, F-Droid feels more familiar to Android users while remaining true to its own identity and values.

Most importantly, these changes provide a foundation for the next decade of F-Droid development. By reducing maintenance burden and making contributions easier, we can spend more time improving the experience for users and less time fighting technical debt. Some new tools also depend on fixes in Android itself, one such fix was added in Android 7 forcing us to drop support for Android 6. As always, old F-Droid releases will continue to work on old Android versions.

A community effort

F-Droid 2.0 is one of the largest and most ambitious projects in our history. Bringing it to life required much more than software development. It involved user research, design, testing, documentation, community feedback, quality assurance, lots of new code, a security audit and countless discussions about how F-Droid should evolve over the next decade.

This work was made possible by the support of many organizations and individuals. Torsten Grote’s development work on the new app was funded by NLnet through the Mobifree fund. The Open Technology Fund’s User Experience & Discovery Lab supported user research and design work, bringing in Ura Design to help conduct user testing, develop user stories, and refine our Human Interface Guidelines.

Additional support came from the Open Technology Fund’s Free and Open Source Software (FOSS) Sustainability Fund, NGI, Mobifree, and the Calyx Institute, whose sponsorship helps support the ongoing maintenance and long-term sustainability of the F-Droid ecosystem.

As part of this effort, the Open Technology Fund’s Security Lab in conjunction with Convocation conducted an independent security review of F-Droid 2.0. We analysed and addressed all findings relevant to the new application, helping ensure that the release meets the high security standards our users expect. We look forward to sharing the full audit report once it has been cleared for publication.

Just as importantly, F-Droid 2.0 reflects the contributions of many volunteers. Community members contributed code, testing, bug reports, design feedback, translations, documentation, UX discussions, and countless ideas throughout the redesign process. Both long-time contributors and people making their first contribution helped shape the final result.

Finally, this work would not have been possible without your support. Donations help fund many of the less visible but essential activities that grants don’t always cover, including community management, handling the issue backlog, quality assurance, release management, and project coordination. These contributions help keep F-Droid healthy long after a specific grant-funded project has ended.

The journey continues

F-Droid 2.0 represents a major milestone, but it is not the end of the story. Rebuilding the app has given us a stronger foundation, yet there is still plenty of work ahead.

As the rollout reaches more users, we expect to learn a great deal from real-world usage. Community feedback has shaped F-Droid 2.0 from its earliest design discussions through many alpha and RC releases, and it will continue to guide future improvements. Some ideas did not make it into the initial release, while other features are still evolving as we gather feedback and refine their design.

In the coming months, we will continue improving performance, accessibility, app discovery, and overall usability. We’ll also keep listening to users as they adapt to the new experience and help us identify opportunities for further improvement.

Like every major F-Droid release before it, version 2.0 is not a destination, it’s the beginning of the next chapter.

The future of Nearby

One area that continues to evolve is Nearby, the feature that allows users to share apps directly between devices without relying on a central server.

The broader F-Droid 2.0 redesign gave us an opportunity to rethink Nearby from the ground up. We have been working on a new implementation based on improved connection methods that should make sharing apps more reliable and easier to use.

This work is not quite ready for inclusion in the initial F-Droid 2.0 release, but development is actively underway and the foundations are already in place. If Nearby sharing is important to you, now is an excellent time to get involved. Community feedback and testing can help shape the next generation of the feature before it reaches a wider audience.

How you can help

F-Droid 2.0 is the result of thousands of hours of work from developers, designers, testers, translators, donors, and community members around the world. Now that it is reaching users, we’d love your help making it even better.

If you’re receiving the update, take some time to explore the new experience and let us know what you think. Whether you’ve found a bug, have an idea for an improvement, or simply want to tell us what works well, your feedback helps guide future development.

If you’d like to get more involved, there are many ways to contribute.

Help us test, translate and review

You can help test upcoming features, improve translations and documentation, review issues, contribute code, or join discussions about the future of the project. New contributors are always welcome.

Consider donating to F-Droid

And if you’re able, please consider supporting F-Droid financially. Donations through Liberapay or OpenCollective help fund the ongoing work that keeps the project healthy between major releases, from infrastructure and quality assurance to community support and project coordination.

F-Droid 2.0 is a major milestone, but the work continues. Thank you for helping us build a free, open, and sustainable app ecosystem for Android.

Ranked #1 on Hacker News with 90 points and 46 comments.