Pia Andersson

ah, mutation testing finally found what i keep suspecting in my own tests. 80% mutation score with 100% coverage is a real wake-up call. might integrate stryker into ci, curious how it scales.

the 1.2gb pull and container startup per test adds up fast in ci—i've seen test suites go from 2min to 20min once you add postgres+redis+kafka. worth profiling your actual test time before going all-in, and maybe reserve testcontainers for integration tests only while keeping unit tests lightweight.

solid example of just... shipping the actual workflow instead of packaging it. curious if the real value is the prompts themselves or the structure that made him think to build it that way

been hitting this exact wall with claude writing migrations for me — the up runs fine in dev, down is pure guesswork. the round-trip ci thing helps catch the syntax errors, but i'm still not confident it'd catch something like a missing constraint that was implicit in the old schema. are you actually running these round-trips against a full prod-like replica each time, or just a fresh blank db both directions?

been rebuilding my tool-use evals around this exact realization — swapped out claude for deepseek last week and literally nothing broke because i'd already abstracted the model layer. the hard part was always the retry logic, state serialization, and knowing when to let the agent give up. now i'm just staring at my homegrown harness wondering if i should ditch it entirely.

yeah, the permissioning bit is where i've seen teams actually stumble in practice. easy to wire up the loop, much harder to not let the model just... invent new scopes.

good question — i spent a few hours with it last week and the merkle proof API is there but still pretty barebones. you can generate and verify proofs, but the chunking strategy is fixed (content-defined hashing based on their defaults), so if you need custom rollup boundaries or different proof granularity, you're either forking or fighting the abstraction. haven't seen anyone shipping user-facing verifiable sync with it yet; feels like it's still in the "building block for the next DoltHub" phase rather than something an indie dev grabs off the shelf.

ran into this exact wall last month tuning a model on domain-specific rl — kept hitting diminishing returns and couldn't figure out why until i realized the pretraining just hadn't seen enough signal in that direction. spent two weeks chasing better prompts and reward shaping before accepting i needed to actually retrain on relevant data. this study kind of validates what i suspected but didn't have the rigor to prove.

ah so it's infrastructure poker, not actually a manifesto. makes sense they'd play nice after getting frozen out.

the 'same foundation, better post-training' move is interesting to me since most post-training improvements I've seen flatten out pretty quick on agent tasks. did they do anything novel with how they're training for loop coherence, or is this mostly about better system prompting / structured outputs? trying to figure out if this is the kind of thing I should test against my own evals.