There is no such thing as AI mastery

I was just slacking with another agentic athlete about the releases of OpenAI's latest model Astra. Even though Astra is just 2 days old, I already had the expectation that my fellow AA would have tried and battle tested this newly minted model.

The answer? “Not yet, I’m still behind and still trying to master Grokbot.”

While my memory is a bit fuzzy, Grok Bots are also only about 3 weeks old and in themselves were a whole new surface of experimentation and continued excitement.

During the time, Fable 5.1 also came around and I am sure a host of other open source capabilities (Ox?) that we haven’t even scratched the surface of.

The pace of innovation is relentless.

Yet the reality is despite all these dizzying and progressive improvements, the way that I at least interact with these tools has become more fluid.

This fluidity doesn’t come with a dogmatic approach to building the tidy library of skills and sticking with one specific set of workflows, rather it comes with making sure that when something new comes out I already have all the primordial frictions out of the way.

Namely, the integrations (across multiple harnesses) the ways to access these tools (across desktop/mobile apps or CLIs), and most important some established applications and workflows that I am already overly familiar with to know what good looks like.

The last part I would call a non-technical eval. Evals are often used by the most advanced teams to assess model improvements and benchmark against peers against a host of technical dimensions.

For operators, the eval should rest on whether these advancements make life easier to accomplish the same known tasks, and open doors to moving creatively thinking on solving the next bottlenecks.

The hardest road to adoption is when it's unclear what the objective should be and what datasets, application, and human judgement should be called on.

‘Templates’ and pulling down known skills can help, but they can also hurt your ability to solve problems.

Outsourcing multi-step thinking can lead to the forbidden AI slop (or at the very least makes work and your role in delivering this work, less propreitary) and is where agentic athletes have a serious leg up.

I’ve also started to think less about token management, the level of model effort, and keeping my GitHub as tidy as possible. Not because these aren’t important, but rather they’ve become second nature and when there is a more serious technical challenge or I need to make something hardened for external purposes, I call on engineers who actually know what they are doing.

Just like any language, the minute you stop thinking about how to speak and instead immerse yourself in the everyday life of using and improving on these tools, the more fluid and conversational you’ll start to become.