Evals, reducing hallucinations, & AI-native development
In this episode, Deejay sits down with Amy Heineike, founding AI engineer at TESSL, to explore the structural shift toward AI-native development. They discuss the necessity of machine-optimized documentation registries to eliminate agent hallucinations and the cultural transition from deterministic logic to a biological science mindset. Amy details the mechanics of building evaluation harnesses, the pitfalls of contradictory steering, and how the role of the software engineer is evolving into a high-level architect of intentional outcomes and anti-fragile systems.
Episode Transcript
Daniel Jones (00:00) Amy Heineike, founding AI engineer at TESSL. ⁓ It's great to have you with us. What are you and the folks at TESSL doing at the moment? Amy (00:07) ⁓ hi, Daniel. great to be here. Yeah. So we are building tools to help people who are using coding agents every day. So we've rele...
Episode Highlights
- •TESL builds documentation registries to ground coding agents and stop API hallucinations in enterprise environments.
- •Moving to AI-native development requires shifting from deterministic logic to biological science and probabilistic experimentation.
- •Evaluations measure agent success across baskets of scenarios rather than traditional binary pass-fail unit tests.
- •Hyper-detailed task prompts paradoxically trigger models to ignore broader system instructions and core steering rules.
- •The software engineering role is evolving into a Product Engineer focused on high-level intentional outcomes.
- •System non-determinism acts as a feature enabling anti-fragility and escapes from logical local maxima.
- •Multi-pass agentic loops manage distinct concerns like security and performance more effectively than single prompts.
