In this keynote, Faktion's CEO, Bart Baeyens, takes the stage at the Flanders AI Forum to share hard-earned insights from the frontier of Generative AI.
Bart unpacks The Good, The Bad, and The Ugly of building agentic AI systems from early exploration to user adoption. Drawing from real-world cases in mental healthcare, knowledge management assistant for an ERP provider, and enterprise AI, this talk is a reality check and a roadmap for any organisation serious about turning GenAI into a strategic asset.
Let’s dive into our multi-agent AI framework and evaluation driven development.
Contact us for an inspiration session, to co-create your AI strategy, or to start your short list of use cases and accelerate your AI journey.
We're witnessing the emergence of multi-agent systems that can execute complex tasks autonomously and collaborate across workflows.
The potential is genuinely phenomenal, these systems are already displaying what appears to be human expert-level intelligence, opening doors to capabilities that seemed impossible just two years ago.
We can build multi-agent systems in which specialised agents can perform domain expert tasks, reason and collaborate autonomously across the workflows within complex processes.
Don’t stop at efficiency! Multi-agent systems can now free up domain experts from repetitive tasks to engage in strategic work. The experts can expand the knowledge base, deepen expertise, develop new offerings, and drive revenue.
Domain experts aren’t being replaced. They’re stepping into a new role: designing, validating, and shaping the systems that reflect their expertise.
Agent-powered chat platforms are already streamlining intake and triage, especially in human-centred domains like healthcare without losing the nuance and empathy real conversations demand.
A multi-agent system built with Faktion’s evaluation driven development process is now empowering Ferranti’s consultants to work with higher reliability and less friction, already delivering 15% efficiency gains for 100+ users.
While the possibilities are exciting, most organisations are still stuck in the same place: proofs of concept that never make it to production.
We’re seeing recurring patterns that hold companies back from realising GenAI’s promise.
The excitement is real but so is the chaos. Teams rush into GenAI without alignment, strategy, or a plan for reliability. It’s easy to build something that demos well. Much harder to make it work in the real world.
Waiting for a feature, buying SaaS point solutions just like my competitors, DIY low/no-code, building internal capabilities or partnering with a development provider? But beware for cowboys!
Flashy PoCs, quick to setup but often reliable in production and the user adoption is often a mirage. Why? No user involvement. No validation loops. No bridge between technical feasibility and actual adoption.
It takes various skills and deep understanding of the knowledge base, business context and process to scale up the PoC to production.
Domain expertise is needed for development and evaluation, but domain experts lack AI tools and developers lack domain knowledge. Large companies have greater challenges in aligning development and domain expertise precisely because of their complexity.
Getting to production-ready agentic AI systems require strong collaboration between the developers and domain experts within evaluation driven development. This is the part most companies underestimate.
Just like training new team members, onboarding domain experts, setting up evaluation frameworks, and tuning agents takes time and iteration. But with the right foundations—hybrid evaluation, feedback loops, and aligned metrics—systems improve steadily and become trusted parts of daily workflows.
Developers, domain experts, and agents must work together from day one: during knowledge base analysis, data ingestion, prototype testing, evaluation & feedback loops.
Combine automated checks (LLM-as-a-Judge), expert reviews with clear criteria, and real-world A/B tests to see what actually works in practice. It’s the only way to catch blind spots and make meaningful improvements over time.
instead of one-off feedback sessions and manual reviews, you need to build systems that evolve, you need software-driven workflows that embed evaluation into the real work: tracing outcomes, surfacing friction, and closing the loop continuously.
LLM-as-a-Judge starts by learning from expert feedback but over time, it takes over more of the validation itself. The result: fewer human bottlenecks, more autonomy, and a system that improves as it’s used.
Set up multi-agent orchestration that aligns with business logic and breaks complex workflows into manageable, specialized agents.
Design evaluation-first development loops that go beyond gut feeling, embedding feedback, certainty scores, and real-world test cases from day one.
Build knowledge management assistants that don’t replace your experts, but augment them.
Navigate organisational complexity with agentic AI that fits your workflows, governance, and tools — not the other way around.
Explore our cutting-edge AI solutions and success stories.
.png)
