The Internet: “[ ] I’m not a robot”... Ask Juliana: “Damned !”

Ask Juliana: how we built a frugal, human-controlled AI assistant

By Damien Cappelle, Software Engineer at Indao

The story of Ask Juliana, the AI assistant built into the Indao platform, began at roughly the same time I joined Indao. To be precise, it actually began during my job interview, with a question about my views on LLMs (Large Language Models, the type of model behind services like Claude or ChatGPT). My honest answer (a mix of fascination, fears and big ethical questions) apparently didn't go down badly, since I now find myself in charge of integrating LLMs into our products.

So what does Ask Juliana do?

Ask Juliana is an assistance tool based on an LLM. It can be any LLM: we want to remain independent of any single provider and let our customers choose according to their own realities. Its main purposes are to answer users' questions, whether about our product or their industrial data, and to automate the configuration process.

For example, imagine we have a great model trained in Indao that requires 50 inputs, and the names of the real-time data don't necessarily match the names specified in the model. Ask Juliana can take the model, browse the tree of data coming from the industrial process and configure the model's inference. In a case like this, we estimate that up to an hour of manual configuration can be saved.

Of course, its capabilities go much further, for example creating dashboards from a photo of a flipchart, a feature currently in testing. Other features are under development, such as model selection and training, and diagnostics.

Ethical questions

As I mentioned in the introduction, the development of generative AI, and of LLMs in particular, raises a lot of ethical questions for me: water and electricity consumption, raw material mining, sovereignty, loss of skills... When I was asked to work on this agent, I wanted to find ways to address these questions. Here are the practices we've put in place to reduce its impact.

Not tying ourselves to a single provider

By choosing compatibility with multiple providers, we give our customers the option of picking a solution that fits their needs and constraints (cost, sovereignty, open-source or proprietary, etc.). This means additional maintenance work, but I truly believe it matters.

Optimizing resource use

I've worked a lot on this: loading only the relevant data into the context, optimizing the way the tools available to the agent are documented, orchestrating sub-agents, and so on. Today, the vast majority of tasks run on lightweight models. Creating dashboards from a prompt or a photo is entirely possible with Claude Haiku 4.5, for example. Some tasks even run on very lightweight models: a lot can already be done with Mistral's Ministral 3 14B, an open-source model with 14 billion parameters.

This brings a number of benefits:

• Lower usage costs
• A smaller environmental footprint
• A wider choice of providers and models
• Faster execution

It also opens up a possibility: running models on local machines, an option we haven't explored yet.

The operator stays in control

In our architecture, it is guaranteed programmatically that any write operation (a configuration, for instance) is carried out only with the approval of the person using the tool. We have interfaces for viewing and validating tool calls, which give complete control over the result. This keeps humans at the heart of the decision-making process. The goal is to save you time, not to take away your critical thinking.

Used sparingly and for concrete purposes, such as optimizing industrial processes, an LLM becomes a tool whose impact is justified.

The technical challenge

Before wrapping up, I'd like to talk about the difficulties of working on this kind of subject. As a software engineer, I'm used to translating feature requests into technical requirements, into code and, above all, into deterministic tests that prove a given requirement or feature meets expectations. Working with an LLM requires rethinking the way you approach features.

First, the list of features is open-ended. Just a week ago, I was discussing with a colleague the possibility of identifying faulty sensors with Juliana, and he told me: "but that already works." And sure enough, using a statistics tool I had implemented myself, the agent already knew how to draw conclusions. I just hadn't thought of it! The list of tools available to the agent is finite, but the ways they can be orchestrated leave an enormous range of possibilities, all the more so when the LLM is highly capable.

Second, nothing is deterministic (especially in summer 🌡️🤭). A user might use slightly different wording in their prompt ("folder" instead of "directory") and, boom, everything goes off the rails! Not to mention switching from one model to another. We can no longer validate features the way we used to, and ensuring non-regression when making changes becomes all the harder.

To tackle these difficulties, we've set up a benchmarking tool. It runs a series of context-specific prompts and checks both tool usage and consumption, measured in tokens and cost, for a given model. Our team also carries out a human review.

Conclusion

Despite these technical difficulties, we now have a working LLM agent that can genuinely help users in the field. Ideas keep sprouting, and the stack of features to implement grows by the day.

And who knows, maybe my work will make yours easier tomorrow?

Want to see Ask Juliana at work on your data? Request a demo.