Building a Telemetry Data-Centric AI Assistant

Introduction

In today's AI-driven world, businesses rely on large language models (LLMs) for insights. But there's a fundamental issue—many AI solutions extract knowledge from the web or the model trained data instead of using an organization's own specific data. When it comes to real-time decision-making, we need a better approach: direct integration with stored telemetry data. This ensures accuracy and consistency without depending on external sources.

Our solution? An AI-driven AI Assistant that directly connects to internal data sources while maintaining strict separation from public knowledge bases. In this blog, we'll explore the core architecture, key components, and why this approach is critical for organizations that demand precise, real-time insights from their own data.

The Challenge: Data-Driven Insights Without External Dependencies

The key requirement for an effective AI-powered data system is ensuring insights come exclusively from internal data, not from scraping or pre-trained knowledge. This presents several challenges:

• Direct integration with stored telemetry data – The AI must retrieve data in real-time from internal systems.
• Strict separation from public knowledge bases – The AI Assistant should not rely on web-scraped information.
• Language-agnostic query processing – The system should support multiple languages for broader accessibility.
• Multi-language support – The AI should respond in various languages while maintaining accuracy.

Architecture: AI as a Data Interpreter

Our AI Assistant follows a structured architecture designed to process queries efficiently and provide real-time insights.

Core Components

1. Data Gateway Layer

• Establishes real-time connections to telemetry databases.
• Validates queries against known metrics to ensure accuracy.

2. AI Processing Layer

• Designed with an LLM-agnostic pattern, ensuring adaptability.
• Supports multiple model providers, enabling flexibility.
• Implements fallback strategies when specific data is unavailable.

Why This Matters

• 100% of numerical results come from internal data – No reliance on pre-trained AI knowledge.
• Consistent responses across LLM providers – Ensures reliability even when switching models.
• Future-proof model switching – Organizations can upgrade AI models without compromising functionality.

Data Flow

The flow to process the queries and retrieve real-time data:

1. User Prompt → First the user submits a query, asking for insights from internal data or asking questions on relevant topics.

2. AI Agents → The agent interprets the query, determines if it needs translation to English, and assesses whether it requests data. If data is needed, it decides the best approach to retrieve relevant information. Otherwise, it relies on the LLM’s built-in knowledge to provide an appropriate response.

3. Response Formatting → The AI Agent receives the raw data, and then processes it and structures it into meaningful response, and then returns it to the user in the original prompt language.

Roles of Each Agent in the Chain

1. Translation Agent

• Detects the language of the input query.
• If the query is not in English, translates it to English.
• Ensures all downstream processes work with a common language.

2. Metric Detector

• Identifies if the query involves measurements, or specific metrics.
• Helps determine whether the query requires data fetching or can be answered using general knowledge.

3. Query Constructor

• Analyzes the intent of the query and structures it for efficient data retrieval.

4. Query Formatter

• Fetches & execute the query from the datasource
• Formats the constructed query into a structured form
• Ensures the query is in the correct syntax for execution.

5. Prepare for Translation

• Prepares the query for possible translation back to the user’s original language.
• Ensures clarity and maintains contextual meaning.

6. Translation Agent (Second Pass)

• If the response needs to be translated back to the user’s original language, this agent handles it.
• Ensures seamless interaction in multilingual contexts.

System Message Role

The system message provides initial instructions to guide the AI agent’s behavior. It ensures that the agent:

• Detects language and translates if necessary.
• Differentiates between data-fetching and knowledge-based queries.
• Routes queries correctly through the processing chain.
• Returns responses in a user-friendly and contextually relevant manner.

Using LangChain

• The orchestration of these agents is implemented using LangChain, a well-known framework for building AI-powered applications.
• LangChain allows seamless chaining of multiple agents, ensuring efficient query processing and response generation.

Example Scenarios

Case 1: Querying Internal Data

To illustrate how this system work, here’s an interaction between a user and the AI AI Assistant.

In this screenshot, the user asks for telemetry data insights, and the AI Agent retrieves real-time data from the internal source, the response is structured and context-aware, ensuring accuracy without external dependencies.

Case 2: Querying Non-Existing Data

As we mentioned earlier, there are strict instructions on the AI Assistant to respond with accurate and relevant text, so let’s try asking for a data that doesn’t have a record in the data source:

As shown, the user asks about telemetry data which isn’t presented in the data source, and the agent replies with No records found as it didn’t find the data that the user mentioned.

Case 3: Answering Contextually Relevant Questions

The AI Assistant isn’t limited to just query your data from the database, It can also answer your questions if it’s in the related context.

Case 4: Ignoring Irrelevant Questions

But, If a user, feeling a bit hungry, asks the AI Assistant for something unrelated to telemetry data—such as a Koshary recipe—would the AI Assistant still respond appropriately despite the change in context?

This demonstrates that the AI Assistant won’t answer any irrelevant questions, maintaining its focus on the intended context.

Case 5: Supporting multi-language

To ensure accessibility, the AI Assistant supports multiple languages (English, Arabic and German) providing insights without losing accuracy.

German:

Arabic (With Dialect support):

Tech Stack

Frontend

To build modern and interactive AI-powered UIs, frameworks like Next.js and Tailwind CSS are highly effective.

AI (Agent Orchestration)

For AI development, using LangChain or Ollama provides powerful capabilities for building intelligent applications.

Backend and Data Management

For backend development, frameworks like FastAPI and Python ensure efficient API handling and scalability. InfluxDB serves as a great choice for time-series data storage and analytics.

LLM

The model is capable of using different models like OpenAI ChatGPT, DeepSeek and Llama, providing flexibility.

Conclusion

By ensuring that insights come directly from internal telemetry data, organizations can trust their AI-driven decisions without the risks of external misinformation. Our AI-powered AI Assistant guarantees:

✅ Accurate, real-time insights from internal sources
✅ Strict separation from public knowledge bases
✅ LLM-agnostic and future-proof architecture
✅ Consistent, context-aware responses
✅ Multi-language support for global accessibility

As businesses continue to rely on AI for critical decision-making, it's essential to establish a system that prioritizes data integrity and relevance. With this structured approach, organizations can confidently extract insights without the pitfalls of scraping or external dependencies.