Setting up sovereign and secure AIOps for IT teams in regulated sectors with OpManager and Ollama

The adoption of AI is part of the new world order. Whether you’re making a deposit in your bank, shopping online, or booking a taxi, chances are there’s AI involved in the process.

“CEOs in our survey say that 25% of operational decisions are made by AI without human intervention. But by 2030, CEOs expect the share of operational decisions made by AI to almost double to 48%” ~ Source: IBM

Adoption of AI in ITOps or AIOps.

AIOps o Artificial intelligence in IT operations is a technology that emerged from the adoption of AI technologies in IT tools. AIOps platforms incorporate artificial intelligence and machine learning as a fundamental part of their operational workflow.

AIOps is particularly relevant for monitoring tools. Monitoring tools collect telemetry from IT systems, detect anomalies, and generate alarms. However, the large volume of telemetry and the architectural complexity of modern IT systems have made this more difficult. AIOps simplifies this with the following advantages:

• Reduce noise and false positives from telemetry data and find the real problems
• Analyze network behavior and point out subtle anomalies that are more difficult to detect manually
• Find the true root cause of problems when multiple systems fail with event correlation
• Forecast future network trends based on past data
• Summarize failure events and analyze system status to speed up repair
• Information generation and troubleshooting steps

Many monitoring and observability platforms have adopted AIOps into their operational workflows. However, most providers use cloud-based AI providers. This is not ideal for IT teams in some industries.

Some sectors are ‘excluded’ from AIOps

For organizations in the government, defence, healthcare and BFSI (banking, financial services and insurance) sectors, data is not just an asset; It is a matter of national, financial or personal security. Sending this data to the public cloud poses compliance and security risks. AI technologies are also notoriously volatile when these sectors need stable and consistent technologies to execute critical operations. These are some of the risks and limitations faced with AI technologies:

• Data exfiltration risks: Public endpoints are a very specific target. Attackers use fast injection to trick public LLMs into revealing previous session histories.
• Loss of control and telemetry: IT security and procurement teams cannot apply internal role-based access control (RBAC) on external platforms.
• Data residency regulations: Compliance standards like DORA (Digital Operational Resilience Act) and CJIS (Criminal Justice Information Services) could be violated by sending data to the public cloud.
• AI Policy Changes: Cloud-based AI providers often make unexpected changes to pricing, usage, privacy policies, capabilities, and access.

Only 23% of IT leaders are very confident in their organization’s ability to manage security and governance components when implementing GenAI tools ~ Source: Gartner

Enter ManageEngine OpManager, Sovereign AIOps for Secure Industries

OpManager is a network and infrastructure monitoring tool with powerful AIOps capabilities. Unlike other SaaS providers in the monitoring and observability market, you can deploy it locally. Most importantly, OpManager integrates with a local AI runtime like Ollama. Integrating OpManager with Ollama Helps IT teams achieve sovereign, truly isolated AIOps.

Integrating OpManager with Ollama offers the following advantages:

• No data ever leaves the network.
• All costs are upfront and easy, with no anxiety about the cost of AI tokens.
• Network latency issues will not impact AIOps functionalities.
• IT teams gain full control over system prompts, temperature settings, and custom context injection.

How to configure the OpManager-Ollama integration

Let’s take a look at the configuration steps for OpManager’s Ollama integration.

Phase 1: Ollama installation

You must deploy the Ollama runtime on a machine that your OpManager server can access.

1. Download and install the runtime

Navigate to the official Ollama download page and obtain the binary or installation package tailored to your host operating system (Windows, Linux or macOS). Run the installer on your target machine.

2. Model selection and hardware assignment.

Make sure your host machine has adequate computing and memory resources. While full execution on a CPU is supported, leveraging a dedicated GPU significantly optimizes token processing and inference speeds. Allocate resources based on the chosen model size:

• Basic models (3B–4B): Minimum 4 GB of RAM (8 GB recommended).
• Mid-range models (8B–9B): Minimum 12 GB of RAM (16 to 24 GB recommended).
• Enterprise/high-end models (20,000 to 70,000 million): From 32 GB to 48+ GB of RAM.

3. Pull local LLM weights.

Uses of OpManager gpt-oss:20b as your default model architecture. If your hardware meets the 20B requirements, use the default weights. Otherwise, you can opt for a highly optimized mid-range alternative like llama3.

Open your terminal or command prompt and run:

ollama pull gpt-oss:20b

(Alternatively, replace it with ollama pull llama3 if resources are limited).

Verify that the model weights have been successfully initialized on your local disk:

ollama list

Phase 2: Establish the integration in OpManager

With your local AI actively running, you can now instruct OpManager to route your telemetry messages to your private endpoint.

1. Log in to your OpManager Web Client console using an administrator profile.
2. Navigate to Settings >> General settings >> Integrations >> Be. Click configure.
3. Fill in the required configuration parameters:

• Organization name: Enter your company or organization ID.
• Basic URL: Enter the network path of your Ollama deployment (e.g. http://localhost:11434 either http://:11434).

4. Please read the explicit data privacy and local compliance notice that appears on the screen, and then check the acknowledge box.
5. Click Save to terminate the underlying API connection.

Phase 3: Redirect the default AI model (optional)

If you downloaded an alternative model like llama3 during the installation phase instead of the default value gpt-oss:20bYou need to modify the OpManager backend parameters to point to the correct model ID.

1. Access the file system of your underlying OpManager server.
2. Navigate to the main configuration directory: /conf/OpManager/
3. Open the serverparameters.conf file using a plain text editor.
4. Scroll to the end of the file and add the following configuration parameter line:

OLLAMA_GENAI_MODEL 

Example of mapping to Llama 3:

OLLAMA_GENAI_MODEL llama3

5. Save and close the configuration file.
6. Restart the OpManager service to reload the updated server parameters into active system memory.

What can you do with this integration?

Once the handshake is established between OpManager and Ollama, your IT team can immediately leverage local intelligence across five critical workflows without a single byte of data leaving your perimeter:

• Summary of active alarms: OpManager generates alarms if your systems do not behave as expected. With the summary, you get a quick overview of everything that is wrong in your environment.
• Alarm summary and analysis: You can drill down into any individual alarm and analyze raw system telemetry to gain immediate context, potential root causes, and troubleshooting steps.
• Asset Group Summary: If you have created groups to manage your IT assets in OpManager, you can perform quick checks on the operational status of your network assets as a whole.
• IT Asset Summary: Drill down into any IT asset and instantly translate graphs and real-time performance metrics into a clear text narrative.
• Automated script generation: Monitor any custom metrics by generating tracking scripts within the AI-powered OpManager interface. Describe your requirements in a natural language message. The local model will instantly write ready-to-run monitoring scripts in PowerShell, Python, or Shell.

What are the AIOps features available in OpManager?

Beyond on-premise LLM integrations, OpManager also enables other AIOps features to simplify your IT operations.

• Built-in AI assistance: OpManager has Zoho’s Zia AI integrated into its interface. Zia’s machine learning (ML) engine can help you run system resource forecasts and generate monitoring information.
• Model Context Protocol (MCP) Server: OpManager supports a native MCP server. You can use its natural language interface to bring together your monitoring stack, helpdesk, DevOps tools, and coding wizards into a single console.
• Cloud-based GenAI integrations: You can also integrate with public AI vendors like OpenAI and DeepSeek to get real-time IT insights.

Start your AI sovereignty journey today!

OpManager offers a thirty day free trial. Contact our product experts to see how your IT team can implement secure, sovereign AIOps!