Artificial Intelligence

Hardened AI for Enterprise Linux.

Overview

Having a background in hard-core Python, AI infrastructure and applications is second-nature to me. I ship a vast amount of Python - from OpenStack, to Airflow, to my Plone/ECM, so I ship the leading AI suites in the same manner: LangChain, LangFlow, Phoenix, and more. My BastionLinux/Desktop is focused upon both security and curated AI tools for developers and users.

Drivers and GPU’s

I have great expertise and resources building kernel modules, drivers, and firmware, even for proprietary NVIDIA, AMD’s ROCm and build high-performance physical and virtual machines to drive GPU-intensive workloads.

Infrastructure Services

I ship LiteLLM for configuration/API keys/authorisation, instrumentation and Prometheus metrics/analytics. I use MLFlow and Phoenix, and even proprietary LangSmith and a lot of open telemetry <python3-opentelemetry> and open inference <python3-openinference> for observability, and LangFlow to LiteLLM for UI/prompt engineering assistance. I run vllm as a dedicated inference engine. These all ship as RPM packages, and have orchestration/lifecycle management for long term usage and support.

Prompt Engineering

I have quite some expertise writing prompt engines. These use JSONSchema to force structured data from the providers, and then do loads of interesting things with Pydantic to guarantee validity of results. These are all traceable with LangFlow and Phoenix. My prompts live in relational and object databases and are accessible via MCP. My prompts may be exported to the filesystem so they may be source code controlled and managed as software.

Model Context Protocol

I ship a bunch of MCP servers supporting my application suites: Grafana, PostgreSQL, shell <mcp-shell>. I have written MCP and LangChain services around my own BastionLinux build/deploy and Yum/DNF services aimed at validating, building, healing, and administering RHEL stacks - based on my complete Distro/Vendor backends.

I use and coordinate all my MCP <mcp>’ services with the :rpmsearch_like:`Argus and IBM/Chuk CLIs clients and Agent/Managers of MCP backends. This is all a highly evolving field, and I am of course not proscriptive about any of this - I am simply explaining the ecosystem I deploy.

Vector Databases

I have built and tried a bunch of Vector databases. In my main BastionLinux build stack, which is built upon an Enterprise Content Management stack - so well aware of document indexing, stemming, and other features, I have built a vector index for Zope and this supports my own local BastionLinux/LLM services.