AI is moving fast. Organizations are deploying local LLMs - Ollama, vLLM, LM Studio - directly on internal infrastructure, often without the security controls they'd apply to any other network service. The result is a growing attack surface that most security teams haven't caught up to yet.

This guide exists to close that gap.

What you'll find here is a practical, end-to-end walkthrough for hardening a local AI deployment on Ubuntu Server, integrated into a Windows Active Directory environment. Every step has been tested in a real lab. No theory-only recommendations - every control shown is one you can implement today.

What this volume covers:

Securing the Ubuntu host: system updates, UFW firewall, SSH hardening, Fail2Ban brute-force protection, and AppArmor mandatory access control
Ollama installation and hardening: localhost-only binding, CORS lockdown, model allow-listing, and request timeout controls
Joining Ubuntu to a Windows domain via Kerberos and SSSD
Deploying an internal Certificate Authority using Active Directory Certificate Services (AD CS)
Reverse proxy setup with Nginx: enforcing HTTPS (TLS 1.2/1.3), Basic authentication, rate limiting, and security headers - for both the raw Ollama API and the Open WebUI chat interface
Container hardening: resource limits, capability drops, privilege escalation prevention, and network isolation

Whether you're a sysadmin standing up your first enterprise AI server, a security engineer building controls around an existing deployment, or a developer who wants to understand how the security layer wraps the model backend - this guide walks you through it step by step.

Note: For domain setup and Windows Active Directory configuration, refer to the companion guide at antusnet.ca/blog-grid/guides/active-directory-lab/

Task Details

1. Install Docker and update Ubuntu server.

2. Ollama Installation & Hardening

3. Ubuntu Server System Security: SSH hardening, Fail2Ban for brute-force protection, and AppArmor/SELinux.

4. Join Ubuntu Server to the Windows Domain

5. Install and Configure the CA on Windows domain.

6. Reverse Proxy & HTTPS: Implement authentication mechanizm for raw Ollama API - for developers, scripts, API calls

7. Reverse Proxy & HTTPS: Implement authentication mechanizm for Open WebUI - for end users, chat interface.

8. Container hardening on Ubuntu

All commands can be copied from here.

Steps

Install Docker and update the Ubuntu server.

Before deploying any AI/LLM tools, the Ubuntu server must be fully updated to apply the latest security patches and system fixes. This reduces vulnerabilities and ensures a stable base system.

Docker is then installed to run services like Ollama and supporting security tools in isolated containers, improving portability and reducing system risk.

*
1. Update Ubuntu Server

sudo apt update && sudo apt upgrade -y

Note: I used the Ubuntu Server GUI for demonstration purposes. Command: sudo apt install ubuntu-desktop

2. Install Docker.

curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
newgrp docker

--SNIP--

3. Then verify Docker works:

docker run hello-world

Ollama Installation & Hardening

Local LLM deployments allow AI models to run directly on your own infrastructure instead of relying on external cloud APIs. In this guide, we use Ollama to demonstrate a secure and practical way to host and interact with large language models locally.

This approach improves privacy, reduces data exposure, and gives full control over model access, logging, and network exposure. It also introduces new security considerations such as local API protection, access control, and safe integration with internal applications.

1. Install Ollama on Ubuntu server

curl -fsSL https://ollama.com/install.sh | sh

Note: In enterprise environments, Linux (Ubuntu/RHEL) is dominant for running LLMs.

Note: VMs do not natively access the host GPU, but GPU access is available on a bare-metal Ubuntu installation.

2. Verify Binding (Critical)

ss -tulpn | grep 11434

Note: The single most important check: Ollama must only listen on localhost. If you see '0.0.0.0:11434' or ':::11434' Restrict immediately.

Linux - edit systemd service

sudo systemctl edit ollama

Add the following block: (Will edit this later)

[Service]
Environment="OLLAMA_HOST=127.0.0.1"

Reload and restart

sudo systemctl daemon-reload
sudo systemctl restart ollama

3. Pull a model to test with.

ollama pull llama3.2

4. Ollama Hardening Checklist

Ubuntu Server System Security: SSH hardening, Fail2Ban for brute-force protection, and AppArmor/SELinux.

Securing the underlying OS is foundational. A compromised host gives attackers direct access to model weights, embeddings, and all API traffic.

Linux Hardening

1. Apply all security updates.

sudo apt update && sudo apt upgrade -y

2. Enable automatic security updates

sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure --priority=low unattended-upgrades

--SNIP--

3. Enable and configure UFW firewall

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 443/tcp
sudo ufw enable

3. Verify rules

sudo ufw status verbose

SSH Hardening

1. Configure the /etc/ssh/sshd_config file using the following recommended settings:

nano /etc/ssh/sshd_config

Uncomment / comment or add those lines and adjust the values if needed.

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2

2. Restart SSH for the settings to take effect.

sudo systemctl restart ssh

Fail2Ban for Brute-Force Protection

Fail2Ban is a security tool used to protect services like SSH by automatically blocking IPs that show repeated failed login attempts. It helps reduce brute-force attacks and strengthens the overall security posture of the system.

1. Install Fail2Ban

sudo apt install fail2ban -y

2. Configure the /etc/fail2ban/jail.local file using the following recommended settings:

nano /etc/fail2ban/jail.local

Paste this

[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 5

[sshd]
enabled = true

3. Restart Fail2Ban to apply settings.

sudo systemctl restart fail2ban

4. Check the status and ensure it is enabled. If not, run sudo systemctl enable fail2ban.

sudo systemctl is-enabled fail2ban

AppArmor / SELinux

AppArmor and SELinux are mandatory access control (MAC) frameworks that restrict what processes can do on a system, even if they are compromised. They add an extra security layer by enforcing fine-grained policies to limit application behavior and reduce the impact of potential breaches.

1. Verify AppArmor is active

sudo aa-status

Note: What you see is actually perfect:

apparmor module is loaded
185 profiles are loaded
88 profiles are in enforce mode

Join Ubuntu Server to the Windows Domain

All commands can be copied from here.

Note: For instructions on installing and configuring a Windows domain and joining hosts to the domain, refer to this guide. : https://antusnet.ca/blog-grid/guides/active-directory-lab/

Joining the Ubuntu machine to the domain before requesting a certificate makes the entire process significantly cleaner and reduces configuration issues.

1. Define your gateway first.

ip route

2. Change IPv4 settings before joining the domain.

Go to Network → → IPv4 tab → set to manual → fill up your details.

2. Install required packages on Ubuntu.

sudo apt install -y realmd sssd sssd-tools adcli krb5-user samba-common packagekit

3. Set the Kerberos realm as your domain.

sudo nano /etc/krb5.conf

Replace all default lines with:

sudo bash -c 'cat > /etc/krb5.conf << EOF
[libdefaults]
default_realm = ANTUSNET.CA
dns_lookup_realm = false
dns_lookup_kdc = true
forwardable = true

[realms]
ANTUSNET.CA = {
kdc = DC1.antusnet.ca
admin_server = DC1.antusnet.ca
}

[domain_realm]
.antusnet.ca = ANTUSNET.CA
antusnet.ca = ANTUSNET.CA
EOF'

Note: Replace antusnet.ca with your domain

4. Now test that Ubuntu can resolve the domain.

nslookup antusnet.ca
nslookup DC1.antusnet.ca

5. Join the domain.

sudo realm join --user=Administrator antusnet.ca

Note: It will prompt for the Administrator password - enter your Windows domain admin password.

6. Then verify.

sudo realm list

Ubuntu is now a fully joined domain member.

7. Verify you can see the Ubuntu computer object in AD from Windows

Get-ADComputer -Filter {Name -eq "9052206"}

8. Test AD authentication from Ubuntu

id Administrator@antusnet.ca

Install and Configure the CA on Windows domain

Note: For instructions on installing and configuring a Windows domain and joining hosts to the domain, refer to this guide. : https://antusnet.ca/blog-grid/guides/active-directory-lab/

A Certificate Authority (CA) provides centralized certificate management for secure internal communications and HTTPS services within a Windows domain environment. In this section, Active Directory Certificate Services (AD CS) will be installed and configured to issue trusted certificates for systems and applications such as Nginx and Ollama.

1. Install AD CS on Windows Server (run as Administrator)

Get-WindowsFeature ADCS-Cert-Authority
Install-WindowsFeature ADCS-Cert-Authority -IncludeManagementTools

2. Configure the CA.

Go to server manager
Click the flag/notification icon at the top right
Click "Configure Active Directory Certificate Services"
Follow the wizard - select Enterprise Root CA, accept all defaults
Click through to finish

3. In server roles also check:

Certificate Enrollment Policy Web Service
Certificate Enrollment Web Service

Click four times next, then install

Note: You need to also enable the Certificate Enrollment Web Service components so Ubuntu can request certificates via the web interface.

4. Work through the wizard:

Credentials: use your domain admin account
Role Services: check Certification Authority
Setup Type: Enterprise CA
CA Type: Root CA
Private Key: Create a new private key
Cryptography: SHA256, 2048 key length
CA Name: leave default or set something like YourDomain-CA
Validity Period: set to 1 year (avoids the trial issue we hit earlier)
Click through to Configure

5. Select Certification Authority then click Next.

Note: You will install the Certificate Enrollment Web Service and Certificate Enrollment Policy Web Service later on.

6. Select Setup Type "Enterprise CA" and then click next

7. Select CA Type "Root CA" and then click "Next."

8. Private Key: Create a new private key, then click next

9. Cryptography: SHA256, 2048 key length then click text

10. CA Name: leave default or set something like YourDomain-CA

11. Validity Period: set to 1 year (avoids the trial issue if you using a trial).

Click next twice

12. Click through to configure.

13. Click Yes - this will let you configure the additional web enrollment services we need so Ubuntu can request certificates from the CA over the network.

Click next twice

14. Select Windows integrated authentication (leave it as default) and click Next.

Note: Select a username and password if your Ubuntu server is not domain-joined.

15. Select “Use the built-in application pool identity."

Click Next to complete the wizard

Note: On production deployments, run the Certificate Enrollment Web Service on a dedicated member server, not the Domain Controller. This avoids the IIS_IUSRS local group limitation and follows the principle of role separation.

16. Your CA has already automatically issued certificates. You can see two certificates issued by antusnet-DC1-CA valid until 2027.

Select DC1.antusnet.ca - this is the correct server certificate for the web enrollment service since it matches the server's fully qualified domain name.
Then click Next and continue to the Confirmation screen.

17. Click "Configure."

Reverse Proxy & HTTPS: Implement authentication mechanism for the raw Ollama API for developers, scripts, API calls

All commands can be copied from here.

This section covers securing the raw Ollama API for developers, scripts, and API integrations by implementing a reverse proxy, HTTPS encryption, and authentication controls.

A reverse proxy is mandatory between users and Ollama. It enforces HTTPS, adds authentication, enables rate limiting, and centralizes logging - none of which Ollama provides natively.

Important: Modern browsers (Chrome, Edge, Firefox) require a Subject Alternative Name (SAN) field in the certificate. A CN-only certificate will show NET::ERR_CERT_COMMON_NAME_INVALID and be rejected. Always use a SAN config when generating CSRs.

1. Install Nginx

sudo apt install nginx certbot python3-certbot-nginx -y
sudo systemctl enable nginx

Note: Nginx uses port 80, so if your Apache2 server is running stop and disable it

--SNIP--

2. Generate the private key and CSR (Certificate Signing Request) on Ubuntu.

Create config with Subject Alternative Name

sudo bash -c 'cat > /etc/ssl/ollama.cnf << EOF
[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no

[req_distinguished_name]
CN = ollama.antusnet.ca
O = Antusnet
C = CA

[v3_req]
subjectAltName = @alt_names

[alt_names]
DNS.1 = ollama.antusnet.ca
IP.1 = 192.168.248.102
EOF'

Note: IP.1 is the Ubuntu server IP

Generate key and CSR with SAN

sudo openssl req -new -newkey rsa:2048 -nodes \
-keyout /etc/ssl/private/ollama.key \
-out /etc/ssl/ollama.csr \
-config /etc/ssl/ollama.cnf

3. View and copy the CSR (Certificate Signing Request).

cat /etc/ssl/ollama.csr

4. Copy paste CSR (Certificate Signing Request) to Windows DC C:\ollama.csr

5. On Windows, submit certificate to CA

certreq -submit -attrib "CertificateTemplate:WebServer" C:\ollama.csr C:\ollama.cer
Select DC1.antusnet.ca then click ok

Note: This command submits the ollama.csr certificate signing request to a Windows Certificate Authority using the WebServer template and saves the issued certificate as ollama.cer

6. On Windows, create a DNS record for the Ubuntu server.

Add-DnsServerResourceRecordA -ZoneName "antusnet.ca" -Name "ollama" -IPv4Address "192.168.248.102"

Note: This command creates an A record in the antusnet.ca DNS zone that maps ollama.antusnet.ca to the IP address 192.168.248.102

7. On Ubuntu Copy / paste c:\ollama.cer cert back to Ubuntu in the correct location.

gedit /etc/ssl/certs/ollama.crt

8. Assign appropriate permissions to ollama.crt

sudo chmod 644 /etc/ssl/certs/ollama.crt
sudo chmod 600 /etc/ssl/private/ollama.key
sudo chown root:www-data /etc/nginx/.htpasswd
sudo systemctl reload nginx

Note: These commands set secure permissions for the SSL certificate, private key, and Nginx authentication file, then reload Nginx to apply the changes.

9. Configure Nginx to use the ollama.cer certificate.

sudo nano /etc/nginx/sites-available/ollama

Replace the contents with:

limit_req_zone $binary_remote_addr zone=ai_limit:10m rate=10r/m;

server {
listen 443 ssl;
server_name ollama.antusnet.ca;

ssl_certificate /etc/ssl/certs/ollama.crt;
ssl_certificate_key /etc/ssl/private/ollama.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;

auth_basic "AI Access";
auth_basic_user_file /etc/nginx/.htpasswd;

limit_req zone=ai_limit burst=5 nodelay;

add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header Strict-Transport-Security "max-age=31536000";

location / {
proxy_pass http://127.0.0.1:11434;
proxy_set_header Host $host;
proxy_read_timeout 300s;
}
}

server {
listen 80;
server_name ollama.antusnet.ca;
return 301 https://$host$request_uri;
}

Note: In enterprise environments this is exactly how it works - the security layer is completely decoupled from the AI backend, so you can swap models or runtimes without touching any security configuration.

You just change the proxy_pass line to point to whichever LLM server you're running.

10. Enable the Nginx site and create the auth credentials for aiuser.

sudo ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/ollama
sudo htpasswd -c /etc/nginx/.htpasswd aiuser

11. Test and reload Nginx

sudo nginx -t
sudo systemctl reload nginx

12. On Windows and Ubuntu, verify the DNS record exists.

nslookup ollama.antusnet.ca

13. Test if Ollama responds locally.

curl http://127.0.0.1:11434/api/generate \
-d '{"model":"llama3.2","prompt":"hello","stream":false}'

14. On Windows domain, export the root CA certificate and install it to the trusted root store.

Export

$cert = Get-ChildItem -Path Cert:\LocalMachine\Root | Where-Object {$_.Subject -like "*antusnet*"} | Select-Object -First 1
Export-Certificate -Cert $cert -FilePath C:\antusnet-root-ca.cer

Note: These commands locate the antusnet root CA certificate in the local machine certificate store and export it to C:\antusnet-root-ca.cer.

Install

certutil -addstore "Root" C:\antusnet-root-ca.cer

Note: CA certificate is trusted - antusnet-DC1-CA already in store and command completed successfully

15. install the CA cert on Ubuntu so curl works

Note: You need the Base64 PEM format for curl to work.

On Windows, run this command and copy:

$cert = Get-ChildItem -Path Cert:\LocalMachine\Root | Where-Object {$_.Subject -like "*antusnet*"} | Select-Object -First 1 Export-Certificate -Cert $cert -FilePath C:\antusnet-root-ca.cer certutil -encode C:\antusnet-root-ca.cer C:\antusnet-root-ca-b64.cer type C:\antusnet-root-ca-b64.cer

Note: These commands export the antusnet root CA certificate, convert it to Base64 format, and display the encoded certificate contents.

16. Copy & Paste output to Ubuntu

sudo nano /usr/local/share/ca-certificates/antusnet-dc1-ca.crt

17. Add these lines to ollama config

sudo systemctl edit ollama

Add these lines:

[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_ORIGINS=https://ai.antusnet.ca,https://ollama.antusnet.ca"

Note: We also specify Https://ai.antusnet.ca for WebUI

Then restart Ollama

sudo systemctl daemon-reload
sudo systemctl restart ollama

18. Add firewall rule

sudo ufw allow in on docker0 to any port 11434
sudo ufw reload

19. Update the certificate and test the Ollama API.

sudo update-ca-certificates
curl http://127.0.0.1:11434/api/tags
curl -v -u aiuser:@Password123 https://ollama.antusnet.ca/api/tags

Note: The first curl returned empty because of the rate limiting we configured - you hit the limit from previous attempts. It's working, the rate limiter just blocked that specific request.

Your full security stack is now operational:

✅ Ollama bound to localhost only
✅ Nginx reverse proxy with HTTPS (TLS 1.3)
✅ Certificate from your own AD CS CA
✅ Basic authentication enforced
✅ Rate limiting active
✅ Security headers in place
✅ Open WebUI running at https://ai.antusnet.ca

20. How would enterprise developers integrate your secured Ollama endpoint into their applications?

Here are the common ways a developer would interact with your secured Ollama endpoint:

List available models:

curl -u aiuser:@Password123 https://ollama.antusnet.ca/api/tags

Generate a response:

curl -u aiuser:@Password123 https://ollama.antusnet.ca/api/generate \
-d '{"model":"llama3.2","prompt":"What is cybersecurity?","stream":false}'

Chat completion (multi-turn):

curl -u aiuser:@Password123 https://ollama.antusnet.ca/api/chat \
-d '{
"model": "llama3.2",
"messages": [
{"role": "user", "content": "Explain prompt injection in 2 sentences"}
],
"stream": false
}'

From Windows PowerShell:

$cred = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("aiuser:@Password123"))
Invoke-RestMethod -Uri "https://ollama.antusnet.ca/api/tags" -Headers @{Authorization="Basic $cred"}

Reverse Proxy & HTTPS: Implement an authentication mechanism for Open WebUI - for end users, a chat interface.

All commands can be copied from here.

This section covers securing Open WebUI for end users by implementing a reverse proxy, HTTPS encryption, and authentication for the chat interface.

1. Install Open WebUI - that's what gives you a proper browser interface.

On Ubuntu

Get Docker bridge (gateway) IP

docker network inspect bridge | grep Gateway

2. Update the Open WebUI container to use the gateway IP

docker run -d \
--name open-webui \
--restart always \
-p 3000:8080 \
-e OLLAMA_BASE_URL=http://172.17.0.1:11434 \
-v open-webui:/app/backend/data \
ghcr.io/open-webui/open-webui:main

Note: This command deploys the Open WebUI Docker container, configures it to connect to the Ollama API, exposes it on port 3000, and enables persistent storage and automatic restarts.

2. Then add a DNS record on Windows for the WebUI

On Windows domain

Add-DnsServerResourceRecordA -ZoneName "antusnet.ca" -Name "ai" -IPv4Address "192.168.248.102"

Note: This command creates an A record in the antusnet.ca DNS zone that maps ai.antusnet.ca to the IP address 192.168.248.102

3. Verify WebUI is running

docker ps | grep open-webui

4. Add WebUI to Nginx so you can access it securely via HTTPS.

Create a new Nginx config for Open WebUI

sudo nano /etc/nginx/sites-available/ai-webui

Paste:

server {
listen 443 ssl;
server_name ai.antusnet.ca;

ssl_certificate /etc/ssl/certs/ai.crt;
ssl_certificate_key /etc/ssl/private/ai.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;

add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header Strict-Transport-Security "max-age=31536000";

location / {
proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 300s;

proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}

server {
listen 80;
server_name ai.antusnet.ca;
return 301 https://$host$request_uri;
}

5. Enable and reload.

sudo ln -s /etc/nginx/sites-available/ai-webui /etc/nginx/sites-enabled/ai-webui
sudo nginx -t
sudo systemctl reload nginx

Note: These commands enable the Nginx configuration for the AI WebUI site, test the Nginx configuration for errors, and reload Nginx to apply the changes.

6. Dont forget to add DNS on Windows

Add-DnsServerResourceRecordA -ZoneName "antusnet.ca" -Name "ai" -IPv4Address "192.168.248.102"

Note: This command creates an A record in the antusnet.ca DNS zone that maps ai.antusnet.ca to the IP address 192.168.248.102

7. Open WebUI is live and working! The "Not secure" warning is just because ai.antusnet.ca needs its own certificate - the current cert was issued only for ollama.antusnet.ca.

Two things to do:

Fix the certificate - issue a new one covering both domains. We'll do that shortly.
First, create your admin account - click Get started and register your admin user. This is the first account, so it automatically gets admin privileges.

Create the account, and then we'll fix the certificate warning.

Important!

Lock down signup now that admin is created

We do it because in an enterprise you control who gets access - not the users themselves. Without it anyone on the network can self-register and start using your AI system immediately.

On Ubuntu run:

docker stop open-webui
docker rm open-webui

docker run -d \
--name open-webui \
--restart always \
-p 3000:8080 \
-e OLLAMA_BASE_URL=http://172.17.0.1:11434 \
-e ENABLE_SIGNUP=false \
-e DEFAULT_USER_ROLE=pending \
-v open-webui:/app/backend/data \
ghcr.io/open-webui/open-webui:main

Note: These commands stop and remove the existing Open WebUI container, then restart it with updated settings including disabled sign-up and a default pending user role while keeping Ollama connectivity and persistent storage.

8. Configure HTTPS for WebUI user interface.

Generate CSR on Ubuntu:

sudo bash -c 'cat > /etc/ssl/ai.cnf << EOF
[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no

[req_distinguished_name]
CN = ai.antusnet.ca
O = Antusnet
C = CA

[v3_req]
subjectAltName = @alt_names

[alt_names]
DNS.1 = ai.antusnet.ca
DNS.2 = ollama.antusnet.ca
IP.1 = 192.168.248.102
EOF'

sudo openssl req -new -newkey rsa:2048 -nodes \
-keyout /etc/ssl/private/ai.key \
-out /etc/ssl/ai.csr \
-config /etc/ssl/ai.cnf

Note: These commands create an OpenSSL configuration for a certificate with SAN entries and then generate a new private key and CSR for ai.antusnet.ca and related domains.

9. View CSR and copy everything including -----BEGIN CERTIFICATE REQUEST----- and -----END CERTIFICATE REQUEST-----, paste into C:\ai.csr on Windows domain.

cat /etc/ssl/ai.csr

Then on Windows domain submit:

certreq -submit -attrib "CertificateTemplate:WebServer" C:\ai.csr C:\ai.cer
Select DC1.antusnet.ca then click ok.

Note: This command submits the ai.csr certificate request to a Windows CA using the Web Server template and saves the issued certificate as ai.cer

Note: If you get an error related validity period then disable CRL checking for now.

certutil -setreg ca\CRLFlags +CRLF_REVCHECK_IGNORE_OFFLINE
net stop certsvc
net start certsvc

Note: These commands configure the CA to ignore offline revocation checks and then restart the Certificate Services to apply the change.

10. Copy C:\ai.cer (On Windows domain) to Ubuntu. Paste the contents:

Copy ai.cer on Windows domain

Paste ito /etc/ssl/certs/ai.crt on Ubuntu

sudo nano /etc/ssl/certs/ai.crt

11. Make sure you completed this step

Then update the Nginx config to use the new cert.

sudo nano /etc/nginx/sites-available/ai-webui

Replace the content with:

server {
listen 443 ssl;
server_name ai.antusnet.ca;

ssl_certificate /etc/ssl/certs/ai.crt;
ssl_certificate_key /etc/ssl/private/ai.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;

add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header Strict-Transport-Security "max-age=31536000";

location / {
proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 300s;

proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}

server {
listen 80;
server_name ai.antusnet.ca;
return 301 https://$host$request_uri;
}

12. Then reload Nginx.

sudo nginx -t
sudo systemctl reload nginx

13. Test the WebUI interface.

Open the browser at ai.antusnet.ca, log in with your credentials, and ask a question.

Everything is working perfectly now!

✅ https://ai.antusnet.ca - trusted certificate, no warning
✅ Open WebUI responding with llama3.2
✅ WebSocket working - real-time streaming response
✅ Full answer to "what is cybersecurity?"

Bonus: To create new users go to admin panel

Click + icon to add new user

Container hardening on Ubuntu

Stop and remove the current container

docker stop open-webui
docker rm open-webui

Run with full security hardening

docker run -d \
--name open-webui \
--restart always \
-p 127.0.0.1:3000:8080 \
-e OLLAMA_BASE_URL=http://172.17.0.1:11434 \
-e ENABLE_SIGNUP=false \
-e DEFAULT_USER_ROLE=pending \
--memory 2g \
--cpus 2.0 \
--security-opt no-new-privileges:true \
--cap-drop ALL \
-v open-webui:/app/backend/data \
ghcr.io/open-webui/open-webui:main

Note: This command runs the Open WebUI Docker container with restricted resource limits, hardened security options, disabled sign-up, and local-only access while connecting it to the Ollama backend.

Conclusion

*
You've Built a Production-Grade Secure AI Stack
At this point, your deployment has controls that most enterprise AI rollouts skip entirely. Here's what you now have running:

Ollama bound to localhost - the API is not reachable from the network without going through the proxy
Nginx reverse proxy enforcing HTTPS with a certificate signed by your own internal CA
Basic authentication on the raw API endpoint - developers and scripts authenticate before they get a response
Rate limiting preventing GPU exhaustion from runaway or malicious request loops
Security headers (X-Frame-Options, X-Content-Type-Options, HSTS) on all exposed endpoints
Open WebUI behind HTTPS with self-registration disabled — only admin-provisioned accounts can log in
Docker container hardened with memory and CPU limits, all Linux capabilities dropped, and no privilege escalation path
Ubuntu host with automatic security updates, SSH key-only authentication,
Fail2Ban active, and AppArmor enforcing profiles

This is the security layer. It's completely decoupled from the model backend - swap llama3.2 for any other model, or replace Ollama with vLLM or LM Studio, and your security configuration stays exactly the same. That's the architecture working as intended.

What's Coming in Volume 2

Volume 1 secures the infrastructure layer. Volume 2 goes deeper - into the AI layer itself.
Enterprise AI & LLM Security Hardening Guide - Volume 2 will cover:

RAG pipeline security - protecting retrieval-augmented generation systems from data poisoning and unauthorized document access
Vector database security - hardening Qdrant, Weaviate, and similar stores against unprotected REST and gRPC endpoints
Prompt injection defense - detecting and mitigating direct and indirect prompt injection attacks against your models
AI monitoring & logging - building observability into model inference: what was asked, what was returned, and when something looks wrong
GPU protection - isolating GPU access, preventing resource exhaustion, and securing hardware in shared environments
Secrets management - keeping API keys, credentials, and model access tokens out of configs, containers, and logs
Red teaming - adversarial testing of your AI deployment: jailbreaks, prompt leakage, model extraction attempts
Governance & compliance - aligning your AI security posture with NIST AI RMF, OWASP LLM Top 10, and enterprise policy frameworks

Follow along at antusnet.ca to get Volume 2 when it drops.

➤ Want more? Browse all our Azure implementation guides.

Need help implementing secure Azure solutions?

Contact us for a free consultation.

Azure Quick Links

Enterprise AI & LLM Projects

Enterprise AI & LLM Security Hardening

This guide provides a practical, hands-on walkthrough for securing a local AI deployment on Ubuntu Server integrated with a Windows Active Directory environment.

Enterprise AI & LLM Security Hardening Guide - Introduction

➤ Want more? Browse all our Azure implementation guides.

Need help implementing secure Azure solutions?

Contact us for a free consultation.

Looking for a free quote?

Company

Quick Link

Contact Us

Enterprise AI & LLM Security Hardening Guide - antusnet.ca

Azure Quick Links

Enterprise AI & LLM Projects

Enterprise AI & LLM Security Hardening

This guide provides a practical, hands-on walkthrough for securing a local AI deployment on Ubuntu Server integrated with a Windows Active Directory environment.

Enterprise AI & LLM Security Hardening Guide - Introduction

➤ Want more? Browse all our Azure implementation guides.

Need help implementing secure Azure solutions?

Contact us for a free consultation.

Looking for a free quote?

Company

Quick Link

Contact Us