Enterprise-grade vector database for AI applications at scale on Ubuntu 24.04
Get your Milvus instance running in minutes
Use your key pair to connect to the instance.
ssh -i your-key.pem ubuntu@your-instance-ip
Auto-generated root password on first boot.
sudo cat /opt/flowops/credentials.txt
Milvus takes ~90 seconds to fully start.
docker ps | grep milvus
curl http://localhost:9091/healthz
Verify Milvus is responding.
curl http://localhost:19530/v1/vector/collections
Startup Time: Milvus needs ~90 seconds to initialize. Health checks will fail until ready. Configure load balancer health checks with appropriate start periods.
Important locations and service management
# Root password and connection info
/opt/flowops/credentials.txt
# Contents include:
# - Root password for authentication
# - Connection endpoints
# - Instance metadata
# Docker Compose
/opt/flowops/docker-compose.yml
# Milvus configuration
/opt/flowops/milvus.yaml
# Embedded etcd configuration
/opt/flowops/embedEtcd.yaml
# Separate 20GB EBS volume
/milvus-data/milvus # Vector data
/milvus-data/etcd # Metadata
/milvus-data/logs # Application logs
# Check status
sudo systemctl status milvus
# Restart service
sudo systemctl restart milvus
# View logs
docker logs milvus-standalone -f
# View all container logs
cd /opt/flowops && docker compose logs -f
| Port | Service | Description |
|---|---|---|
19530 | Milvus API | gRPC and REST API endpoint |
9091 | Health/Metrics | Health checks and Prometheus metrics |
2379 | etcd | Metadata storage (internal) |
Collections are like tables - they hold your vectors and metadata
Define schema with vector field and metadata.
from pymilvus import (
connections, Collection,
FieldSchema, CollectionSchema, DataType
)
# Connect to Milvus
connections.connect(
host="localhost",
port="19530",
user="root",
password="your-password"
)
# Define fields
fields = [
FieldSchema(name="id", dtype=DataType.INT64,
is_primary=True, auto_id=True),
FieldSchema(name="embedding",
dtype=DataType.FLOAT_VECTOR, dim=1536),
FieldSchema(name="text",
dtype=DataType.VARCHAR, max_length=65535),
FieldSchema(name="metadata",
dtype=DataType.JSON)
]
# Create collection
schema = CollectionSchema(fields, "My documents")
collection = Collection("documents", schema)
Add data to your collection.
# Prepare data (1536-dim vectors)
embeddings = [[0.1, 0.2, ...], [0.3, 0.4, ...]]
texts = ["Hello world", "Milvus is great"]
metadata = [
{"source": "doc1", "page": 1},
{"source": "doc2", "page": 1}
]
# Insert data
collection.insert([embeddings, texts, metadata])
collection.flush() # Persist to disk
# Check row count
print(f"Rows: {collection.num_entities}")
Find similar vectors with optional filters.
# Load collection into memory
collection.load()
# Define search parameters
search_params = {
"metric_type": "COSINE",
"params": {"ef": 64}
}
# Search for similar vectors
results = collection.search(
data=[query_embedding],
anns_field="embedding",
param=search_params,
limit=10,
output_fields=["text", "metadata"],
expr='metadata["source"] == "doc1"'
)
for hits in results:
for hit in hits:
print(f"{hit.distance}: {hit.entity.get('text')}")
Remove vectors or entire collections.
# Delete by expression
collection.delete(expr='id in [1, 2, 3]')
# Delete by filter
collection.delete(expr='metadata["source"] == "old"')
# Drop entire collection
collection.drop()
# List all collections
from pymilvus import utility
print(utility.list_collections())
Choose the right index for your use case
| Index | Best For | Memory | Accuracy | Speed |
|---|---|---|---|---|
HNSW | General purpose, high recall | High | Excellent | Fast |
IVF_FLAT | High accuracy, moderate scale | Medium | Excellent | Medium |
IVF_SQ8 | Memory-constrained, large scale | Low | Good | Fast |
IVF_PQ | Billion-scale, memory-limited | Very Low | Moderate | Very Fast |
FLAT | Small datasets, 100% accuracy | High | Perfect | Slow |
SCANN | Large scale, good recall | Medium | Good | Fast |
Best balance of speed and accuracy for most use cases.
index_params = {
"index_type": "HNSW",
"metric_type": "COSINE", # or L2, IP
"params": {
"M": 16, # Connections (8-64)
"efConstruction": 256 # Build quality
}
}
collection.create_index("embedding", index_params)
# Search params for HNSW
search_params = {
"metric_type": "COSINE",
"params": {"ef": 64} # Higher = accurate
}
Good for exact search with clustering. Lower memory than HNSW.
index_params = {
"index_type": "IVF_FLAT",
"metric_type": "COSINE",
"params": {
"nlist": 1024 # Number of clusters
}
}
collection.create_index("embedding", index_params)
# Search params for IVF
search_params = {
"metric_type": "COSINE",
"params": {"nprobe": 16} # Clusters to search
}
Organize data for faster filtered queries
Split data by logical groups (e.g., by tenant, date).
# Create partitions
collection.create_partition("tenant_a")
collection.create_partition("tenant_b")
collection.create_partition("2024_q1")
# Insert into specific partition
collection.insert(data, partition_name="tenant_a")
# List partitions
print(collection.partitions)
# Check if partition exists
collection.has_partition("tenant_a")
Query specific partitions for faster results.
# Load specific partitions only
collection.load(partition_names=["tenant_a"])
# Search in partition only - Much faster!
results = collection.search(
data=[query_vector],
anns_field="embedding",
param=search_params,
limit=10,
partition_names=["tenant_a"]
)
# Search across multiple partitions
results = collection.search(
data=[query_vector],
anns_field="embedding",
param=search_params,
limit=10,
partition_names=["tenant_a", "tenant_b"]
)
Complete examples with PyMilvus
import openai
from pymilvus import connections, Collection, utility
from pymilvus import FieldSchema, CollectionSchema, DataType
# Connect to Milvus
connections.connect(
host="your-instance-ip",
port="19530",
user="root",
password="your-root-password" # From credentials.txt
)
def get_embedding(text):
"""Get embedding from OpenAI"""
response = openai.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding
def create_collection():
"""Create a collection if it doesn't exist"""
if utility.has_collection("documents"):
return Collection("documents")
fields = [
FieldSchema("id", DataType.INT64, is_primary=True, auto_id=True),
FieldSchema("embedding", DataType.FLOAT_VECTOR, dim=1536),
FieldSchema("text", DataType.VARCHAR, max_length=65535),
FieldSchema("metadata", DataType.JSON)
]
schema = CollectionSchema(fields, "RAG documents")
collection = Collection("documents", schema)
# Create HNSW index
collection.create_index("embedding", {
"index_type": "HNSW",
"metric_type": "COSINE",
"params": {"M": 16, "efConstruction": 256}
})
return collection
def store_documents(collection, documents):
"""Store documents with embeddings"""
embeddings = [get_embedding(doc["text"]) for doc in documents]
texts = [doc["text"] for doc in documents]
metadata = [doc.get("metadata", {}) for doc in documents]
collection.insert([embeddings, texts, metadata])
collection.flush()
def search_similar(collection, query, limit=5):
"""Find similar documents"""
collection.load()
query_embedding = get_embedding(query)
results = collection.search(
data=[query_embedding],
anns_field="embedding",
param={"metric_type": "COSINE", "params": {"ef": 64}},
limit=limit,
output_fields=["text", "metadata"]
)
return [
{
"text": hit.entity.get("text"),
"score": 1 - hit.distance,
"metadata": hit.entity.get("metadata")
}
for hit in results[0]
]
# Example usage
collection = create_collection()
# Store documents
docs = [
{"text": "Milvus is a vector database for AI", "metadata": {"source": "docs"}},
{"text": "RAG combines retrieval with generation", "metadata": {"source": "blog"}}
]
store_documents(collection, docs)
# Search
results = search_similar(collection, "How do I build AI search?")
for r in results:
print(f"{r['score']:.3f}: {r['text']}")
from langchain_milvus import Milvus
from langchain_openai import OpenAIEmbeddings
vectorstore = Milvus(
embedding_function=OpenAIEmbeddings(),
connection_args={
"host": "your-ip",
"port": "19530"
},
collection_name="docs"
)
from llama_index.vector_stores.milvus import MilvusVectorStore
vector_store = MilvusVectorStore(
uri="http://your-ip:19530",
dim=1536,
collection_name="docs"
)
pip install pymilvus
# With OpenAI
pip install pymilvus openai
# With LangChain
pip install langchain-milvus
Secure your Milvus instance
Authentication is enabled by default on this AMI.
# Default user: root
# Password: auto-generated on first boot
# Location: /opt/flowops/credentials.txt
# Connect with auth
connections.connect(
host="your-ip",
port="19530",
user="root",
password="your-password"
)
Add users with specific roles.
from pymilvus import utility
# Create a new user
utility.create_user(
user="app_user",
password="secure_password_here"
)
# List users
print(utility.list_users(include_role_info=True))
# Delete a user
utility.delete_user(user="old_user")
Manage permissions with roles.
from pymilvus import Role
# Create a role
role = Role("read_only")
role.create()
# Grant collection permissions
role.grant("Collection", "documents", "Search")
role.grant("Collection", "documents", "Query")
# Assign role to user
role.add_user("app_user")
# List roles
print(utility.list_roles())
Update the default password.
from pymilvus import utility
# Change password (as root)
utility.update_password(
user="root",
old_password="current-password",
new_password="new-secure-password"
)
# Reset another user's password
utility.reset_password(
user="app_user",
old_password="old_pass",
new_password="new_pass"
)
Security Best Practices: Change the root password after first login. Create application-specific users with minimal required permissions. Never expose port 19530 to the public internet without authentication. Use security groups to restrict access to known IPs only.
Protect your vector data
One-command setup for automated backups to Amazon S3.
# Run the setup script
sudo /opt/flowops/setup-backups.sh
# You'll be prompted for:
# - S3 bucket name
# - AWS region
# - Backup schedule (cron format)
# The script configures:
# - IAM role permissions (if attached)
# - Automated daily backups
# - 7-day retention policy
Create a backup on demand.
# Flush all collections first
from pymilvus import utility
for name in utility.list_collections():
Collection(name).flush()
# Stop Milvus
sudo systemctl stop milvus
# Backup data directory to S3
aws s3 sync /milvus-data s3://my-bucket/milvus-backup/$(date +%Y%m%d)/
# Start Milvus
sudo systemctl start milvus
Recover data from an S3 backup.
# Stop Milvus
sudo systemctl stop milvus
# Clear existing data
sudo rm -rf /milvus-data/milvus/*
sudo rm -rf /milvus-data/etcd/*
# Restore from S3
aws s3 sync s3://my-bucket/milvus-backup/20240115/ /milvus-data/
# Start Milvus
sudo systemctl start milvus
Use AWS EBS snapshots for point-in-time recovery.
# Find the data volume ID
lsblk
# Look for /milvus-data mount
# Create snapshot via AWS CLI
aws ec2 create-snapshot \
--volume-id vol-0123456789abcdef \
--description "Milvus backup $(date)"
# Or use AWS Console:
# EC2 > Volumes > Actions > Create Snapshot
Track performance and health
curl http://localhost:9091/healthz
# Response when healthy:
# {"status":"ok"}
# Use in load balancer:
# Health check path: /healthz
# Port: 9091
# Start period: 120s
curl http://localhost:9091/metrics
# Add to prometheus.yml:
scrape_configs:
- job_name: 'milvus'
static_configs:
- targets: ['your-ip:9091']
# Follow real-time logs
docker logs milvus-standalone -f
# Last 100 lines
docker logs milvus-standalone --tail 100
# All service logs
cd /opt/flowops
docker compose logs -f
milvus_datanode_flush_duration_seconds - Data persistence latencymilvus_querynode_search_latency_seconds - Search response timesmilvus_datacoord_stored_rows_count - Total vectors storedmilvus_querynode_collection_loaded_count - Collections in memoryprocess_resident_memory_bytes - Memory usage# Install CloudWatch agent
sudo /opt/flowops/setup-cloudwatch.sh
# Metrics sent to CloudWatch:
# - CPU, Memory, Disk usage
# - Container metrics
# - Custom Milvus metrics
# Set up alarms in AWS Console:
# - Memory > 80%
# - Disk > 85%
# - Health check failures
# Memory usage
free -h
# Disk usage
df -h /milvus-data
# CPU and process info
htop
# Docker stats
docker stats --no-stream
Get the most out of your instance
| Instance Type | RAM | Recommended For | Max Vectors (1536-dim) |
|---|---|---|---|
m5.large | 8GB | Development, testing | ~500K |
m5.xlarge | 16GB | Small production | ~2M |
r5.xlarge | 32GB | Production | ~5M |
r5.2xlarge | 64GB | Large scale | ~15M |
r5.4xlarge | 128GB | Enterprise | ~40M |
# Tune ef for HNSW search
# Higher ef = more accurate, slower
search_params = {
"metric_type": "COSINE",
"params": {
"ef": 64 # Default, good balance
# ef: 128 for 99%+ recall
# ef: 32 for faster search
}
}
# For IVF indexes, tune nprobe
search_params = {
"metric_type": "COSINE",
"params": {
"nprobe": 16 # More = accurate
}
}
# Release collection from memory
collection.release()
# Load only needed partitions
collection.load(partition_names=["active"])
# Check what's loaded
print(utility.loading_progress("documents"))
# Compact segments periodically
# Reduces memory fragmentation
collection.compact()
# Wait for compaction
collection.wait_for_compaction_completed()
# Insert in batches for better perf
BATCH_SIZE = 1000
for i in range(0, len(data), BATCH_SIZE):
batch = data[i:i+BATCH_SIZE]
collection.insert(batch)
# Flush after all inserts
collection.flush()
# Batch search - multiple queries
results = collection.search(
data=[query1, query2, query3],
anns_field="embedding",
param=search_params,
limit=10
)
# Check current index
print(collection.indexes)
# Drop and rebuild with new params
collection.release()
collection.drop_index()
# Create optimized index
collection.create_index("embedding", {
"index_type": "HNSW",
"metric_type": "COSINE",
"params": {
"M": 32, # More connections
"efConstruction": 512
}
})
SIMD Required: Milvus requires SSE4.2/AVX/AVX2 CPU instructions for optimal performance. Use m5/m6i/r5/r6i instance families. T2/T3 instances work but are slower. T2 instances are NOT recommended for production.
Common issues and solutions
# Check container status
docker ps -a
# View startup logs
docker logs milvus-standalone
# Check disk space
df -h /milvus-data
# Check memory
free -h
# Restart the service
sudo systemctl restart milvus
# Wait 90 seconds for startup
sleep 90
# Check if port is listening
netstat -tlnp | grep 19530
# Test locally
curl http://localhost:19530/v1/vector/collections
# Check security group allows 19530
# from your IP address
# Check container is running
docker ps | grep milvus
# Check memory usage
free -h
docker stats --no-stream
# Release unused collections
collection.release()
# Use smaller ef for search
search_params = {"params": {"ef": 32}}
# Solutions:
# 1. Upgrade to r5.xlarge or larger
# 2. Use IVF_SQ8 index (less memory)
# 3. Use partitions, load only needed
# Check index exists
print(collection.indexes)
# Ensure collection is loaded
collection.load()
print(utility.loading_progress("documents"))
# Lower ef for faster search
param = {"params": {"ef": 32}}
# Use partitions for filtered queries
results = collection.search(
...,
partition_names=["relevant"]
)
# Compact to optimize segments
collection.compact()
# etcd stores Milvus metadata
# Check etcd logs
docker logs milvus-etcd
# Check etcd data directory
ls -la /milvus-data/etcd/
# If etcd is corrupted, restore
# from backup or rebuild
# Clear etcd (CAUTION: loses data)
sudo systemctl stop milvus
sudo rm -rf /milvus-data/etcd/*
sudo systemctl start milvus
# Check credentials file
sudo cat /opt/flowops/credentials.txt
# Test connection
connections.connect(
host="localhost",
port="19530",
user="root",
password="your-password"
)
# Reset password if needed:
# Edit /opt/flowops/milvus.yaml
# Set authorizationEnabled: false
# Restart, change password, re-enable
Our UK-based support team is available 24/7 with guaranteed response within 24 hours
support@flowopsconsulting.co.uk