Home
AWS Marketplace AMI

Milvus Vector Database

Enterprise-grade vector database for AI applications at scale on Ubuntu 22.04

Milvus v2.4 Standalone OS Ubuntu 22.04 Docker Powered

Quick Start Guide

Get your Milvus instance running in minutes

1

Connect via SSH

Use your key pair to connect to the instance.

ssh -i your-key.pem ubuntu@your-instance-ip
2

View Credentials

Auto-generated root password on first boot.

sudo cat /opt/flowops/credentials.txt
3

Check Service Status

Milvus takes ~90 seconds to fully start.

docker ps | grep milvus curl http://localhost:9091/healthz
4

Test the API

Verify Milvus is responding.

curl http://localhost:19530/v1/vector/collections

Startup Time: Milvus needs ~90 seconds to initialize. Health checks will fail until ready. Configure load balancer health checks with appropriate start periods.

Credentials & File Paths

Important locations and service management

Credentials

# Root password and connection info /opt/flowops/credentials.txt # Contents include: # - Root password for authentication # - Connection endpoints # - Instance metadata

Configuration Files

# Docker Compose /opt/flowops/docker-compose.yml # Milvus configuration /opt/flowops/milvus.yaml # Embedded etcd configuration /opt/flowops/embedEtcd.yaml

Data Directories

# Separate 20GB EBS volume /milvus-data/milvus # Vector data /milvus-data/etcd # Metadata /milvus-data/logs # Application logs

Service Management

# Check status sudo systemctl status milvus # Restart service sudo systemctl restart milvus # View logs docker logs milvus-standalone -f # View all container logs cd /opt/flowops && docker compose logs -f
PortServiceDescription
19530Milvus APIgRPC and REST API endpoint
9091Health/MetricsHealth checks and Prometheus metrics
2379etcdMetadata storage (internal)

Working with Collections

Collections are like tables - they hold your vectors and metadata

Create a Collection

Define schema with vector field and metadata.

from pymilvus import ( connections, Collection, FieldSchema, CollectionSchema, DataType ) # Connect to Milvus connections.connect( host="localhost", port="19530", user="root", password="your-password" ) # Define fields fields = [ FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=True), FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=1536), FieldSchema(name="text", dtype=DataType.VARCHAR, max_length=65535), FieldSchema(name="metadata", dtype=DataType.JSON) ] # Create collection schema = CollectionSchema(fields, "My documents") collection = Collection("documents", schema)

Insert Vectors

Add data to your collection.

# Prepare data (1536-dim vectors) embeddings = [[0.1, 0.2, ...], [0.3, 0.4, ...]] texts = ["Hello world", "Milvus is great"] metadata = [ {"source": "doc1", "page": 1}, {"source": "doc2", "page": 1} ] # Insert data collection.insert([embeddings, texts, metadata]) collection.flush() # Persist to disk # Check row count print(f"Rows: {collection.num_entities}")

Search Vectors

Find similar vectors with optional filters.

# Load collection into memory collection.load() # Define search parameters search_params = { "metric_type": "COSINE", "params": {"ef": 64} } # Search for similar vectors results = collection.search( data=[query_embedding], anns_field="embedding", param=search_params, limit=10, output_fields=["text", "metadata"], expr='metadata["source"] == "doc1"' ) for hits in results: for hit in hits: print(f"{hit.distance}: {hit.entity.get('text')}")

Delete & Drop

Remove vectors or entire collections.

# Delete by expression collection.delete(expr='id in [1, 2, 3]') # Delete by filter collection.delete(expr='metadata["source"] == "old"') # Drop entire collection collection.drop() # List all collections from pymilvus import utility print(utility.list_collections())

Index Types

Choose the right index for your use case

IndexBest ForMemoryAccuracySpeed
HNSWGeneral purpose, high recallHighExcellentFast
IVF_FLATHigh accuracy, moderate scaleMediumExcellentMedium
IVF_SQ8Memory-constrained, large scaleLowGoodFast
IVF_PQBillion-scale, memory-limitedVery LowModerateVery Fast
FLATSmall datasets, 100% accuracyHighPerfectSlow
SCANNLarge scale, good recallMediumGoodFast

HNSW Index (Recommended)

Best balance of speed and accuracy for most use cases.

index_params = { "index_type": "HNSW", "metric_type": "COSINE", # or L2, IP "params": { "M": 16, # Connections (8-64) "efConstruction": 256 # Build quality } } collection.create_index("embedding", index_params) # Search params for HNSW search_params = { "metric_type": "COSINE", "params": {"ef": 64} # Higher = accurate }

IVF_FLAT Index

Good for exact search with clustering. Lower memory than HNSW.

index_params = { "index_type": "IVF_FLAT", "metric_type": "COSINE", "params": { "nlist": 1024 # Number of clusters } } collection.create_index("embedding", index_params) # Search params for IVF search_params = { "metric_type": "COSINE", "params": {"nprobe": 16} # Clusters to search }

Index Recommendations

  • <1M vectors: HNSW with M=16, efConstruction=256
  • 1M-100M vectors: IVF_FLAT with nlist=4096 or HNSW with M=32
  • 100M+ vectors: IVF_PQ for memory efficiency, or scale horizontally
  • Need 100% accuracy: FLAT (but only for <100K vectors)

Partitions

Organize data for faster filtered queries

Create Partitions

Split data by logical groups (e.g., by tenant, date).

# Create partitions collection.create_partition("tenant_a") collection.create_partition("tenant_b") collection.create_partition("2024_q1") # Insert into specific partition collection.insert(data, partition_name="tenant_a") # List partitions print(collection.partitions) # Check if partition exists collection.has_partition("tenant_a")

Search in Partitions

Query specific partitions for faster results.

# Load specific partitions only collection.load(partition_names=["tenant_a"]) # Search in partition only - Much faster! results = collection.search( data=[query_vector], anns_field="embedding", param=search_params, limit=10, partition_names=["tenant_a"] ) # Search across multiple partitions results = collection.search( data=[query_vector], anns_field="embedding", param=search_params, limit=10, partition_names=["tenant_a", "tenant_b"] )

When to Use Partitions

  • Multi-tenant applications (partition per customer)
  • Time-based data (partition per month/week)
  • Category-based filtering (partition per product type)
  • Load only "hot" data into memory

Python Integration

Complete examples with PyMilvus

Full RAG Example with OpenAI

import openai from pymilvus import connections, Collection, utility from pymilvus import FieldSchema, CollectionSchema, DataType # Connect to Milvus connections.connect( host="your-instance-ip", port="19530", user="root", password="your-root-password" # From credentials.txt ) def get_embedding(text): """Get embedding from OpenAI""" response = openai.embeddings.create( model="text-embedding-3-small", input=text ) return response.data[0].embedding def create_collection(): """Create a collection if it doesn't exist""" if utility.has_collection("documents"): return Collection("documents") fields = [ FieldSchema("id", DataType.INT64, is_primary=True, auto_id=True), FieldSchema("embedding", DataType.FLOAT_VECTOR, dim=1536), FieldSchema("text", DataType.VARCHAR, max_length=65535), FieldSchema("metadata", DataType.JSON) ] schema = CollectionSchema(fields, "RAG documents") collection = Collection("documents", schema) # Create HNSW index collection.create_index("embedding", { "index_type": "HNSW", "metric_type": "COSINE", "params": {"M": 16, "efConstruction": 256} }) return collection def store_documents(collection, documents): """Store documents with embeddings""" embeddings = [get_embedding(doc["text"]) for doc in documents] texts = [doc["text"] for doc in documents] metadata = [doc.get("metadata", {}) for doc in documents] collection.insert([embeddings, texts, metadata]) collection.flush() def search_similar(collection, query, limit=5): """Find similar documents""" collection.load() query_embedding = get_embedding(query) results = collection.search( data=[query_embedding], anns_field="embedding", param={"metric_type": "COSINE", "params": {"ef": 64}}, limit=limit, output_fields=["text", "metadata"] ) return [ { "text": hit.entity.get("text"), "score": 1 - hit.distance, "metadata": hit.entity.get("metadata") } for hit in results[0] ] # Example usage collection = create_collection() # Store documents docs = [ {"text": "Milvus is a vector database for AI", "metadata": {"source": "docs"}}, {"text": "RAG combines retrieval with generation", "metadata": {"source": "blog"}} ] store_documents(collection, docs) # Search results = search_similar(collection, "How do I build AI search?") for r in results: print(f"{r['score']:.3f}: {r['text']}")

LangChain

from langchain_milvus import Milvus from langchain_openai import OpenAIEmbeddings vectorstore = Milvus( embedding_function=OpenAIEmbeddings(), connection_args={ "host": "your-ip", "port": "19530" }, collection_name="docs" )

LlamaIndex

from llama_index.vector_stores.milvus import MilvusVectorStore vector_store = MilvusVectorStore( uri="http://your-ip:19530", dim=1536, collection_name="docs" )

Install PyMilvus

pip install pymilvus # With OpenAI pip install pymilvus openai # With LangChain pip install langchain-milvus

Authentication

Secure your Milvus instance

Default Credentials

Authentication is enabled by default on this AMI.

# Default user: root # Password: auto-generated on first boot # Location: /opt/flowops/credentials.txt # Connect with auth connections.connect( host="your-ip", port="19530", user="root", password="your-password" )

Create Additional Users

Add users with specific roles.

from pymilvus import utility # Create a new user utility.create_user( user="app_user", password="secure_password_here" ) # List users print(utility.list_users(include_role_info=True)) # Delete a user utility.delete_user(user="old_user")

Role-Based Access Control

Manage permissions with roles.

from pymilvus import Role # Create a role role = Role("read_only") role.create() # Grant collection permissions role.grant("Collection", "documents", "Search") role.grant("Collection", "documents", "Query") # Assign role to user role.add_user("app_user") # List roles print(utility.list_roles())

Change Root Password

Update the default password.

from pymilvus import utility # Change password (as root) utility.update_password( user="root", old_password="current-password", new_password="new-secure-password" ) # Reset another user's password utility.reset_password( user="app_user", old_password="old_pass", new_password="new_pass" )

Security Best Practices: Change the root password after first login. Create application-specific users with minimal required permissions. Never expose port 19530 to the public internet without authentication. Use security groups to restrict access to known IPs only.

Backups & Recovery

Protect your vector data

Setup S3 Backups

One-command setup for automated backups to Amazon S3.

# Run the setup script sudo /opt/flowops/setup-backups.sh # You'll be prompted for: # - S3 bucket name # - AWS region # - Backup schedule (cron format) # The script configures: # - IAM role permissions (if attached) # - Automated daily backups # - 7-day retention policy

Manual Backup

Create a backup on demand.

# Flush all collections first from pymilvus import utility for name in utility.list_collections(): Collection(name).flush() # Stop Milvus sudo systemctl stop milvus # Backup data directory to S3 aws s3 sync /milvus-data s3://my-bucket/milvus-backup/$(date +%Y%m%d)/ # Start Milvus sudo systemctl start milvus

Restore from Backup

Recover data from an S3 backup.

# Stop Milvus sudo systemctl stop milvus # Clear existing data sudo rm -rf /milvus-data/milvus/* sudo rm -rf /milvus-data/etcd/* # Restore from S3 aws s3 sync s3://my-bucket/milvus-backup/20240115/ /milvus-data/ # Start Milvus sudo systemctl start milvus

EBS Snapshots

Use AWS EBS snapshots for point-in-time recovery.

# Find the data volume ID lsblk # Look for /milvus-data mount # Create snapshot via AWS CLI aws ec2 create-snapshot \ --volume-id vol-0123456789abcdef \ --description "Milvus backup $(date)" # Or use AWS Console: # EC2 > Volumes > Actions > Create Snapshot

IAM Role Required for S3

  • Attach an IAM role to your EC2 instance
  • Required permissions: s3:PutObject, s3:GetObject, s3:ListBucket, s3:DeleteObject
  • Enable S3 versioning for additional protection

Monitoring & Metrics

Track performance and health

Health Check

curl http://localhost:9091/healthz # Response when healthy: # {"status":"ok"} # Use in load balancer: # Health check path: /healthz # Port: 9091 # Start period: 120s

Prometheus Metrics

curl http://localhost:9091/metrics # Add to prometheus.yml: scrape_configs: - job_name: 'milvus' static_configs: - targets: ['your-ip:9091']

Container Logs

# Follow real-time logs docker logs milvus-standalone -f # Last 100 lines docker logs milvus-standalone --tail 100 # All service logs cd /opt/flowops docker compose logs -f

Key Metrics to Watch

  • milvus_datanode_flush_duration_seconds - Data persistence latency
  • milvus_querynode_search_latency_seconds - Search response times
  • milvus_datacoord_stored_rows_count - Total vectors stored
  • milvus_querynode_collection_loaded_count - Collections in memory
  • process_resident_memory_bytes - Memory usage

CloudWatch Integration

# Install CloudWatch agent sudo /opt/flowops/setup-cloudwatch.sh # Metrics sent to CloudWatch: # - CPU, Memory, Disk usage # - Container metrics # - Custom Milvus metrics # Set up alarms in AWS Console: # - Memory > 80% # - Disk > 85% # - Health check failures

System Resources

# Memory usage free -h # Disk usage df -h /milvus-data # CPU and process info htop # Docker stats docker stats --no-stream

Performance Tuning

Get the most out of your instance

Instance TypeRAMRecommended ForMax Vectors (1536-dim)
m5.large8GBDevelopment, testing~500K
m5.xlarge16GBSmall production~2M
r5.xlarge32GBProduction~5M
r5.2xlarge64GBLarge scale~15M
r5.4xlarge128GBEnterprise~40M

Search Optimization

# Tune ef for HNSW search # Higher ef = more accurate, slower search_params = { "metric_type": "COSINE", "params": { "ef": 64 # Default, good balance # ef: 128 for 99%+ recall # ef: 32 for faster search } } # For IVF indexes, tune nprobe search_params = { "metric_type": "COSINE", "params": { "nprobe": 16 # More = accurate } }

Memory Management

# Release collection from memory collection.release() # Load only needed partitions collection.load(partition_names=["active"]) # Check what's loaded print(utility.loading_progress("documents")) # Compact segments periodically # Reduces memory fragmentation collection.compact() # Wait for compaction collection.wait_for_compaction_completed()

Batch Operations

# Insert in batches for better perf BATCH_SIZE = 1000 for i in range(0, len(data), BATCH_SIZE): batch = data[i:i+BATCH_SIZE] collection.insert(batch) # Flush after all inserts collection.flush() # Batch search - multiple queries results = collection.search( data=[query1, query2, query3], anns_field="embedding", param=search_params, limit=10 )

Index Tuning

# Check current index print(collection.indexes) # Drop and rebuild with new params collection.release() collection.drop_index() # Create optimized index collection.create_index("embedding", { "index_type": "HNSW", "metric_type": "COSINE", "params": { "M": 32, # More connections "efConstruction": 512 } })

SIMD Required: Milvus requires SSE4.2/AVX/AVX2 CPU instructions for optimal performance. Use m5/m6i/r5/r6i instance families. T2/T3 instances work but are slower. T2 instances are NOT recommended for production.

Troubleshooting

Common issues and solutions

Milvus Won't Start

# Check container status docker ps -a # View startup logs docker logs milvus-standalone # Check disk space df -h /milvus-data # Check memory free -h # Restart the service sudo systemctl restart milvus

Connection Refused

# Wait 90 seconds for startup sleep 90 # Check if port is listening netstat -tlnp | grep 19530 # Test locally curl http://localhost:19530/v1/vector/collections # Check security group allows 19530 # from your IP address # Check container is running docker ps | grep milvus

Out of Memory

# Check memory usage free -h docker stats --no-stream # Release unused collections collection.release() # Use smaller ef for search search_params = {"params": {"ef": 32}} # Solutions: # 1. Upgrade to r5.xlarge or larger # 2. Use IVF_SQ8 index (less memory) # 3. Use partitions, load only needed

Slow Queries

# Check index exists print(collection.indexes) # Ensure collection is loaded collection.load() print(utility.loading_progress("documents")) # Lower ef for faster search param = {"params": {"ef": 32}} # Use partitions for filtered queries results = collection.search( ..., partition_names=["relevant"] ) # Compact to optimize segments collection.compact()

etcd Errors

# etcd stores Milvus metadata # Check etcd logs docker logs milvus-etcd # Check etcd data directory ls -la /milvus-data/etcd/ # If etcd is corrupted, restore # from backup or rebuild # Clear etcd (CAUTION: loses data) sudo systemctl stop milvus sudo rm -rf /milvus-data/etcd/* sudo systemctl start milvus

Authentication Errors

# Check credentials file sudo cat /opt/flowops/credentials.txt # Test connection connections.connect( host="localhost", port="19530", user="root", password="your-password" ) # Reset password if needed: # Edit /opt/flowops/milvus.yaml # Set authorizationEnabled: false # Restart, change password, re-enable

Need Help?

Our UK-based support team is available 24/7 with guaranteed response within 24 hours

support@flowopsconsulting.co.uk