Cache Management

Manage Redis/Valkey cache for optimal performance

The Bovine Pages Server utilizes Redis or Valkey for caching to enhance performance and facilitate advanced features. The cache stores file content, custom domain mappings, password hashes, directory listings, and Traefik router configurations.

Cache Key Structure

File Content Cache

username:repository:filepath

Examples:

squarecows:bovine-website:public/index.html
john:my-blog:public/assets/style.css
alice:docs:public/api/v1/reference.html

Custom Domain Mappings

Forward Mapping:

custom_domain:bovine.squarecows.com
custom_domain:www.example.com

Reverse Mapping:

squarecows:bovine-website
john:personal-site

Password Hash Cache

password_hash:username:repository

Examples:

password_hash:squarecows:private-docs
password_hash:alice:team-wiki

Traefik Router Configurations

traefik/http/routers/{sanitized-domain}/{property}

Examples:

traefik/http/routers/custom-bovine-squarecows-com/entrypoints
traefik/http/routers/custom-bovine-squarecows-com/rule
traefik/http/routers/custom-bovine-squarecows-com/service
traefik/http/routers/custom-bovine-squarecows-com/middlewares
traefik/http/routers/custom-bovine-squarecows-com/tls/certresolver
traefik/http/routers/custom-bovine-squarecows-com/priority

Accessing Redis/Valkey CLI

Docker/Podman Setup

# List all containers
docker ps

# Find Redis/Valkey container
docker ps | grep -E "redis|valkey"

Connection Methods

Method 1: Direct Command Execution

# Redis
docker exec <container-name> redis-cli <command>

# Valkey
docker exec <container-name> valkey-cli <command>

Method 2: Interactive Shell

# Redis
docker exec -it <container-name> redis-cli

# Valkey
docker exec -it <container-name> valkey-cli

Method 3: Container Shell + CLI

# Enter container
docker exec -it <container-name> sh

# Then run CLI commands
redis-cli
# or
valkey-cli

With Password Authentication

# Add -a flag
docker exec <container-name> redis-cli -a your-password <command>

# Or in interactive mode, authenticate after connecting
docker exec -it <container-name> redis-cli
127.0.0.1:6379> AUTH your-password
OK

Inspecting Cache Contents

View All Keys

# Redis
docker exec redis redis-cli KEYS "*"

# Valkey
docker exec valkey valkey-cli KEYS "*"

# Scan all keys (safer than KEYS)
docker exec redis redis-cli --scan

# Scan with pattern
docker exec redis redis-cli --scan --pattern "*bovine*"

# Count total keys
docker exec redis redis-cli DBSIZE

View Specific Key Value

# Get a single key
docker exec redis redis-cli GET "custom_domain:bovine.squarecows.com"

# Example output: squarecows:bovine-website

Check Key Type and TTL

# Check key type
docker exec redis redis-cli TYPE "custom_domain:bovine.squarecows.com"

# Check time-to-live (-1 = no expiration, -2 = doesn't exist)
docker exec redis redis-cli TTL "custom_domain:bovine.squarecows.com"

# Check if key exists
docker exec redis redis-cli EXISTS "custom_domain:bovine.squarecows.com"

Search for Keys by Pattern

# Find all keys for a specific user
docker exec redis redis-cli --scan --pattern "squarecows:*"

# Find all custom domains
docker exec redis redis-cli --scan --pattern "custom_domain:*"

# Find all password hashes
docker exec redis redis-cli --scan --pattern "password_hash:*"

# Find all Traefik router configs
docker exec redis redis-cli --scan --pattern "traefik/http/routers/*"

Flushing Cache

Flush Cache for Specific Website

Custom Domain Site (e.g., bovine.squarecows.com)

Step 1: Find what repository the domain maps to

docker exec redis redis-cli GET custom_domain:bovine.squarecows.com
# Output: squarecows:bovine-website

Step 2: Delete all cached content

# Redis
docker exec redis sh -c "redis-cli --scan --pattern 'squarecows:bovine-website:*' | xargs -r redis-cli DEL"

# Valkey
docker exec valkey sh -c "valkey-cli --scan --pattern 'squarecows:bovine-website:*' | xargs -r valkey-cli DEL"

Step 3: Delete custom domain mappings

# Redis
docker exec redis redis-cli DEL custom_domain:bovine.squarecows.com
docker exec redis redis-cli DEL squarecows:bovine-website

# Valkey
docker exec valkey valkey-cli DEL custom_domain:bovine.squarecows.com
docker exec valkey valkey-cli DEL squarecows:bovine-website

Step 4: Delete Traefik router configurations

# Redis (sanitized domain: bovine.squarecows.com → bovine-squarecows-com)
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/entrypoints
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/rule
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/service
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/middlewares
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/tls/certresolver
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/priority

# Valkey (same commands, just replace redis-cli with valkey-cli)

Quick Method: Delete Everything Related to Domain

# Redis
docker exec redis sh -c "redis-cli --scan --pattern '*bovine.squarecows.com*' | xargs -r redis-cli DEL"
docker exec redis sh -c "redis-cli --scan --pattern '*bovine-squarecows-com*' | xargs -r redis-cli DEL"

# Valkey
docker exec valkey sh -c "valkey-cli --scan --pattern '*bovine.squarecows.com*' | xargs -r valkey-cli DEL"
docker exec valkey sh -c "valkey-cli --scan --pattern '*bovine-squarecows-com*' | xargs -r valkey-cli DEL"

Pages Domain Site (e.g., john.pages.example.com/blog)

Flush all cached content for a repository:

# Redis
docker exec redis sh -c "redis-cli --scan --pattern 'john:blog:*' | xargs -r redis-cli DEL"

# Valkey
docker exec valkey sh -c "valkey-cli --scan --pattern 'john:blog:*' | xargs -r valkey-cli DEL"

Delete password hash (if password protected):

# Redis
docker exec redis redis-cli DEL password_hash:john:blog

# Valkey
docker exec valkey valkey-cli DEL password_hash:john:blog

Flush Cache for Specific User

# Redis
docker exec redis sh -c "redis-cli --scan --pattern 'john:*' | xargs -r redis-cli DEL"

# Valkey
docker exec valkey sh -c "valkey-cli --scan --pattern 'john:*' | xargs -r valkey-cli DEL"

Flush Entire Cache

# Redis
docker exec redis redis-cli FLUSHDB

# Valkey
docker exec valkey valkey-cli FLUSHDB

# Flush all databases (if using multiple)
docker exec redis redis-cli FLUSHALL

Common Cache Management Tasks

Invalidate Cache After Content Update

# Delete all cached files for the repository
docker exec redis sh -c "redis-cli --scan --pattern 'username:repository:*' | xargs -r redis-cli DEL"

# The next request will fetch fresh content from Forgejo

Remove Stale Custom Domain Mapping

# 1. Find current mapping
docker exec redis redis-cli GET custom_domain:old-domain.com

# 2. Delete forward mapping
docker exec redis redis-cli DEL custom_domain:old-domain.com

# 3. Delete reverse mapping (using output from step 1)
docker exec redis redis-cli DEL username:repository

# 4. Delete Traefik router configs
docker exec redis sh -c "redis-cli --scan --pattern 'traefik/http/routers/custom-old-domain-com/*' | xargs -r redis-cli DEL"

Clear Password Cache

# Redis
docker exec redis redis-cli DEL password_hash:username:repository

# Valkey
docker exec valkey valkey-cli DEL password_hash:username:repository

Monitor Cache Activity

# Watch commands in real-time
docker exec redis redis-cli MONITOR

# Get cache statistics
docker exec redis redis-cli INFO stats

# Check memory usage
docker exec redis redis-cli INFO memory

Automated Cache Management

Using the Cache Reaper Script

cd reaper/
python3 reaper.py --forgejo-host https://git.example.com \
                  --redis-host localhost \
                  --redis-port 6379 \
                  --dry-run

Scheduled Cache Cleanup

# Edit crontab
crontab -e

# Run reaper daily at 2 AM
0 2 * * * cd /path/to/reaper && python3 reaper.py --forgejo-host https://git.example.com --redis-host localhost

Troubleshooting

Cache Not Working

Check if Redis/Valkey is running:

docker ps | grep -E "redis|valkey"
docker exec redis redis-cli PING
# Expected: PONG

Check Traefik can connect to Redis:

# Check Traefik logs for Redis connection errors
docker logs traefik | grep -i redis

Verify cache configuration in middleware:

  • Check enableRedisCache: true in Traefik middleware config
  • Verify redisAddress points to correct container/host
  • Test Redis connectivity: docker exec traefik ping redis

Custom Domain Not Working

Check if domain mapping exists:

docker exec redis redis-cli GET custom_domain:yourdomain.com
# Should return: username:repository

Check if Traefik router exists:

docker exec redis redis-cli --scan --pattern "traefik/http/routers/custom-yourdomain-com/*"
# Should return 6 keys

Re-register domain:

  1. Delete existing mappings (see above)
  2. Visit the pages URL: https://username.pages.example.com/repository/
  3. Check cache again - mappings should be recreated

High Memory Usage

Check cache size:

docker exec redis redis-cli INFO memory | grep used_memory_human
docker exec redis redis-cli DBSIZE

Find largest keys:

docker exec redis redis-cli --bigkeys

# Or use redis-cli memory doctor
docker exec redis redis-cli --memkeys

Reduce cache size:

  • Lower cache TTL in middleware configuration
  • Run cache reaper more frequently
  • Flush unused cached content
  • Consider increasing maxmemory and enabling eviction policy

Authentication Issues

Test authentication:

# Without password
docker exec redis redis-cli PING
# Error: NOAUTH Authentication required

# With password
docker exec redis redis-cli -a your-password PING
# PONG

Update Traefik middleware configuration:

redisPassword: your-redis-password

Best Practices

Production Recommendations

  1. Enable Redis/Valkey - Don’t rely on in-memory cache in production
  2. Set Appropriate TTLs - Balance freshness vs performance (default 300s is good)
  3. Monitor Memory - Set up alerts for high memory usage
  4. Regular Cleanup - Run cache reaper script daily or weekly
  5. Backup Strategy - Redis persistence (RDB/AOF) for critical mappings
  6. Security - Use password authentication for Redis/Valkey

Cache Invalidation Strategy

When to manually invalidate:

  • After repository content changes
  • After .pages file modifications
  • After password changes
  • When custom domain configuration changes

Automatic invalidation:

  • File cache expires after TTL (default 300s)
  • Directory listings expire after 60s
  • Password hashes expire after 60s

Performance Tuning

Optimize cache hit rate:

  • Increase TTL for rarely-changing content
  • Pre-warm cache for popular pages
  • Monitor cache hit/miss ratio

Optimize memory usage:

  • Set maxmemory limit in Redis
  • Enable eviction policy (e.g., allkeys-lru)
  • Compress large cached values

Reference Commands

Quick Reference Card

# === INSPECTION ===
# List all keys
docker exec redis redis-cli KEYS "*"

# Scan keys (production-safe)
docker exec redis redis-cli --scan

# Get key value
docker exec redis redis-cli GET key-name

# Check if key exists
docker exec redis redis-cli EXISTS key-name

# === DELETION ===
# Delete single key
docker exec redis redis-cli DEL key-name

# Delete by pattern
docker exec redis sh -c "redis-cli --scan --pattern 'pattern:*' | xargs -r redis-cli DEL"

# Flush entire cache
docker exec redis redis-cli FLUSHDB

# === MONITORING ===
# Real-time monitoring
docker exec redis redis-cli MONITOR

# Cache statistics
docker exec redis redis-cli INFO stats

# Memory usage
docker exec redis redis-cli INFO memory

# === MAINTENANCE ===
# Count keys
docker exec redis redis-cli DBSIZE

# Find large keys
docker exec redis redis-cli --bigkeys

# Test connection
docker exec redis redis-cli PING

Additional Resources