Good Guidance

Power Management

Understanding Power Modes

The Jetson Orin Nano supports multiple power modes:

# List available modes
sudo nvpmodel -q

# Mode 0: MAXN (Maximum Performance) - ~15W
sudo nvpmodel -m 0

# Mode 1: 10W Mode (Power Saving)
sudo nvpmodel -m 1

# Mode 2: 15W Mode (Balanced)
sudo nvpmodel -m 2

When to Use Each Mode

MAXN Mode (0): Development, training, real-time inference
15W Mode (2): Balanced workloads, extended operation
10W Mode (1): Battery-powered, low-power applications

Power Optimization Tips

Use appropriate power mode for your workload
Enable jetson_clocks for maximum performance when needed:
```
sudo jetson_clocks
```
Monitor power consumption:
```
sudo tegrastats
```
Disable unused peripherals to save power
Use sleep/suspend when idle

Performance Optimization

GPU Acceleration Best Practices

Use TensorRT for inference
- Converts models to optimized format
- Significantly faster than PyTorch/TensorFlow alone

Enable mixed precision

# Use FP16 when possible
model = model.half()

Batch processing
- Process multiple inputs together
- Better GPU utilization
Pipeline processing
- Overlap data loading and inference
- Use multiple threads/processes

Critical: The Jetson Orin Nano Field Kit has 8GB total RAM, with only ~6GB practically available for applications. Overloading RAM can cause system instability, OOM (Out of Memory) errors, and crashes.

Monitor memory usage continuously

# Watch memory usage in real-time
watch -n 1 free -h

# Check detailed memory breakdown
cat /proc/meminfo

# Monitor with tegrastats (includes GPU memory)
sudo tegrastats

Set memory limits for applications
- LLMs: Maximum 3-4GB per model
- Vision models: Monitor GPU and system RAM usage
- Multiple services: Ensure total usage stays under 6GB
- Leave at least 1-2GB free for system operations
Use model quantization
- 4-bit or 8-bit quantization reduces memory footprint significantly
- Essential for running models on the Field Kit

Clear GPU cache regularly

import torch
import gc

# Clear GPU cache
torch.cuda.empty_cache()
# Force garbage collection
gc.collect()

Use generators for large datasets
- Don't load entire dataset into memory
- Process in batches
- Stream data when possible
Avoid running multiple memory-intensive services simultaneously
- Don't run multiple LLMs at once
- Stop unused services before starting new ones
- Use docker stats to monitor container memory usage

CPU Optimization

Set CPU governor to performance
```
sudo cpufreq-set -g performance
```
Pin processes to specific cores
```
taskset -c 0,1 python3 your_script.py
```
Use appropriate number of threads
- Too many threads can hurt performance
- Monitor CPU usage to find optimal count

Development Best Practices

Code Organization

# Good: Modular structure
# models/
#   __init__.py
#   vision.py
#   audio.py
# utils/
#   __init__.py
#   preprocessing.py
#   postprocessing.py
# main.py

# Bad: Everything in one file
# main.py (1000+ lines)

Error Handling

# Good: Proper error handling
try:
    result = process_image(image)
except ValueError as e:
    logger.error(f"Invalid image format: {e}")
    return None
except RuntimeError as e:
    logger.error(f"Processing failed: {e}")
    return None

# Bad: Silent failures
result = process_image(image)  # What if this fails?

Logging

import logging

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('app.log'),
        logging.StreamHandler()
    ]
)

logger = logging.getLogger(__name__)

# Use appropriate log levels
logger.debug("Detailed debugging information")
logger.info("General information")
logger.warning("Warning message")
logger.error("Error occurred")
logger.critical("Critical error")

Configuration Management

# Good: Use configuration files
import json
from dataclasses import dataclass

@dataclass
class Config:
    model_path: str
    batch_size: int
    device: str
    
    @classmethod
    def from_file(cls, path):
        with open(path) as f:
            data = json.load(f)
        return cls(**data)

config = Config.from_file('config.json')

# Bad: Hardcoded values
model_path = "/path/to/model"  # What if path changes?
batch_size = 32  # What if we need different batch sizes?

Security Best Practices

System Security

Change default passwords
```
passwd
```
Keep system updated
```
sudo apt update
sudo apt upgrade
```

Use SSH keys instead of passwords

# On your computer
ssh-keygen -t ed25519
ssh-copy-id user@jetson-ip

Disable unnecessary services

sudo systemctl disable <unnecessary-service>

Configure firewall

sudo ufw enable
sudo ufw allow ssh
sudo ufw allow 5000/tcp  # If running web services

Application Security

Validate all inputs
Use parameterized queries (if using databases)
Sanitize user data before processing
Use HTTPS for network communication
Store secrets securely (use environment variables, not hardcoded)

Deployment Best Practices

Production Checklist

Monitoring

# Monitor system resources
import psutil
import time

def monitor_system():
    while True:
        cpu_percent = psutil.cpu_percent(interval=1)
        memory = psutil.virtual_memory()
        disk = psutil.disk_usage('/')
        
        print(f"CPU: {cpu_percent}%")
        print(f"Memory: {memory.percent}%")
        print(f"Disk: {disk.percent}%")
        
        # Alert if thresholds exceeded
        if cpu_percent > 90:
            logger.warning("High CPU usage!")
        if memory.percent > 80:
            logger.warning("High memory usage! Consider freeing resources.")
        if memory.percent > 90:
            logger.critical("CRITICAL: Memory usage exceeds 90%! System may become unstable.")
        if memory.available < 1 * (1024**3):  # Less than 1GB free
            logger.critical("CRITICAL: Less than 1GB RAM available! Stop unnecessary processes immediately.")
        
        time.sleep(60)  # Check every minute

Health Checks

# Implement health check endpoint
from flask import Flask, jsonify

app = Flask(__name__)

@app.route('/health')
def health_check():
    # Check system resources
    memory = psutil.virtual_memory()
    disk = psutil.disk_usage('/')
    
    health = {
        'status': 'healthy',
        'memory_percent': memory.percent,
        'disk_percent': disk.percent,
        'timestamp': time.time()
    }
    
    # Mark as unhealthy if resources are low
    # Use stricter thresholds for RAM (80% warning, 90% critical)
    if memory.percent > 90 or disk.percent > 95:
        health['status'] = 'unhealthy'
        health['warning'] = 'Memory usage critical - stop unnecessary processes'
        return jsonify(health), 503
    elif memory.percent > 80:
        health['status'] = 'degraded'
        health['warning'] = 'Memory usage high - monitor closely'
        return jsonify(health), 200
    
    return jsonify(health), 200

Model Optimization

Model Selection

Choose appropriate model size
- Larger models = better accuracy but slower
- Smaller models = faster but may be less accurate
- Test multiple models to find best trade-off
Use quantized models
- 4-bit quantization: ~4x smaller, minimal accuracy loss
- 8-bit quantization: ~2x smaller, very minimal accuracy loss
Consider specialized models
- Use domain-specific models when available
- Fine-tune general models for your use case

Inference Optimization

Use TensorRT

# Convert model to TensorRT
# (implementation depends on framework)

Batch inference

# Process multiple inputs together
results = model(batch_inputs)

Async processing

import asyncio

async def process_async(inputs):
    # Process asynchronously
    return await process(inputs)

Resource Management

RAM Overload Prevention

Preventing RAM overload is critical on the Field Kit. Follow these practices:

Monitor RAM usage before and during operations

import psutil

def check_memory():
    memory = psutil.virtual_memory()
    print(f"Total: {memory.total / (1024**3):.2f} GB")
    print(f"Available: {memory.available / (1024**3):.2f} GB")
    print(f"Used: {memory.percent}%")
    
    # Warn if memory usage is high
    if memory.percent > 80:
        print("WARNING: Memory usage is high!")
    if memory.available < 1 * (1024**3):  # Less than 1GB free
        print("CRITICAL: Less than 1GB RAM available!")
        return False
    return True

# Check before loading large models
if not check_memory():
    print("Cannot proceed - insufficient RAM")
    exit(1)

Set memory limits for Docker containers

# docker-compose.yml
services:
  my-service:
    mem_limit: 3g  # Limit container to 3GB
    mem_reservation: 2g  # Reserve 2GB

Unload models when not in use

# Good: Unload model when done
model = load_model()
result = model.infer(input)
del model  # Free memory
import gc
gc.collect()

# Bad: Keep model loaded indefinitely
model = load_model()
# Model stays in memory even when not used

Use context managers for resource cleanup

class ModelContext:
    def __init__(self, model_path):
        self.model_path = model_path
        self.model = None
    
    def __enter__(self):
        self.model = load_model(self.model_path)
        return self.model
    
    def __exit__(self, *args):
        del self.model
        import gc
        gc.collect()

# Usage: Model automatically unloaded when done
with ModelContext('model.pth') as model:
    result = model.infer(input)
# Model is freed here

Monitor and limit concurrent processes

# Check memory usage per process
ps aux --sort=-%mem | head -10

# Kill processes using too much memory if needed
kill -9 <PID>

Memory Management

# Good: Clear references when done
def process_large_dataset():
    for batch in dataset:
        result = process(batch)
        yield result
        # Batch is automatically garbage collected

# Bad: Keep all results in memory
def process_large_dataset():
    results = []
    for batch in dataset:
        results.append(process(batch))
    return results  # All results kept in memory - can cause OOM!

File Handling

# Good: Use context managers
with open('file.txt', 'r') as f:
    data = f.read()
# File automatically closed

# Bad: Manual file handling
f = open('file.txt', 'r')
data = f.read()
# Forgot to close file!

Testing

Unit Testing

import unittest

class TestImageProcessing(unittest.TestCase):
    def test_resize(self):
        image = create_test_image()
        result = resize_image(image, (224, 224))
        self.assertEqual(result.shape, (224, 224, 3))
    
    def test_invalid_input(self):
        with self.assertRaises(ValueError):
            resize_image(None, (224, 224))

if __name__ == '__main__':
    unittest.main()

Integration Testing

def test_camera_pipeline():
    # Test entire camera processing pipeline
    cap = cv2.VideoCapture(0)
    frame = capture_frame(cap)
    processed = process_frame(frame)
    assert processed is not None
    cap.release()

Documentation

Code Documentation

def process_image(image, resize_to=(224, 224)):
    """
    Process an image for model inference.
    
    Args:
        image: Input image as numpy array (H, W, C)
        resize_to: Target size as (width, height) tuple
    
    Returns:
        Processed image as numpy array
    
    Raises:
        ValueError: If image is None or invalid format
    
    Example:
        >>> img = cv2.imread('test.jpg')
        >>> processed = process_image(img)
        >>> print(processed.shape)
        (224, 224, 3)
    """
    if image is None:
        raise ValueError("Image cannot be None")
    
    # Implementation...
    return processed_image

Version Control

Git Best Practices

Use meaningful commit messages

git commit -m "Add camera calibration support"
# Not: git commit -m "fix"

Keep commits focused
- One logical change per commit
- Don't mix unrelated changes

Use branches for features

git checkout -b feature/new-model
# Work on feature
git checkout main
git merge feature/new-model

Don't commit sensitive data
- Use .gitignore for secrets
- Use environment variables for configuration

Next Steps

Apply these practices to your projects
Review Troubleshooting when issues arise
Explore advanced topics in:

On this page