Computer Vision AI: Building Image Recognition Systems

Published on December 15, 2024 • 16 min read

Computer vision is transforming industries from healthcare to autonomous vehicles. This comprehensive guide teaches you how to build production-ready image recognition systems using modern AI frameworks and techniques.

Table of Contents

1. Introduction to Computer Vision
2. Setting Up Your Development Environment
3. Image Classification with CNNs
4. Object Detection Systems
5. Real-Time Processing
6. Production Deployment
7. Advanced Techniques
8. Best Practices

Introduction to Computer Vision {#introduction}

Computer vision enables machines to interpret and understand visual information from the world. Modern deep learning techniques have revolutionized this field, achieving human-level performance in many tasks.

Key Applications

• Medical Imaging: Disease detection and diagnosis
• Autonomous Vehicles: Object detection and navigation
• Manufacturing: Quality control and defect detection
• Security: Facial recognition and surveillance
• Retail: Product recognition and inventory management

Setting Up Your Development Environment {#setup}

Prerequisites

1pip install tensorflow opencv-python pillow numpy matplotlib
2pip install torch torchvision torchaudio
3pip install ultralytics roboflow supervision

Basic Image Processing

1import cv2
2import numpy as np
3import matplotlib.pyplot as plt
4from PIL import Image
5import tensorflow as tf
6
7class ImageProcessor:
8    def __init__(self):
9        self.supported_formats = ['.jpg', '.jpeg', '.png', '.bmp', '.tiff']
10    
11    def load_image(self, image_path):
12        """Load and preprocess image"""
13        image = cv2.imread(image_path)
14        if image is None:
15            raise ValueError(f"Could not load image: {image_path}")
16        
17        # Convert BGR to RGB
18        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
19        return image
20    
21    def resize_image(self, image, target_size=(224, 224)):
22        """Resize image while maintaining aspect ratio"""
23        return cv2.resize(image, target_size, interpolation=cv2.INTER_AREA)
24    
25    def normalize_image(self, image):
26        """Normalize pixel values to [0, 1]"""
27        return image.astype(np.float32) / 255.0
28    
29    def augment_image(self, image):
30        """Apply data augmentation"""
31        # Random rotation
32        angle = np.random.uniform(-15, 15)
33        center = (image.shape[1]//2, image.shape[0]//2)
34        rotation_matrix = cv2.getRotationMatrix2D(center, angle, 1.0)
35        rotated = cv2.warpAffine(image, rotation_matrix, (image.shape[1], image.shape[0]))
36        
37        # Random brightness adjustment
38        brightness = np.random.uniform(0.8, 1.2)
39        brightened = np.clip(rotated * brightness, 0, 255).astype(np.uint8)
40        
41        return brightened
42
43# Usage example
44processor = ImageProcessor()
45image = processor.load_image('sample.jpg')
46processed = processor.resize_image(image)
47normalized = processor.normalize_image(processed)

Image Classification with CNNs {#classification}

Building a Custom CNN

1import tensorflow as tf
2from tensorflow.keras import layers, models
3from tensorflow.keras.preprocessing.image import ImageDataGenerator
4
5class ImageClassifier:
6    def __init__(self, num_classes, input_shape=(224, 224, 3)):
7        self.num_classes = num_classes
8        self.input_shape = input_shape
9        self.model = self.build_model()
10    
11    def build_model(self):
12        """Build CNN architecture"""
13        model = models.Sequential([
14            # First convolutional block
15            layers.Conv2D(32, (3, 3), activation='relu', input_shape=self.input_shape),
16            layers.BatchNormalization(),
17            layers.MaxPooling2D((2, 2)),
18            layers.Dropout(0.25),
19            
20            # Second convolutional block
21            layers.Conv2D(64, (3, 3), activation='relu'),
22            layers.BatchNormalization(),
23            layers.MaxPooling2D((2, 2)),
24            layers.Dropout(0.25),
25            
26            # Third convolutional block
27            layers.Conv2D(128, (3, 3), activation='relu'),
28            layers.BatchNormalization(),
29            layers.MaxPooling2D((2, 2)),
30            layers.Dropout(0.25),
31            
32            # Fourth convolutional block
33            layers.Conv2D(256, (3, 3), activation='relu'),
34            layers.BatchNormalization(),
35            layers.MaxPooling2D((2, 2)),
36            layers.Dropout(0.25),
37            
38            # Classifier
39            layers.Flatten(),
40            layers.Dense(512, activation='relu'),
41            layers.BatchNormalization(),
42            layers.Dropout(0.5),
43            layers.Dense(self.num_classes, activation='softmax')
44        ])
45        
46        return model
47    
48    def compile_model(self, learning_rate=0.001):
49        """Compile model with optimizer and loss function"""
50        self.model.compile(
51            optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate),
52            loss='categorical_crossentropy',
53            metrics=['accuracy', 'top_5_accuracy']
54        )
55    
56    def create_data_generators(self, train_dir, val_dir, batch_size=32):
57        """Create data generators with augmentation"""
58        train_datagen = ImageDataGenerator(
59            rescale=1./255,
60            rotation_range=20,
61            width_shift_range=0.2,
62            height_shift_range=0.2,
63            horizontal_flip=True,
64            zoom_range=0.2,
65            shear_range=0.2,
66            fill_mode='nearest'
67        )
68        
69        val_datagen = ImageDataGenerator(rescale=1./255)
70        
71        train_generator = train_datagen.flow_from_directory(
72            train_dir,
73            target_size=self.input_shape[:2],
74            batch_size=batch_size,
75            class_mode='categorical'
76        )
77        
78        val_generator = val_datagen.flow_from_directory(
79            val_dir,
80            target_size=self.input_shape[:2],
81            batch_size=batch_size,
82            class_mode='categorical'
83        )
84        
85        return train_generator, val_generator
86    
87    def train(self, train_generator, val_generator, epochs=50):
88        """Train the model"""
89        callbacks = [
90            tf.keras.callbacks.EarlyStopping(
91                monitor='val_loss',
92                patience=10,
93                restore_best_weights=True
94            ),
95            tf.keras.callbacks.ReduceLROnPlateau(
96                monitor='val_loss',
97                factor=0.2,
98                patience=5,
99                min_lr=1e-7
100            ),
101            tf.keras.callbacks.ModelCheckpoint(
102                'best_model.h5',
103                monitor='val_accuracy',
104                save_best_only=True
105            )
106        ]
107        
108        history = self.model.fit(
109            train_generator,
110            epochs=epochs,
111            validation_data=val_generator,
112            callbacks=callbacks
113        )
114        
115        return history
116
117# Usage example
118classifier = ImageClassifier(num_classes=10)
119classifier.compile_model()
120
121# Create data generators
122train_gen, val_gen = classifier.create_data_generators(
123    'data/train', 
124    'data/validation'
125)
126
127# Train model
128history = classifier.train(train_gen, val_gen, epochs=100)

Transfer Learning with Pre-trained Models

1from tensorflow.keras.applications import ResNet50, VGG16, InceptionV3
2from tensorflow.keras.applications.resnet50 import preprocess_input
3
4class TransferLearningClassifier:
5    def __init__(self, num_classes, base_model_name='resnet50'):
6        self.num_classes = num_classes
7        self.base_model_name = base_model_name
8        self.model = self.build_transfer_model()
9    
10    def build_transfer_model(self):
11        """Build model using transfer learning"""
12        # Load pre-trained base model
13        if self.base_model_name == 'resnet50':
14            base_model = ResNet50(
15                weights='imagenet',
16                include_top=False,
17                input_shape=(224, 224, 3)
18            )
19        elif self.base_model_name == 'vgg16':
20            base_model = VGG16(
21                weights='imagenet',
22                include_top=False,
23                input_shape=(224, 224, 3)
24            )
25        elif self.base_model_name == 'inception':
26            base_model = InceptionV3(
27                weights='imagenet',
28                include_top=False,
29                input_shape=(224, 224, 3)
30            )
31        
32        # Freeze base model layers
33        base_model.trainable = False
34        
35        # Add custom classifier
36        model = models.Sequential([
37            base_model,
38            layers.GlobalAveragePooling2D(),
39            layers.BatchNormalization(),
40            layers.Dropout(0.5),
41            layers.Dense(512, activation='relu'),
42            layers.BatchNormalization(),
43            layers.Dropout(0.3),
44            layers.Dense(self.num_classes, activation='softmax')
45        ])
46        
47        return model
48    
49    def fine_tune(self, learning_rate=1e-5):
50        """Fine-tune the pre-trained layers"""
51        # Unfreeze the base model
52        self.model.layers[0].trainable = True
53        
54        # Use a lower learning rate for fine-tuning
55        self.model.compile(
56            optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate),
57            loss='categorical_crossentropy',
58            metrics=['accuracy']
59        )
60
61# Usage
62transfer_classifier = TransferLearningClassifier(num_classes=5)
63transfer_classifier.model.compile(
64    optimizer='adam',
65    loss='categorical_crossentropy',
66    metrics=['accuracy']
67)

Object Detection Systems {#detection}

YOLO Implementation

1from ultralytics import YOLO
2import cv2
3import numpy as np
4
5class ObjectDetector:
6    def __init__(self, model_path='yolov8n.pt'):
7        self.model = YOLO(model_path)
8        self.class_names = self.model.names
9    
10    def detect_objects(self, image_path, confidence_threshold=0.5):
11        """Detect objects in image"""
12        results = self.model(image_path, conf=confidence_threshold)
13        
14        detections = []
15        for result in results:
16            boxes = result.boxes
17            if boxes is not None:
18                for box in boxes:
19                    # Extract box coordinates
20                    x1, y1, x2, y2 = box.xyxy[0].cpu().numpy()
21                    confidence = box.conf[0].cpu().numpy()
22                    class_id = int(box.cls[0].cpu().numpy())
23                    class_name = self.class_names[class_id]
24                    
25                    detections.append({
26                        'bbox': [x1, y1, x2, y2],
27                        'confidence': confidence,
28                        'class_id': class_id,
29                        'class_name': class_name
30                    })
31        
32        return detections
33    
34    def draw_detections(self, image_path, detections, output_path=None):
35        """Draw bounding boxes on image"""
36        image = cv2.imread(image_path)
37        
38        for detection in detections:
39            x1, y1, x2, y2 = map(int, detection['bbox'])
40            confidence = detection['confidence']
41            class_name = detection['class_name']
42            
43            # Draw bounding box
44            cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
45            
46            # Draw label
47            label = f"{class_name}: {confidence:.2f}"
48            label_size = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.5, 2)[0]
49            cv2.rectangle(image, (x1, y1 - label_size[1] - 10), 
50                         (x1 + label_size[0], y1), (0, 255, 0), -1)
51            cv2.putText(image, label, (x1, y1 - 5), 
52                       cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 0), 2)
53        
54        if output_path:
55            cv2.imwrite(output_path, image)
56        
57        return image
58    
59    def detect_video(self, video_path, output_path=None):
60        """Detect objects in video"""
61        cap = cv2.VideoCapture(video_path)
62        
63        if output_path:
64            fourcc = cv2.VideoWriter_fourcc(*'mp4v')
65            fps = int(cap.get(cv2.CAP_PROP_FPS))
66            width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
67            height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
68            out = cv2.VideoWriter(output_path, fourcc, fps, (width, height))
69        
70        while True:
71            ret, frame = cap.read()
72            if not ret:
73                break
74            
75            # Run detection
76            results = self.model(frame, conf=0.5)
77            
78            # Draw results
79            annotated_frame = results[0].plot()
80            
81            if output_path:
82                out.write(annotated_frame)
83            
84            # Display frame (optional)
85            cv2.imshow('Object Detection', annotated_frame)
86            if cv2.waitKey(1) & 0xFF == ord('q'):
87                break
88        
89        cap.release()
90        if output_path:
91            out.release()
92        cv2.destroyAllWindows()
93
94# Usage
95detector = ObjectDetector()
96detections = detector.detect_objects('image.jpg')
97detector.draw_detections('image.jpg', detections, 'output.jpg')

Real-Time Processing {#realtime}

Webcam Object Detection

1import threading
2import queue
3import time
4
5class RealTimeDetector:
6    def __init__(self, model_path='yolov8n.pt'):
7        self.model = YOLO(model_path)
8        self.frame_queue = queue.Queue(maxsize=10)
9        self.result_queue = queue.Queue(maxsize=10)
10        self.running = False
11    
12    def capture_frames(self, source=0):
13        """Capture frames from camera"""
14        cap = cv2.VideoCapture(source)
15        cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
16        cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
17        cap.set(cv2.CAP_PROP_FPS, 30)
18        
19        while self.running:
20            ret, frame = cap.read()
21            if ret:
22                if not self.frame_queue.full():
23                    self.frame_queue.put(frame)
24            time.sleep(0.01)
25        
26        cap.release()
27    
28    def process_frames(self):
29        """Process frames for object detection"""
30        while self.running:
31            if not self.frame_queue.empty():
32                frame = self.frame_queue.get()
33                
34                # Run detection
35                results = self.model(frame, conf=0.5, verbose=False)
36                annotated_frame = results[0].plot()
37                
38                if not self.result_queue.full():
39                    self.result_queue.put(annotated_frame)
40            time.sleep(0.01)
41    
42    def display_results(self):
43        """Display processed frames"""
44        while self.running:
45            if not self.result_queue.empty():
46                frame = self.result_queue.get()
47                cv2.imshow('Real-time Object Detection', frame)
48                
49                if cv2.waitKey(1) & 0xFF == ord('q'):
50                    self.running = False
51            time.sleep(0.01)
52        
53        cv2.destroyAllWindows()
54    
55    def start(self, source=0):
56        """Start real-time detection"""
57        self.running = True
58        
59        # Start threads
60        capture_thread = threading.Thread(target=self.capture_frames, args=(source,))
61        process_thread = threading.Thread(target=self.process_frames)
62        display_thread = threading.Thread(target=self.display_results)
63        
64        capture_thread.start()
65        process_thread.start()
66        display_thread.start()
67        
68        # Wait for threads to complete
69        capture_thread.join()
70        process_thread.join()
71        display_thread.join()
72
73# Usage
74real_time_detector = RealTimeDetector()
75real_time_detector.start()  # Press 'q' to quit

Production Deployment {#deployment}

FastAPI Web Service

1from fastapi import FastAPI, File, UploadFile, HTTPException
2from fastapi.responses import JSONResponse
3import uvicorn
4import io
5from PIL import Image
6import numpy as np
7
8app = FastAPI(title="Computer Vision API")
9
10# Initialize model
11detector = ObjectDetector()
12
13@app.post("/detect/")
14async def detect_objects_api(file: UploadFile = File(...)):
15    """API endpoint for object detection"""
16    try:
17        # Validate file type
18        if not file.content_type.startswith('image/'):
19            raise HTTPException(status_code=400, detail="File must be an image")
20        
21        # Read and process image
22        contents = await file.read()
23        image = Image.open(io.BytesIO(contents))
24        
25        # Convert to numpy array
26        image_array = np.array(image)
27        
28        # Save temporarily for processing
29        temp_path = f"temp_{file.filename}"
30        image.save(temp_path)
31        
32        # Run detection
33        detections = detector.detect_objects(temp_path)
34        
35        # Clean up
36        os.remove(temp_path)
37        
38        return JSONResponse(content={
39            "status": "success",
40            "detections": detections,
41            "count": len(detections)
42        })
43    
44    except Exception as e:
45        raise HTTPException(status_code=500, detail=str(e))
46
47@app.get("/health")
48async def health_check():
49    """Health check endpoint"""
50    return {"status": "healthy", "model": "loaded"}
51
52if __name__ == "__main__":
53    uvicorn.run(app, host="0.0.0.0", port=8000)

Docker Deployment

1FROM python:3.9-slim
2
3WORKDIR /app
4
5# Install system dependencies
6RUN apt-get update && apt-get install -y \
7    libglib2.0-0 \
8    libsm6 \
9    libxext6 \
10    libxrender-dev \
11    libgomp1 \
12    libglib2.0-0 \
13    && rm -rf /var/lib/apt/lists/*
14
15# Copy requirements and install Python dependencies
16COPY requirements.txt .
17RUN pip install --no-cache-dir -r requirements.txt
18
19# Copy application code
20COPY . .
21
22# Expose port
23EXPOSE 8000
24
25# Run the application
26CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Advanced Techniques {#advanced-techniques}

Custom Loss Functions

1import tensorflow as tf
2
3class FocalLoss(tf.keras.losses.Loss):
4    """Focal Loss for addressing class imbalance"""
5    
6    def __init__(self, alpha=0.25, gamma=2.0, **kwargs):
7        super().__init__(**kwargs)
8        self.alpha = alpha
9        self.gamma = gamma
10    
11    def call(self, y_true, y_pred):
12        # Compute cross entropy
13        ce_loss = tf.keras.losses.categorical_crossentropy(y_true, y_pred)
14        
15        # Compute p_t
16        p_t = tf.where(tf.equal(y_true, 1), y_pred, 1 - y_pred)
17        
18        # Compute alpha_t
19        alpha_t = tf.where(tf.equal(y_true, 1), self.alpha, 1 - self.alpha)
20        
21        # Compute focal loss
22        focal_loss = alpha_t * tf.pow(1 - p_t, self.gamma) * ce_loss
23        
24        return tf.reduce_mean(focal_loss)
25
26# Usage
27model.compile(
28    optimizer='adam',
29    loss=FocalLoss(alpha=0.25, gamma=2.0),
30    metrics=['accuracy']
31)

Model Optimization

1import tensorflow as tf
2
3class ModelOptimizer:
4    def __init__(self, model):
5        self.model = model
6    
7    def quantize_model(self):
8        """Quantize model for faster inference"""
9        converter = tf.lite.TFLiteConverter.from_keras_model(self.model)
10        converter.optimizations = [tf.lite.Optimize.DEFAULT]
11        quantized_model = converter.convert()
12        
13        return quantized_model
14    
15    def prune_model(self, target_sparsity=0.5):
16        """Prune model to reduce size"""
17        import tensorflow_model_optimization as tfmot
18        
19        prune_low_magnitude = tfmot.sparsity.keras.prune_low_magnitude
20        
21        pruning_params = {
22            'pruning_schedule': tfmot.sparsity.keras.PolynomialDecay(
23                initial_sparsity=0.0,
24                final_sparsity=target_sparsity,
25                begin_step=0,
26                end_step=1000
27            )
28        }
29        
30        pruned_model = prune_low_magnitude(self.model, **pruning_params)
31        return pruned_model
32
33# Usage
34optimizer = ModelOptimizer(model)
35quantized_model = optimizer.quantize_model()

Best Practices {#best-practices}

Performance Optimization

1. Data Pipeline Optimization

   • Use tf.data for efficient data loading
   • Implement prefetching and parallel processing
   • Cache frequently used data

2. Model Architecture

   • Use appropriate model size for your task
   • Implement early stopping to prevent overfitting
   • Use batch normalization for stable training

3. Hardware Utilization

   • Leverage GPU acceleration when available
   • Use mixed precision training for faster training
   • Implement model parallelism for large models

Production Considerations

• Monitoring: Implement comprehensive logging and monitoring
• Versioning: Use model versioning for reproducibility
• Testing: Implement thorough testing pipelines
• Security: Validate inputs and implement rate limiting
• Scalability: Design for horizontal scaling

Conclusion

Computer vision AI has immense potential across industries. By following this guide, you can build robust image recognition systems that perform well in production environments.

Start with simple classification tasks and gradually move to more complex object detection and real-time processing as you gain experience.

---

This guide provides a comprehensive foundation for building computer vision systems. Remember to always validate your models thoroughly before deploying to production.