CSC173-DeepCV-Bautista

Deep Computer Vision for Intelligent Indoor Temperature Management

CSC173 Intelligent Systems Final Project
Mindanao State University - Iligan Institute of Technology
Student: Caine Ivan R. Bautista, 2022-0378
Semester: A.Y. 2025-2026 1st Semester
Python PyTorch

Abstract

Occupant behavior plays a critical role in indoor energy consumption and thermal comfort, yet conventional heating, ventilation, and air-conditioning (HVAC) systems often rely on static schedules that fail to reflect real-time occupancy conditions. This study presents a deep learning–based computer vision framework for real-time human detection, posture recognition, and clothing insulation classification in indoor environments. The proposed system integrates a lightweight YOLO11n model fine-tuned for human detection, a ResNet-18 posture classification model trained on silhouette representations, and a ResNet-50 clothing classification model leveraging transfer learning from ImageNet. A custom indoor dataset consisting of multi-person scenes with seated and standing occupants and varying clothing types was used for training and evaluation. Experimental results demonstrate reliable multi-person detection in crowded settings and accurate classification of sitting and standing postures, as well as light, medium, and heavy clothing categories under partial occlusion. The system achieves strong real-time performance while maintaining low computational overhead, making it suitable for continuous monitoring applications. The main contributions of this work include (1) a multi-model pipeline combining detection, silhouette-based posture analysis, and clothing recognition, (2) efficient handling of occlusion and scale variation in indoor scenes, and (3) a practical vision-based sensing approach that supports occupant-centric HVAC control and energy-efficient building operation.

Table of Contents

Introduction

Problem Statement

Air conditioning systems in the Philippines typically operate at fixed temperatures, leading to energy waste and user discomfort. In tropical climates like in Iligan, rooms with varying occupancy levels and diverse clothing choices often result in either overcooling or insufficient cooling. This project addresses the need for intelligent climate control by developing a computer vision system that analyzes room occupancy, human posture, and clothing type to recommend optimal temperature settings, by providing the necessary environment data to a neural network. Such a system could reduce energy consumption while improving thermal comfort in offices, classrooms, and public spaces across the region.

Objectives

Problem Demo

Deep learning–based object detection and classification frameworks, particularly those belonging to the You Only Look Once family, have become widely adopted for real-time vision applications due to their ability to jointly localize and recognize objects within a single feedforward pass. One-stage detection architectures are explicitly designed to balance high detection speed with competitive accuracy, making them well suited for real-time human detection tasks in both high-performance and resource-constrained environments. Comparative studies of object detection networks indicate that lightweight detection models provide a favorable trade-off between computational efficiency and detection accuracy, improving system responsiveness in scenarios where low latency and continuous operation are critical [1].

In parallel, convolutional neural networks pretrained on large-scale datasets such as ImageNet have demonstrated strong transfer learning capabilities when adapted to domain-specific classification tasks. Residual network architectures are particularly effective due to their ability to learn deep hierarchical feature representations while mitigating vanishing gradient issues. Deeper variants, such as ResNet-50, have been shown to excel in fine-grained visual recognition tasks, including clothing category identification and texture-based classification, while shallower architectures like ResNet-18 offer efficient feature extraction with lower memory and computational requirements. Transfer learning using residual networks accelerates convergence and reduces the dependence on large labeled datasets, as pretrained features capture general visual patterns transferable across tasks [2], [3].

Despite the success of these approaches, challenges remain when applying generic detection or classification models to specialized or localized datasets characterized by diverse body poses, occlusions, and clothing variations. This motivates the adoption of a multi-model pipeline in this work, consisting of a YOLO11n-based detector for efficient human localization, a ResNet-18 model for posture classification optimized for low computational overhead, and a ResNet-50 classifier for robust clothing recognition. By combining real-time detection with task-specific transfer learning–based classification, the proposed approach addresses limitations of prior single-stage or single-task systems and aligns with occupant-centric frameworks such as ECO-FOCUS, where accurate human detection and clothing estimation are critical inputs for higher-level decision-making systems [1].

Methodology

Human Detection

Dataset

Architecture

For human detection, the base YOLO11n archtecture is used but modified to only detect object person/human. YOLO11n Architecture

Parameter Value
Batch Size 16
Image Size 640 × 640
Epochs 100
Optimizer Adam with Weight Decay
Initial Learning Rate 0.01
Final Learning Rate 0.001
Learning Rate Scheduler Cosine Annealing
Momentum 0.937
Weight Decay 0.0005
Warmup Epochs 5
Random Seed 42
Mixed Precision Training Enabled
Pretrained Weights Enabled
Number of Data Loader Workers 2
Non Maximum Suppression Intersection over Union Threshold 0.7
Confidence Threshold During Training 0.001

Training Code Snippet

model_name = 'yolo11n.pt'
model = YOLO(model_name)
model.train(
    data=str(data_yaml),
    epochs=100,
    batch=16,
    imgsz=640,
    seed=42,
    optimizer="AdamW",
    weight_decay=0.0005,
    momentum=0.937,
    cos_lr=True,
    lr0=0.01,
    lrf=0.001,
    warmup_epochs=5,
    warmup_momentum=0.8,
    warmup_bias_lr=0.1,
    close_mosaic=15,
    label_smoothing=0.0,
    dropout=0.0,
    box=7.5,
    cls=0.5,
    dfl=1.5,
    hsv_h=0.015,
    hsv_s=0.7,
    hsv_v=0.4,
    degrees=5.0,
    translate=0.2,
    scale=0.9,
    shear=0.0,
    perspective=0.0,
    flipud=0.0,
    fliplr=0.5,
    mosaic=0.8,
    mixup=0.1,
    copy_paste=0.2,
    iou=0.7,
    conf=0.001,
    verbose=True,
    project=project_name,
    name=experiment_name,
    exist_ok=True,
    pretrained=True,
    save=True,
    save_period=10,
    device=device,
    workers=2,
    amp=True,
    cache=False,
    fraction=1.0,
)

Posture Classification

Dataset

Architecture

ResNet-18 pretrained was used as the base model for Posture Classification.

ResNet18 Diagram

Parameter Value
Image Size 256
Batch Size 32
Learning Rate 0.003 then 0.00001
Epoch 25 + 15
Criterion CrossEntropyLoss
Optimizer SGD
Momentum 0.9
Weight Decay 0.0005
Scheduler ReduceLROnPlateau then CosineAnnealingLR

Training Code Snippet

model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)
DEVICE = torch.device(device)
# Replace classifier head
model.fc = nn.Linear(model.fc.in_features, NUM_CLASSES)
model = model.to(DEVICE)

# Training params
EPOCHS = 25
LR = 3e-2

for param in model.parameters():
    param.requires_grad = False

for param in model.fc.parameters():
    param.requires_grad = True

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=LR, momentum=0.9, weight_decay=5e-4)
scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='max', factor=0.1, patience=3)

best_val_acc = 0.0
for epoch in range(EPOCHS):
    model.train()
    start = time.time()

    running_loss = 0.0
    correct = 0
    total = 0

    print(f"\nEpoch {epoch+1}/{EPOCHS} | Image Size {IMG_SIZE}")
    pbar = tqdm(train_loader, leave=False)
    for imgs, labels in pbar:
        imgs = imgs.to(DEVICE)
        labels = labels.to(DEVICE)

        optimizer.zero_grad()
        outputs = model(imgs)
        loss = criterion(outputs, labels)
        loss.backward()
        
        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
        
        optimizer.step()

        running_loss += loss.item()
        _, preds = torch.max(outputs, 1)
        correct += (preds == labels).sum().item()
        total += labels.size(0)

    train_loss = running_loss / len(train_loader)
    train_acc = correct / total

    metrics = validate(model, val_loader, criterion, DEVICE)
    val_acc = metrics["accuracy"]
    
    scheduler.step(val_acc)

    elapsed = time.time() - start

    history["train_loss"].append(train_loss)
    history["train_acc"].append(train_acc)
    history["val_loss"].append(metrics["loss"])
    history["accuracy"].append(metrics["accuracy"])
    history["precision"].append(metrics["precision"])
    history["recall"].append(metrics["recall"])
    history["confusion_matrices"].append(metrics["confusion_matrix"])

    print(f"Train Loss: {train_loss:.3f} | Train Acc: {train_acc:.3f}")
    print(f"Val Loss:   {metrics['loss']:.3f} | Val Acc:    {val_acc:.3f}")
    print(f"Val Prec:   {metrics['precision']:.3f} | Val Rec:    {metrics['recall']:.3f}")

    print("\nConfusion Matrix:")
    print(f"{'':12}{CLASSES[0]:>12}{CLASSES[1]:>12}")
    for i, row in enumerate(metrics['confusion_matrix']):
        print(f"{CLASSES[i]:12}{row[0]:12}{row[1]:12}")

    if val_acc > best_val_acc:
        best_val_acc = val_acc
        torch.save(model.state_dict(), WEIGHTS_DIR / BEST_MODEL_NAME)
        print(f"New best model saved (val_acc={best_val_acc:.4f})")


    print(f"Epoch time: {elapsed:.1f}s")

Clothing Classification

Dataset

Architecture

ResNet-50 pretrained was used as the base model for Clothing Classification.

ResNet50 Diagram

Parameter Value
Image Size 256
Batch Size 32
Learning Rate 0.03 then 0.00001
Epoch 25 + 15
Criterion CrossEntropyLoss
Optimizer SGD
Momentum 0.9
Weight Decay 0.0005
Scheduler ReduceLROnPlateau then CosineAnnealingLR

Training Code Snippet

model = models.resnet50(weights=models.ResNet50_Weights.DEFAULT)
DEVICE = torch.device(device)
# Replace classifier head
model.fc = nn.Linear(model.fc.in_features, NUM_CLASSES)
model = model.to(DEVICE)

EPOCHS = 25
LR = 3e-2  

# Freeze pretrained layers initially
for param in model.parameters():
    param.requires_grad = False

for param in model.fc.parameters():
    param.requires_grad = True

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=LR, momentum=0.9, weight_decay=5e-4)
scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='max', factor=0.1, patience=3)

best_val_acc = 0.0
for epoch in range(EPOCHS):
    model.train()
    start = time.time()

    running_loss = 0.0
    correct = 0
    total = 0

    print(f"\nEpoch {epoch+1}/{EPOCHS} | Image Size {IMG_SIZE}")
    pbar = tqdm(train_loader, leave=False)
    for imgs, labels in pbar:
        imgs = imgs.to(DEVICE)
        labels = labels.to(DEVICE)

        optimizer.zero_grad()
        outputs = model(imgs)
        loss = criterion(outputs, labels)
        loss.backward()

        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
        optimizer.step()

        running_loss += loss.item()
        _, preds = torch.max(outputs, 1)
        correct += (preds == labels).sum().item()
        total += labels.size(0)

    train_loss = running_loss / len(train_loader)
    train_acc = correct / total

    metrics = validate(model, val_loader, criterion, DEVICE)
    val_acc = metrics["accuracy"]
    
    scheduler.step(val_acc)

    elapsed = time.time() - start

    history["train_loss"].append(train_loss)
    history["train_acc"].append(train_acc)
    history["val_loss"].append(metrics["loss"])
    history["accuracy"].append(metrics["accuracy"])
    history["precision"].append(metrics["precision"])
    history["recall"].append(metrics["recall"])
    history["confusion_matrices"].append(metrics["confusion_matrix"])

    print(f"Train Loss: {train_loss:.3f} | Train Acc: {train_acc:.3f}")
    print(f"Val Loss:   {metrics['loss']:.3f} | Val Acc:    {val_acc:.3f}")
    print(f"Val Prec:   {metrics['precision']:.3f} | Val Rec:    {metrics['recall']:.3f}")

    print("\nConfusion Matrix:")
    print(f"{'':12}{CLASSES[0]:>12}{CLASSES[1]:>12}{CLASSES[2]:>12}")
    for i, row in enumerate(metrics['confusion_matrix']):
        print(f"{CLASSES[i]:12}{row[0]:12}{row[1]:12}{row[2]:12}")

    if val_acc > best_val_acc:
        best_val_acc = val_acc
        torch.save(model.state_dict(), WEIGHTS_DIR / BEST_MODEL_NAME)
        print(f"New best model saved (val_acc={best_val_acc:.4f})")
   
    print(f"Epoch time: {elapsed:.1f}s")

Experiments & Results

Metrics

Human Detection

Metric Train Val
box_loss 1.35 1.34
cls_loss 1.13 1.11
dfl_loss 1.37 1.35
mAP@0.5 95.19% 95.19%
mAP50-95 79.82% 79.82%
Precision 1.0 1.0
Recall 0.90 0.90
Confusion Metrics HD_Confusion_Metrics  

HD Training Curve

Posture Classification

Metric Train Val
Loss 0.0835 0.1619
Accuracy 0.9667 0.9563
Precision 0.9698 0.9652
Recall 0.9637 0.9447
Confusion Metrics PC_Confusion_Metrics  

PC Training Curve

Clothing Classification

Metric Train Val
Loss 0.2686 0.5241
Accuracy 0.9102 0.8073
Precision 0.9102 0.8068
Recall 0.9102 0.8068
Confusion Metrics CC_Confusion_Metrics  

CC Training Curve

Demo

Final Presentation

(Note: This annotation can also be used for videos, just modify the gif_annotator.py)

Discussion

1. YOLO11n-Based Human Detection Model

Strengths

Limitations

Insights

2. Posture Classification Model (ResNet-18, Silhouette-Based)

Strengths

Limitations

Insights

3. Clothing Classification Model (ResNet-50)

Strengths

Limitations

Insights

Ethical Considerations

Conclusion

This work successfully demonstrates a multi-stage deep learning framework for real-time human detection, posture classification, and clothing insulation recognition in indoor environments. By integrating a YOLO11n-based human detector with ResNet-18 and ResNet-50 classifiers, the system achieves accurate multi-person analysis while maintaining computational efficiency. The use of silhouette-based posture recognition effectively decouples body geometry from appearance, while deep residual learning enables robust clothing classification under varied poses and occlusion conditions. The annotated results confirm the system’s applicability to realistic office and waiting-room scenarios, supporting occupant-centric sensing for intelligent HVAC control.

Future work will focus on deploying the system on edge and embedded platforms such as NVIDIA Jetson or Raspberry Pi for Internet-of-Things (IoT) applications. Additional directions include incorporating temporal modeling to improve stability across video frames, expanding the clothing dataset to capture seasonal and regional apparel variations, and integrating comfort prediction models to enable closed-loop adaptive HVAC control.

Installation

  1. Clone repo: git clone https://github.com/caineirb/CSC173-DeepCV-Bautista
  2. Install deps: pip install -r requirements.txt
  3. Select trained models in runs/ under each specific folders

requirements.txt:

asttokens==3.0.1
certifi==2025.11.12
charset-normalizer==3.4.4
comm==0.2.3
contourpy==1.3.3
cycler==0.12.1
debugpy==1.8.17
decorator==5.2.1
executing==2.2.1
filelock==3.20.0
fonttools==4.61.0
fsspec==2025.12.0
idna==3.11
ipykernel==7.1.0
ipython==9.8.0
ipython_pygments_lexers==1.1.1
jedi==0.19.2
Jinja2==3.1.6
joblib==1.5.2
jupyter_client==8.7.0
jupyter_core==5.9.1
kagglehub==0.3.13
kiwisolver==1.4.9
MarkupSafe==3.0.3
matplotlib==3.10.7
matplotlib-inline==0.2.1
mpmath==1.3.0
nest-asyncio==1.6.0
networkx==3.6.1
numpy==2.2.6
nvidia-cublas-cu12==12.8.4.1
nvidia-cuda-cupti-cu12==12.8.90
nvidia-cuda-nvrtc-cu12==12.8.93
nvidia-cuda-runtime-cu12==12.8.90
nvidia-cudnn-cu12==9.10.2.21
nvidia-cufft-cu12==11.3.3.83
nvidia-cufile-cu12==1.13.1.3
nvidia-curand-cu12==10.3.9.90
nvidia-cusolver-cu12==11.7.3.90
nvidia-cusparse-cu12==12.5.8.93
nvidia-cusparselt-cu12==0.7.1
nvidia-nccl-cu12==2.27.5
nvidia-nvjitlink-cu12==12.8.93
nvidia-nvshmem-cu12==3.3.20
nvidia-nvtx-cu12==12.8.90
opencv-python==4.12.0.88
packaging==25.0
pandas==2.3.3
parso==0.8.5
pexpect==4.9.0
pillow==12.0.0
platformdirs==4.5.1
polars==1.36.1
polars-runtime-32==1.36.1
prompt_toolkit==3.0.52
psutil==7.1.3
ptyprocess==0.7.0
pure_eval==0.2.3
Pygments==2.19.2
pyparsing==3.2.5
python-dateutil==2.9.0.post0
pytz==2025.2
PyYAML==6.0.3
pyzmq==27.1.0
requests==2.32.5
scikit-learn==1.8.0
scipy==1.16.3
seaborn==0.13.2
setuptools==80.9.0
six==1.17.0
stack-data==0.6.3
sympy==1.14.0
threadpoolctl==3.6.0
torch==2.9.1
torchvision==0.24.1
tornado==6.5.2
tqdm==4.67.1
traitlets==5.14.3
triton==3.5.1
typing_extensions==4.15.0
tzdata==2025.2
ultralytics==8.3.235
ultralytics-thop==2.0.18
urllib3==2.6.1
wcwidth==0.2.14

References

[1] Ren, Z., Ma, S., Li, X., & Kim, J. I. (2025). ECO-FOCUS: Integrating computer vision and machine learning for personalized comfort prediction and adaptive HVAC control in office buildings. Journal of Building Engineering, 111, 113613. https://doi.org/10.1016/j.jobe.2025.113613

[2] Medina, A., Méndez, J. I., Ponce, P., Peffer, T., Meier, A., & Molina, A. (2022). Using Deep Learning in Real-Time for Clothing Classification with Connected Thermostats. Energies, 15(5), 1811. https://doi.org/10.3390/en15051811

[3] Tung, T. C., Uswah Khairuddin, Mohd Ibrahim Shapiai, Nor, N. M., Wen, M., & Mohd, A. (2022). Calf Posture Recognition Using Convolutional Neural Network. Computers, Materials & Continua/Computers, Materials & Continua (Print), 74(1), 1493–1508. https://doi.org/10.32604/cmc.2023.029277

GitHub Pages

View this project site: https://caineirb.github.io/CSC173-DeepCV-Bautista/