Project information

  • Client: HEIG-VD
  • Project URL: https://modos.heig-vd.ch/
  • Categories: Image Classification, Deep Learning, Computer Vision, Model Deployment
  • Main technologies: Python, Tensorflow, Keras, Tensorflow-Serving, Docker

Summary

MoDoS is a social-impact initiative developed with HEIG-VD to create safe, accessible routing applications for seniors and individuals with reduced mobility. My role was to design, train, and deploy the core Computer Vision engine responsible for analyzing real-world street images and identifying potential mobility hazards.

Using TensorFlow and Keras, I fine-tuned a MobileNetV2 architecture to process crowdsourced images in real-time. Because the application required high availability and low latency, I containerized the final system with Docker and deployed it using TensorFlow Serving, seamlessly integrating the AI directly into the MoDoS user application.

Overcoming Imbalanced Data with a Two-Stage Architecture

The raw dataset was crowdsourced during a local "Mapping Party." Consequently, the data was highly imbalanced, containing an overwhelming number of "normal" street images and very few examples of actual hazards.

To prevent the model from becoming biased, I engineered a specialized two-stage classification pipeline:

  • Stage 1 (Anomaly Detection): A binary classifier that first determines if an image contains a problem or is completely clear. (Achieved an F1-score of 0.81)
  • Stage 2 (Hazard Categorization): If an anomaly is detected, the image is passed to a secondary, highly specialized model that categorizes the exact hazard across seven classes (e.g., steep slopes, dangerous paths, narrow paths, obstacles, surface problems, or missing curb ramps). (Achieved an F1-score of 0.90)

The Result: By cascading these models, the system achieved a combined pipeline score of 0.73, successfully providing a lightweight, highly accurate visual AI tool to route vulnerable pedestrians away from urban hazards.