Like most major cities, Portland, OR was actively seeking innovative ways to reduce traffic and increase road-user and pedestrian safety. This capstone project built upon previous work done by Dr. Fuxin Li and his team to address some of these problems using Deep Learning (DL) and Computer Vision (CV) techniques. The system in its current state was functional but improvements such as all-weather functionality, performance, and data availability needed to be made. A full-stack web application was our proposed solution to deliver to the City of Portland. This application included a website displaying live camera feeds with live object detection and an API feeding data collected by the computer vision and machine learning system. The goal of our project was to help the City of Portland make improvements to traffic safety and flow, thereby improving the quality of life for its citizens.

• 

  Initial demo of our Neural Network performing Object Detection on a clip of The Office before fine-tuning.

Object detection was one of the essential parts of the project and was also the basis of the other features in the project. In this project, we needed to first detect various kinds of objects, then mark the objects according to whether they needed to be obfuscated to maintain privacy. The accuracy of the object detection engine also influenced the performance of other parts, such as if it blurred the right things in the video. Additionally, the object detection engine needed to run quickly because it needed to process the video in real-time.

The object detection engine had to be able to process the video at 30 Frames Per Second (FPS) to be sufficient for the needs of our client. To improve the speed of object detection as well as the accuracy, we converted the module into TensorRT, a technology provided by NVIDIA to improve the performance. This enabled us to achieve a ~40% model prediction improvement. We trained the machine learning model based on YOLOv3 with multiple kinds of objects. These objects included but were not limited to: cars, pedestrians, bikes, scooters, skateboards, and wheelchairs. We used a larger training dataset to improve the accuracy of the models. We also tried different kinds of learning rate models when we were using dynamic learning rates to train the model. This helped us reduce the overfitting problem when we built the machine learning model.

• 

  Depiction of the dataflow between the different elements of the pipeline.

A website that utilized the API was built to make it easier for City of Portland employees to analyze the data. It included multiple live video feeds and supported the HTTP Live Streaming (HLS) and Real-Time Messaging Protocol (RTMP) live-feed video sources, and it showed the detection system working in real time. The website also supported user-defined visualization charts.

The Web API for this project was developed using MongoDB, Node.js, Express.js, and Docker. In terms of the Model-View-Controller (MVC) software architecture, the Web API was responsible for the model (MongoDB) and the controller (Node.js). The MongoDB database was created according to the specifications that were given to us by a representative from the city of Portland. A database was drafted using an Entity Relationship Diagram (ERD) to provide a clear visual representation for all objects, attributes, and relationships in the database. A functioning database was created from the ERD using MongoDB. Then, Express.js was used to communicate between the model and the view. Finally, the whole web framework was containerized with Docker to allow for easy deployment with one-key install scripts.

• 

  Depiction of the Web UI being tested locally before deployment to Portland's CCTV system.

The combination of these parts of the project resulted in a complete system that detected objects in live traffic cameras, extracted data from those objects, and displayed it on a website to make the data easy to analyze. At the same time, ensuring the privacy of individuals and their assets that were captured on camera was also very important. The data collected from the detection system was analyzed, organized, and then stored into the MongoDB database. The web application then queried the database and made the data available with a REST API, which was then used by the front-end to display the data neatly on a webpage. Using this data, the City of Portland was able to make informed decisions regarding traffic safety.

• 

  Final poster for the project.