There is a specific thrill that comes with working in software engineering—the moment you are handed a unique, tangible problem and told to build a bridge between the physical world and a digital system. Recently, we received an inquiry for a custom integration: a client needed an automated way to track, flag, and report physical rule violations using their existing surveillance infrastructure.
Here is a deep dive into how we architected the solution.
The Problem Statement: Catching Bad Neighbours
- The Client: A housing society management company.
- The System: They already had a fully functional CRM to manage residents, but they were dealing with a highly manual, tedious problem.
- The Issue: Residents were violating society rules—specifically, throwing garbage out of their windows.
The client wanted to integrate their CRM with a new API endpoint. The goal was simple but technically demanding: upload estate CCTV footage and automatically receive the exact timestamps and the specific Flat Numbers of the offending residents. They needed actionable data to issue warnings and fines, without having security guards manually watch hours of empty video.
The Solution: Our Swiss Army Knife Approach
When you have a strong foundation in backend development and distributed systems, a problem like this immediately starts looking like a data pipeline. The core idea was to break down the video into images, analyze those images, and return a structured report.
- Web Framework: Django (Later migrated to FastAPI for better asynchronous performance and speed).
- Video Processing: OpenCV.
- Machine Learning/Vision: Google Vision API.
- Infrastructure: AWS API Gateway & AWS Lambda.
- Message Broker: RabbitMQ.
To make this easy to understand, think of a video not as a continuous movie, but as a flipbook of thousands of individual pictures (frames). Processing a whole video is heavy and slow. Processing specific, compressed pictures is fast and scalable.
Here is the step-by-step logic of our pipeline:
- Ingestion: The CRM sends the video to our AWS API Gateway.
- Deconstruction: We use OpenCV to compress the video and extract key frames (images) at specific intervals.
- Analysis: These frames are sent to the Vision Model.
- Mapping: The model draws a “bounding box” around the detected object (e.g., falling garbage). We then cross-reference that box’s location with our static map of the building to identify the CCTV ID and the specific window/Flat Number.
- Reporting: We aggregate these events and send a clean JSON report back to the CRM.
Hard Work and Edge Cases
The theory is always cleaner than the execution. Passing frames to a vision model to create bounding boxes is a neat trick, but making it reliable requires rigorous data engineering.
A massive shoutout to Sai, our Data Scientist, who helped turn this concept into reality. By packaging his computer vision logic into an API hosted on AWS API Gateway and decoupling the heavy lifting with RabbitMQ, we ensured the system could handle massive spikes in video uploads without crashing.
However, the AI is only as good as its context. Our team spent long weekends manually labeling and marking each CCTV viewpoint. We had to map out the exact pixel coordinates for every Flat Number on every camera feed to create our baseline database.
We also had to engineer our way around complex edge cases. What happens if multiple incidents occur in a single video source simultaneously? To solve this, we had to coordinate with the onsite client team to update the expected response schema. We shifted the API payload from returning simple Strings to Lists of Strings , allowing the CRM to ingest multiple timestamps and Flat IDs from a single video upload.
Future Scope: Evolving the Eye
The solution is currently in its testing phase, but the client is already seeing the potential of automated surveillance. They have requested an expansion of the system’s capabilities, including:
- Acoustic Analysis: Detecting loud noises or disturbances.
- ANPR/ALPR: Identifying parked cars and automatically raising fines using registration number recognition.
- Biometrics: Shift attendance and roster management using facial recognition for estate staff.
Growing with the Cloud
If there is one takeaway from this project, it is that data is constantly evolving, and our engineering skills must scale alongside it.
Designing a pipeline that can ingest, process, and return video data seamlessly required a deep reliance on cloud infrastructure. This project coincided with me officially clearing my AWS Certified Cloud Practitioner exam. Understanding the intricacies of how the cloud shapes distributed systems—and how to architect those systems efficiently—was the exact knowledge needed to make this API a success.
Till next time, Happy Coding :)
