StalkedByTheState - Easy to install home security system based on CNN computer vision

Hi all,

I'd like to introduce my new project, "StalkedByTheState", practical and very versatile home security system (State machine). It's not really new actually, I developed the bulk of the GUI based state machine back in 2010 but was using inputs from various sensors to let me know about intruders. The lastest of those that I thought was great at the time was a hardware hack to turn a PIR sensor at 433MHz to a Lorawan things network based sensor.

But then the yolo (You only look once) algorithm came out ( https://pjreddie.com/darknet/yolo/) the performance of this algorithm powered by GPU was really amazing. Shortly after the Nano came out I had a working version of the yolov3 based triggering running, so it's been at several beta sites for around two years. But these were custom setups done by me to hook everything together.

Recently however, I completed a lot of work to make the whole installation and initial configuration complete with just two commands. But in addition, I made it support potentially multiple different AI models, even within the same image processing. Out of the box it supports yolov3 and yolov4 algorithms. It installs easily to the Jetson Nano, the Jetson Xavier NX and the Jetson Xavier AGX.

Configuration of the event triggering is a little more complex at the moment, because it involves editing a json configuration file. But the next big thing I will be working on is a graphical frontend to all of this configuration, then a little later wizards to make it really easy. That's the road map anyway.

A list of features include:

• Security state machine that records triggered video events. Supports time of day/day of week scheduled events. Watchdog timers,
  rate based triggers, synthetic combination events.
• multi-model CNN scanner that can use multiple AI models within the same image even. You
  can choose to use yolov3 in one part of an image, yolov4 in another, or choose to have to triggered confirmed with both.
  Or either… Classification is done via a network service over a websocket so you can add external models to your appliance
  if you like, or models that are not either YoloV3 or YoloV4
• Scanner that reads camera frames, classifies them and then matches them for triggers can be dynamically enabled and disabled, either
  at the whole camera level or down to a specific notification level
• Support for auto-review letsencrypt certificates. Can be setup as a reverse proxy to tunnel access to StalkedByTheState
  and your cameras and other devices over https
• Elicit a real-world response. Supports out of the box Phidget Interfacekit USB I/O controllers for switching relays or
  reading switches, smoke/carbon monoxide detectors and other wired sensors. Read responses from commonly available remote controls
  such as from Klikaanklikuit in the Netherlands. Or control klikaanklikuit devices.
• Builtin Certificate manager. GUI controlled generation of X509 certificates and importing and exporting of certificates. Use
  self-signed certificates or real ones. Use https internally for device-device communications
• Websocket based event subscribe interface
• Resilient mode of operation that runs the OS in read-only mode with a read-write memory overlay layer and the configuration
  stays read-only till it needs updating. Auto-repair on boot so no hung appliances (Waiting for keyboard input for repair).
• Runs off SSD

I don't have a normal website yet so my only documentation currently is via YouTube videos. But that's where a forum like this can be handy. I can answer whatever questions you may have here.

To give you some insights, the nano will process a frame of video (640x480ish) in about 0.7s. The Xavier NX is about 3x faster. Myself, I process around 15x cameras continuously and an rtx 2080ti and will detect an intruder with a video alert with a couple of seconds. Verified against 2x computer vision algorithms for very low false alerts.

Anyway, if you want to give this a go the installation should go very smoothly, it takes around 30 minutes and just 2x commands, but the further configuration I expect a lot of questions on currently, but I'll provide a lot of help to explain things. And once you have it configured the way you want it will just reliably run for a long time, it's very stable. I've been running the core software for more than 10 years.

First install this:

https://github.com/hcfman/sbts-base

and then this:

https://github.com/hcfman/sbts-install

(I'll first read up the guidelines before pasting any code).

Cheers,

Kim Hendrikse