Versatile Camera – Development

The Versatile Mobile Camera demonstrator has three mainly components.

  • The EoT device
  • The configuration application
  • The WebApp

The EoT application starts a MQTT broker in the EoT device. This, the EoT device is able to send images (captured by the EoT Device) to the connected Android device. In the same way, using the MQTT broker, the EoT device is able to receive the capture configuration. Finally, the EoT device is able to communicate directly with the Firebase Cloud Messaging and the Google Storage Cloud services. This way, the EoT Device is able to send push notifications with an image using HTTP requests.

The configuration application can request the captured images by the EoT Device using MQTT and send it to Firebase cloud service using the Firebase. In addition to the images, the labels are also sent. The label files contain the information about the images (key events detected, options enabled, etc.). The images are uploaded to Google Cloud Storage and the labels are stored in a real time database provided by Firebase.

Finally, the WebApp, part of the configuration application, interacts with the Firebase cloud services. The WebApp is able to visualize the images stored in Firebase. The WebApp can filter the images stored in Firebase server by the date or by ‘key events’.

 

The following EoT libraries have been used in order to develop the Versatile Mobile Camera demonstrator: WiFi-Functions, SDCardIO, TimeFunction, Camera, MQTT, tiny_dnn, JsonParser and Libccv/OpenCV.

The EoT software of the Versatile Mobile Camera demonstrator has seven modules:

  • Capture Modes module: This module provides the different capture modes. The demonstrator has two capture modes:
    • Normal mode: In this mode, one image is captured every X seconds
    • Smart mode: Each image is examined and in case it contains a key event (as defined in the configuration), the image will be stored.
  • Classification of images module: This module allows to the demonstrator to classify the following ‘key events’: FaceDetected, LargeFaceDetected, Anger, Disgust, Fear, Happiness, Neutral, Sadness, Surprise and MotionDetection.
  • Image Storage Module: Management of the SD card storage is included in this module
  • MQTT Broker module: The MQTT broker handling Wi-Fi communications is implemented in this module. This broker runs in the EoT Device.
  • Notifications module: This module is responsible for the alarm functionality.
  • Configuration section module:  With this module, the EoT Device configuration is possible. The EoT Device receives the configuration parameters wirelessly using the MQTT protocol. The EoT Device application running in the EoT device stores/updates the configuration in a file in the SD card. This configuration includes:
    • Setting alarms.
    • Setting capture mode.
    • Setting network parameters.
    • Setting the MQTT parameters
  • Connectivity: This module adds to the demonstrator the possibility of the connection of the EoT device with a network and the Android app.

On the other hand, the configuration application of the Versatile Mobile Camera demonstrator has been divided into 6 modules.

  • Sign In Module: This module provides a sign in method with the Google account.
  • Communication between the Android application and the embedded WebApp module: The native application (Android) will have an ‘embedded’ WebApp. The correctly integration between the WebApp and the native application is the purpose of this module.
  • Configuration section module: This module includes the configuration screen of the EoT Device and the configuration Application. This configuration includes:
    • Setting alarms. The user is able to specify events that should generate an alarm. In addition, the user can customize the alarm notifications.
    • Setting capture mode. The user is able to change the capture mode of the EoT Device.
    • Setting the network parameters. The user is able to update the SSID and the password of the bridge (i.e. whatever device that acts as a router and the EoT device connects to).
  • MQTT client module: The Android application includes a MQTT client. This client is in charge of:
    • Connecting with the MQTT broker.
    • Sending configuration parameters.
    • Receiving images and their metadata.
    • Sending application updates.
  • Synchronization in the cloud module: In this module, interaction between the Android application and the Firebase server is managed.
    • Uploading images
    • Updating configuration
  • Connectivity module: This module includes the connection between the EoT Device and the configuration application.

The WebApp is a classic Javascript & Ruby v2.4.2 & Firebase implementation hosted on Heroku using multiple 3rd party APIs & SDKs. Some used APIs are Dropbox, Evercam, Zapier, Mailgun, IFTTT, FFMPeg, etc.

Versatile Camera – Concept http://eyesofthings.eu/?page_id=1883
Versatile Camera – Results http://eyesofthings.eu/?page_id=1887