top of page
This site was designed with the
.com
website builder. Create your website today.Start Now

ML | Training Image Classifier using Tensorflow Object Detection API

 

This article aims to learn how to build an object detector using Tensorflow’s object detection API.

Requirement :

​

  • Python Programming

  • Basics of Machine Learning

  • Basics of neural networks (Not Mandatory)

  • An enthusiasm to build a Cool project(Mandatory) :p

 

Even though if you don’t have the first three essentials, you’re welcome to the adventure. Don’t worry about getting lost, I’ll guide you properly through the journey!

Unmanned Ground Vehicle tracking using IoT techniques

   1. Abstract

An unmanned ground vehicle (UGV) is a vehicle that operates while in contact with the ground and doesn’t require an onboard human presence. In this paper, a novel approach of tracking of unmanned ground vehicles using IoT techniques is proposed.  UGVs are used for many applications where it may be inconvenient, dangerous, or impossible to have a human operator present.  The obstacle avoidance method is experimentally verified. It can perform complete surveillance in remote areas while being connected through a cloud based platform.

   2. Introduction

Generally, the UGVs have a set of sensors to observe the environment while passing the information to a human operator at a different location who will control the vehicle through teleoperation. There are a wide variety of UGVs in use today. These vehicles are used to replace humans in hazardous situations, such as handling explosives or in bomb disabling, where additional strength or smaller size is needed, or where humans cannot easily go. Also, they have military applications which include firefighting, surveillance, reconnaissance, and target acquisition. The UGV used here contains a Raspberry Pi which is programmed to detect and avoid obstacles while providing information about its location to the operator. A remote-operated UGV is a vehicle that is controlled by a human operator via interface. A primary purpose of UGV sensors is navigation, another is environment detection.

   3. Previous Work​

A co-operative target detection and tracking of UGV’s is proposed by Huang Liang [1]. A control method of UGV’s using State estimation and Robotic formalism is proposed by Gyorgy Max [2]. A Model Predictive Control (MPC) for UGV is given by Wang Meiling [3]. The method for moving ground target tracking was proposed by Mark Owen [4]. The path planning and path tracking of UGV’s in a map of battle field is given by Liguo Weng [5]. A UGV outdoor navigation system based on LiDAR and Vision Data is proposed by Ashraf Saleem [6]. A surveillance tele-operated UGV which works on a 3D-hand-tracking simulation system is given by Hassan K. Sawalmeh [7]. Scheduling Strategies of Relay Tracking for Network-Based multiple Unmanned Ground Vehicles was done by Song Zhiqiang [8]. The motion tracking for an Unmanned Ground Vehicle (UGV) under delayed and slow Feedback was proposed by Huahua Tao [9]. Neural-network-based learning control for the high-speed path tracking of unmanned ground vehicles was proposed by Xin Xu [10].

   4. Plan of work

The proposed method is to construct a UGV which is capable of being controlled in remote locations over the cloud.

   5. Total explanation of the UGV construction

The UGV is constructed using a Raspberry Pi, a Pi camera, a 32GB memory card, four 100rpm motors, 4 adequately sized tires, 4 clamps, 2 motor driver shields L293D, 2 ultrasonic sensors, 4 magnets, Hall sensor module, Neo 6M GPS, a standard chassis, pi fan, a pan tilt camera mount and 2 buck converters.

ab1.PNG

6. Explanation of all the parts and modules

​

(a) Raspberry Pi – The speed and performance of the new Raspberry Pi 4 is a step up from earlier models. For the first time, we've built a complete desktop experience. Whether you're editing documents, browsing the web with a bunch of tabs open, juggling spreadsheets or drafting a presentation, you'll find the experience smooth and very recognizable – but on a smaller, more energy-efficient and much more cost-effective machine. Raspberry Pi has long been the gold standard for inexpensive single-board computing, powering everything from robots to smart home devices to digital kiosks. The Raspberry Pi 4 takes Pi to another level, with performance that’s good enough to use in a pinch as a desktop PC, plus the ability to output 4K video at 60 Hz or power dual monitors.

For the same $35 starting price as prior models, you get speeds that are two to four times faster, support for USB 3 and true Gigabit Ethernet. Perhaps more importantly, there is a $45 Raspberry Pi 4 with 2GB of RAM and a $55 unit with 4GB, four times more than any previous Pi has had. Makers and hobbyists should add the Raspberry Pi 4 to their arsenals, and tech enthusiasts who’ve never used a Pi before now have even more reasons to buy one.

While there were some apps that needed updating at launch time, almost everything that ran on the Pi 3 B works on the Raspberry Pi 4 B. If you have old micro SD cards from a prior Pi build, they may not work, because the Pi 4 B requires Raspbian Buster, the latest version of the platform's official OS. Support from other operating systems such as Ubuntu has also improved.  

However, game emulation is still flagging a bit (see below). And because of the subtle differences in the design (dual micro HDMI ports, USB-C power), any case that was designed to hold the Pi 3 B will not fit.

Because the new SoC needs more power, the Raspberry Pi 4 B charges over USB Type-C instead of micro USB. It also requires a power adapter that can deliver at least 3 amps of power and 5 volts, though you may be able to get away with 2.5 amps if you don’t attach many peripherals to the USB ports. Putting aside the power needs, USB Type-C connectors are reversible, which makes them much easier for kids (and adults) to plug in.

ab2.PNG

(b) Pi Camera – The Raspberry Pi Camera Board plugs directly into the CSI connector on the Raspberry Pi. It's able to deliver a crystal clear 5MP resolution image, or 1080p HD video recording at 30fps! Latest Version 1.3! Custom designed and manufactured by the Raspberry Pi Foundation in the UK, the Raspberry Pi Camera Board features a 5MP (2592*1944 pixels) Omnivision 5647 sensor in a fixed focus module. The module attaches to Raspberry Pi, by way of a 15 Pin Ribbon Cable, to the dedicated 15-pin MIPI Camera Serial Interface (CSI), which was designed especially for interfacing to cameras. The CSI bus is capable of extremely high data rates, and it exclusively carries pixel data to the BCM2835 processor. The board itself is tiny, at around 25mm x 20mm x 9mm, and weighs just over 3g, making it perfect for mobile or other applications where size and weight are important. The sensor itself has a native resolution of 5 megapixel, and has a fixed focus lens onboard. In terms of still images, the camera is capable of 2592 x 1944 pixel static images, and also supports 1080p @ 30fps, 720p @ 60fps and 640x480p 60/90 video recording. The camera is supported in the latest version of Raspbian, the Raspberry Pi's preferred operating system.

The Raspberry Pi Camera Board Features:

  • Fully Compatible with Both the Model A and Model B Raspberry Pi

  • 5MP Omnivision 5647 Camera Module

  • Still Picture Resolution: 2592 x 1944

  • Video: Supports 1080p @ 30fps, 720p @ 60fps and 640x480p 60/90 Recording

  • 15-pin MIPI Camera Serial Interface - Plugs Directly into the Raspberry Pi Board

  • Size: 20 x 25 x 9mm

  • Weight 3g

  • Fully Compatible with many Raspberry Pi cases.

ab3.PNG

(c) Motors – These motors are simple DC Motors featuring gears for the shaft for obtaining the optimal performance characteristics. They are known as Center Shaft DC Geared Motors because their shaft extends through the center of their gear box assembly.

These standard size DC motors are very easy to use. Also, you don’t have to spend a lot of money to control motors with an Arduino or compatible board. TheL298N H-bridge module with onboard voltage regulator motor driver can be used with this motor that has a voltage of between 5 and 35V DC.

This DC Motor – 100RPM – 12Volts can be used in all-terrain robots and a variety of robotic applications. These motors have a 3 mm threaded drill hole in the middle of the shaft thus making it simple to connect it to the wheels or any other mechanical assembly.

Nut and threads on the shaft to easily connect and internally threaded shaft for easily connecting it to the wheels.

These DC Geared motors with robust metal/Plastic gearbox for heavy-duty applications, available in the wide RPM range and ideally suited for robotics and industrial applications.

ab4.PNG

(d) L293D motor driver shields – L293D is a typical Motor driver or Motor Driver IC which allows DC motor to drive on either direction. L293D is a 16-pin IC which can control a set of two DC motors simultaneously in any direction. It means that you can control two DC motor with a single L293D IC.

The l293d can drive small and quiet big motors as well, check the Voltage Specification at the end of this page for more info.

You can Buy L293D IC in any electronic shop very easily and it costs around 70 Rupees (INR) or around 1 $ Dollar (approx. Cost) or even lesser cost. You can find the necessary pin diagram, working, a circuit diagram, Logic description and Project as you read through.

It works on the concept of H-bridge. H-bridge is a circuit which allows the voltage to be flown in either direction. As you know voltage need to change its direction for being able to rotate the motor in clockwise or anticlockwise direction, hence H-bridge IC are ideal for driving a DC motor.

In a single L293D chip there are two h-Bridge circuit inside the IC which can rotate two dc motor independently. Due its size it is very much used in robotic application for controlling DC motors. Given below is the pin diagram of a L293D motor controller.

There are two Enable pins on l293d. Pin 1 and pin 9, for being able to drive the motor, the pin 1 and 9 need to be high. For driving the motor with left H-bridge you need to enable pin 1 to high. And for right H-Bridge you need to make the pin 9 to high. If anyone of the either pin1 or pin9 goes low then the motor in the corresponding section will suspend working.

ab5.PNG

(e) Ultrasonic sensor module – The ultrasonic sensor works on the principle of SONAR and RADAR system which is used to determine the distance to an object.

An ultrasonic sensor generates the high-frequency sound (ultrasound) waves. When this ultrasound hits the object, it reflects as echo which is sensed by the receiver.

By measuring the time required for the echo to reach to the receiver, we can calculate the distance. This is the basic working principle of Ultrasonic module to measure distance.

HC-SR-04 has an ultrasonic transmitter, receiver and control circuit.

In ultrasonic module HCSR04, we have to give trigger pulse, so that it will generate ultrasound of frequency 40 kHz. After generating ultrasound i.e. 8 pulses of 40 kHz, it makes echo pin high. Echo pin remains high until it does not get the echo sound back. So the width of echo pin will be the time for sound to travel to the object and return back. Once we get the time we can calculate distance, as we know the speed of sound.

 

HC-SR04 can measure up to range from 2 cm - 400 cm.

  1. We need to transmit trigger pulse of at least 10 us to the HC-SR04 Trig Pin.

  2. Then the HC-SR04 automatically sends Eight 40 kHz sound wave and wait for rising edge output at Echo pin.

  3. When the rising edge capture occurs at Echo pin, start the Timer and wait for falling edge on Echo pin.

  4. As soon as the falling edge is captured at the Echo pin, read the count of the Timer. This time count is the time required by the sensor to detect an object and return back from an object.

Now, to calculate distance, we know that,

 

The speed of sound waves is 343 m/s.

So,

 

Total distance is divided by 2 because signal travels from HC-SR04 to object and returns to the module HC-SR-04.

(f) Hall sensor module – The Hall Effect Magnetic and Proximity Sensor Module can be used to detect the presence (or absence) of nearby objects such as magnets. This sensor accepts a GND and +5V supply, and has a single digital output which goes low (GND) when a magnetic field is detected.

 

The linear sourcing output voltage is set by the supply voltage and varies in proportion to the strength of the magnetic field. It has a reduced current consumption of only 6 mA typically at 5 V dc help make this sensor energy efficient. Based on the Hall Effect, a hall sensor is a one that varies its output voltage in response to a magnetic field. Hall sensors are used for proximity switching, positioning, speed detection, and current sensing applications.

ab6.PNG

(g) Neo 6M GPS module – This is a complete GPS module that is based on the Ublox NEO-6M. This unit uses the latest technology from Ublox to give the best possible positioning information and includes a larger built-in 25 x 25mm active GPS antenna with a UART TTL socket. A battery is also included so that you can obtain a GPS lock faster. This is an updated GPS module that can be used with Ardupilot mega v2. This GPS module gives the best possible position information, allowing for better performance with your Ardupilot or other Multirotor control platform.

The Ublox NEO-6M GPS engine on this board is a quite good one, with a high precision binary output. It has also high sensitivity for indoor applications. UBLOX NEO-6M GPS Module has a battery for power backup and EEPROM for storing configuration settings. The antenna is connected to the module through a ufl cable which allows for flexibility in mounting the GPS such that the antenna will always see the sky for best performance. This makes it powerful to use with cars and other mobile applications.

The Ublox GPS module has serial TTL output, it has four pins: TX, RX, VCC, and GND. You can download the u-center software for configuring the GPS and changing the settings and much more.

ab7.PNG

(h) Pan tilt camera mount – The robotic Pan & Tilt Camera Mount gives you the power of four cameras in one - making it the ultimate accessory for your XCam2 Video Camera! The Ninja Pan & Tilt allows you to sweep your X10 video camera left, right, up and down - BRAND NEW IN ORIGINAL PACKAGING 2-PACK Scan, Pan & Tilt Your Camra by Remote . X10 Ninja Pan & Tilt Robotic Camera Mount (VK74A) and Ninja ScanPad Controller (CR14A) This auction is not just for the Ninja base but includes the remote also! With the robotic NINJA Pan 'n Tilt Camera Mount with Remote Control (VK74A), you can sweep your camera left, right, up and down - featuring 240° views and up to 4 preset positions! Move your X10 camera to view whatever you want with included ScanPad Remote.

It has hundreds of uses: Front door to baby nursery to backyard pool! See 240° views! Preset up to 4 camera positions. About the Ninja Pan & Tilt Camera Mount The robotic Ninja Pan & Tilt Camera Mount gives you the power of four cameras in one - making it the ultimate accessory for your XCam2 Video Camera! The Ninja Pan & Tilt allows you to sweep your XCam2 video camera left, right, up and down - featuring 240-degree views and up to four preset positions. You can even remotely position the camera to view whatever you want with the included ScanPad Remote. About the Ninja ScanPad Remote Use the Ninja ScanPad Remote to sweep and scan your Ninja Pan & Tilt Camera Mounts from anywhere in your home! You can scan between 4 cameras (on/off) and control the position of each camera that is mounted on a Ninja Pan & Tilt Base. The ScanPad remote sends RF (Radio Frequency) signals to an X10 Transceiver, which passes the signals through your house wiring to control the cameras.

ab8.PNG

(i) Buck converter – DC-DC Buck Converter Step down Module LM2596 Power Supply is a step-down (buck) switching regulator, capable of driving a 3-A load with excellent line and load regulation. These devices are available in fixed output voltages of 3.3 V, 5 V, 12 V, and an adjustable output version. The LM2596 series operates at a switching frequency of 150 kHz, thus allowing smaller sized filter components than what would be required with lower frequency switching regulators.

ab9.PNG

Required Components

  • Raspberry Pi 4 B (I have used Raspberry Pi 4, also tested with Raspberry pi 3 ) (1)

  • Arduino Uno (1)

  • Connecting wires

  • GPS module - Neo 6M (1)

  • 12V 100rpm motor. (4)

  • Ultrasonics sensor - (2)

  • Motor driver board L-293D (2)

  • Wheel bigger size (4).

  • UGV chassis. (1)

  • Battery 12V sunca 2.5Ahr. (1)

  • Camera - pi camera.

  • Camera holder

  • 4G Dongle

  • Hall Sensor 3144 module (1)

  • Small magnets.

   7. Implementation strategy

The UGV performs object detection using OpenCV and TensorFlow Lite. It is trained by using a model to detect certain object and can also be trained to detect certain custom objects. The sample model is optimized to run on the Raspberry Pi and perform object detection while on the road.

Object detection UGV.jpg

   8. Algorithm

TensorFlow Lite is an optimized framework for deploying lightweight deep learning models on resource-constrained devices. TensorFlow Lite models have faster inference time and require less processing power, so they can be used to obtain faster performance in realtime applications. The training is done using a custom TensorFlow Object Detection model, converted it into an optimized format that can be used by TensorFlow Lite, and run it on the Raspberry Pi. TensorFlow v1.13 is used which is a stable version that has great support from Anaconda. OpenCV is not needed to run TensorFlow Lite, but the object detection scripts in this repository use it to grab images and draw detection results on them. Google provides a sample quantized SSDLite-MobileNet-v2 object detection model which is trained on the MSCOCO dataset and converted to run on TensorFlow Lite. It can detect and identify 80 different common objects, such as people, cars, cups, etc. We can also design a custom object detection model by moving the model folder into the directory. We can copy that folder to a USB drive, insert the USB drive in your Raspberry Pi, and move the folder into the corresponding directory and run it on the Raspberry Pi.

   9. Flowchart

Object Detection Flowchart(cropped).jpg

   10. Results

The UGV constructed here can be operated in remote areas without human presence and is capable of object detection using TensorFlow Object Detection API. For creating the object detector quite an amount of data is required, which is used for training.

To train a robust classifier, we need a lot of pictures which should differ a lot from each other. So they should have different backgrounds, random object, and varying lighting conditions.

In order to label our data, we need some kind of image labeling software. LabelImg is a great tool for labeling images. It has been open source on GitHub and the prebuilt can be downloaded easily.

With the images labeled, TFRecords folder is created that can served as input data for training of the object detector.Then a label map and a training configuration file is created. The label map maps an id to a name. Then a training configuration file is created.

Now that we have a trained model we need to generate an inference graph, which can be used to run the model.The Tensorflow Object Detection API allows you to create your own object detector using transfer learning.

   11. Discussion

The Tensorflow Object Detection API makes it easy to detect objects by using pretrained object detection models. TensorFlow Lite is an open source deep learning framework for on-device inference.

UGV being of small size is capable of travelling in limited space areas and provide live video feed and accurate object detection results over the cloud.

   12. Future scope of work

In this paper, a simplified vehicle dynamic model is established, and a modified model predictive controller is proposed using Node-RED. Real-time control performance is being evaluated. The control smoothness is improved by using a web based controller. A novel approach to get local reference trajectory can also be obtained.

bottom of page