Object Size Measurement

Objective: Inspect Anomalies of size on the Automotive part if it is within tolerance level ( 4.75mm -6.25mm)

Concepts: Camera Calibration -- Zhangs Method, Image Processing, FastFourier Transform, Embeded systems, Camera Optics, TelePhoto lens.

Why: Due to wear and tear of drill bits and/or putting a incorrect drill caused abnormalities in sizes of Tapered holes. Hence, as a 8D request from GM developed a low cost vision inspection system

How:

To link Pixel coordinates of an image point with corresponding points in camera reference, camera internal parameters were obtained using Zhang method
By undistorting each captured image with camera parameters we process image accordingly by selecting ROI and does process image using gradient techiniques
Using discrete Fast Fourier transform convolution we detect circle. To detect sizes, we find number of pixels that fall around cartesian axis and scale it.
Based on Scaling threshold and analyzing with gauge values we determine if size is within our range

Results:

Performing EDA over groundtruth with processed data on log files, the mean average precision(map) of 150 values was around 96% and with a standard deviation of +/- 0.03mm

ID and Object Check

Objective: To Verify if assembled product goes with its corresponding Left Hand/Right Hand pillar and receiver

Software Concepts: SDK, FLIR,GigE,AWS -- S3 Bucket, Textract and Rekognition, OCR, Color Thresholding,Multi Threading

Hardware Concepts: Industrial Camera -- Flir with GigE, Embedded system, Sony CMOS with 25mm Telephoto lens

Why: Due to human interference in the assembly section of pillars,most of times there was a mismatch of ID and missing objects

How:

Using AWS services such as Textract to find out serial name and number on pillar. Followed by, extracting useful information by EDA based on level of confidence and robustness.
Then comparing serial number with coordinates of center values and radius of detected circle on pillar using FFT convolved. Based on inferencing of detected pixel coordinates we analyze ID accordingly
Now fit a circle by selecting ROI to check of Knurled insert -- not included in image.
Using HSV color thersholding we check for screws and washers and count based on predicted contours ( Pose and Illumination is fixed )
Using SDK provided by FLIR, multi thread all the above modules to enhance execution time and provide binaries

Results:

Overall execution was successful and improved error proofing of production parts

Absence/Presence Knitting

Objective: Check for different parts on Handle of Corvette and Hummer including Slider Bearing, Pin, Buggy Jumper and Bushing using Object detection and Semantic Segmentation using MaskRCNN

Concepts: Deep learning, Tensorflow, TensorRT, Google collab, Data Augumentation,Data Annotation,GPU,CUDA.

Why: To verify the presence of multiclasses on production handles on live video inferencing on conveyor belt

Why MaskRCNN:

MaskRCNN process image pixel-pixel: This has an advantage over other models when dealing with small size objects over a darker background in our case Pins
Has multi-layer inferencing (RPN) where probability of false negatives would be lower over other models
Uses FPN which means MAP value is higher due to availability of low level and high level features across all pixels
Since instances of annotation on some images with different pose overlap each other -- pixel based classification can perform better to segment each other

How:

Colllected data of around 234 images and was split across 70/10/20 for Train,val and test data
Created transform version of images as part of data augumentation to enhance model accuracy
Annotated images using VGG Image annotator as set of Polygon points in JSON
Used standard CNN ResNet architecture to train model for every 50 echos on 100 steps with various batch sizesuntil cross entropy loss was less than 0.2.
Trained and Infered on images,video data and live inferencing both on GPU as well as CPU to check latencies.

Results:

Obtained overall loss value of 0.1904 and performed well on unlabelled data. However, latency is one of major issue in order to deploy this model onto an embedded system.
Inferring on a single image took a average execution time of 14 seconds on CPU and at least 4 seconds on GPU when tested on google collab.
Not compatible with most embedded systems due to older versions of tensorflow and tensor gpu which are not available any more

Objective: Check for different parts on Handle of Corvette and Hummer including Slider Bearing, Pin, Buggy Jumper and Bushing by object detection using YOLOV5

Concepts: Deep learning, PyTorch, EC2 instance, Data Augumentation,Data Annotation,GPU,CUDA

Why: To verify the presence of multiclasses on production handles on live video inferencing on conveyor belt

Why YOLO:

Takes entire image as a single segment and process it, unlike considering pixels based on ROI in RCNN
Very accurate on embedded systems with a proven framework on embedded systems around 45 FPS
Uses grid based approach and predicts bounding boxes and corresponding class probabilities
Utilizes NonMaxSuppresion and IOU to detect prediction of detected Boundingboxes

How:

Collected data and split across 80/20/10 across train, validation and test
Annotated data for multi classes and performed data augumentation to balance data of all classes
Used API service to extract data from roboflow API to extract data onto local and cloud servers
Trained on different batch sizes and hyper parameters to improvise performance on DL model
By analyzing F1,precision and confusion matrix metrics--deployed a model onto production line to infer on parts going through conveyor
Performing EDA on obtained data and stored logs with confidences,class id and images

Results:

Achieved a FPS in range 10-12 after testing on embedded system with a MAP of 98.43%