Artificial Intelligence-Aided Detection of Railroad Trespassing
Understanding how modern digital resources interplay with human activity is essential for creating solutions that are not only productive, but empathetic in their design. Rail transportation in particular is a prime target for safety redesign.
Trespassing — the leading cause of rail fatalities — has dramatically increased over the last 10 years. Over this same period, the quantity of videos possessed by rail organizations has simultaneously grown, capturing many aspects of the rail system (passing automobiles, space around tracks, crossing closings, etc.). However, this video data lacks efficient utilization due to the massive effort it takes to manually monitor the feeds or review them afterward.
In order to make effective use of this video data, my teammates and I leveraged an artificial intelligence to systematically review, label, and extract useful information such as the date, time, and type of trespassing event, as well as identifying whether the trespasser is a pedestrian or vehicle. When delivered to rail agencies, this database of information allows for empirical analysis of rail safety measures in order to make precise engineering recommendations and monitor the effectiveness of safety treatments (for example, a crossing experiencing a statistical overabundance of vehicle trespasses could record the impact of a new boom gate in order to maximize the safety return on investment).
In order to prevent deaths and injuries, it is imperative to understand what happens in the context of these life threatening situations.
—
The algorithm combines an ROI with a R-CNN.
The ROI (Region of Interest) is a user-set delineation of boundaries within the camera’s field of vision. Defining the ROI allows the AI to claim a trespassing event when an object enters the defined restricted area.
Mask R-CNN (Regional Convolution Neural Network) is a neural network designed for object recognition. The objective of the CNN is to distinguish objects within their real-world contexts by using collective multiple feature abstraction levels. Each level is arranged within the network to accomplish specific functions, namely the convolutional layer, rectified unit layer, and pooling layer. The convolutional layer searches for features (such as colors, forms, shapes, or other classifying information) in the visual field. The rectified unit layer (ReLU) removes all elements that do not match with the preexisting features that the program is looking for, creating an image free of non-essential materials. The pooling level amalgamates each of the feature identifying layers to collectively create a single, recognizable image. As this process is repeated, the neural network is able to continually learn and improve recognition of objects. In the case of trespassing, Mask R-CNN is able to learn how to visually identify subjects such as cars, trucks, trains, and people within video footage for the user.
Currently, the algorithm is able to use both of these computer vision techniques to sufficiently relay information. However, because the visual data still presents inaccuracies related to recognition and whether or not an object is truly trespassing the ROI, the footage needs to be manually reviewed by a human in order to validate that the system is outputting factual information about the footage. This data must be validated as rail agencies demand fully accurate information to inform future decisions about rail safety, engineering, research, and design.
While the labor associated with analyzing data has certainly been trimmed down as a result of the AI, manually validating each video set continues to be an arduous process. With an accurate and automated AI system, compiling the data and presenting trespassing events in consumable formats will ensure only correct information gets relayed at a much faster pace than manually reviewing each event.
—
In order to curb common mistakes made by the AI, three detectors were established to ensure trespassing events are indeed true.
False Signal Detector:
A False Signal event occurs when the AI registers trespassing without the presence of a crossing stoplight. In a true trespassing event, the crossing signal will rapidly flash red light. The False Signal algorithm searches for changes in “redness” during the video. If the rate of change does not pass a certain limit, the AI will mark the occurrence as a False Signal.
Duplicate Detector:
A Duplicate is recorded when the detection model both loses track of an object and subsequently regains it, causing the singular event to be labeled as two separate instances of trespassing. The Duplicate Detector internally compares the trajectory of objects as well as similarity of frames between two videos. Analysis of these factors calculates the probability of a true duplicate. If the probability value meets a specific threshold, the videos are labeled as duplicates.
Not In ROI Detector:
During a Not In ROI incident, an object is outside the designated ROI but is identified as trespassing. If this error occurs, the detector evaluates the trajectory, direction, and type of the object to classify the trespassing signal as false.
—
Additional improvements to the algorithm are continually being made to determine a slew of different validation errors. Alongside these developments, my teammates and I hope to build a web application that presents the viewer with a data visualization produced via an uploaded CSV. A well designed chart allows for observers to easily recognize trends present in data.
Through the use of AI detection, surveillance of railway crossings can be substantially improved as the processed footage can display not only what happens before an accident, but who it happens to, how it happens, and the patterns that are created through these observations.
—
Firstly, we acknowledge our PI Asim Zaman for his help in directing and organizing our team’s efforts. We are also grateful to our advisors Huixiong, Zhe, and Dhiraj for their guidance in our algorithm development. Thank you to our non-Aresty peers for their continued support. Finally, a very special thank you to Professor Liu, for opening up this opportunity for us.