Contextual Driving Platform (CDP) ? 2017 InterDigital, Inc. All Rights Reserved. Introduction ? With technology advances, cars are and will continue to be outfitted with hundreds of outward sensors including cameras, radars, LiDARs, GNSS, and V2X that collect information about their environment. ? Cars are getting transformed to become mobile sensors that that are continuously collecting terabytes of data per day about their driving conditions, road conditions, the weather and the behavior of the surrounding cars. ? As cars become more and more autonomous, the software in the car needs to replace human senses and perceptions with automated means to continuously assess, perceive and respond to risks on the road. ? The system aims to demonstrate cooperative sensor data fusion techniques based on data from on-board sensors and V2X communications for situational awareness. ? 2017 InterDigital, Inc. All Rights Reserved. Contextual Driving Platform (CDP) ? Each car has an V2V sensor fusion component that continuously requests and receives contextual sensor input data from a network of vehicles in the vicinity. ? CDP in each car aggregates the individual sensor data from neighboring cars to build a contextual awareness of environment. ? Based on the assessed risk of the environment, the CDP instructs the self- driving car to drive more conservatively or aggressively. ? e.g. reduce speed, increase inter-vehicle distance between cars ?2017 Innovation Partners Technology Under the Hood: Contextual Driving Platform (CDP) ? CDP iteratively computes the risk profile of each car (?risk score?) based on driving and sensor performances, and adjusts the self-driving parameters based on aggregated risk score. ? Risk scores are based on sensor measurements and include speed, acceleration, inter-vehicle distance, lane departures, and contact with others or side barriers. ? Risk scores are exchanged between vehicles and with the cloud server entity over V2X communication. Data Fusion Module 1 Data Fusion Module 2 Data Fusion Module 3 Sensor 1 Sensor 2 Sensor 3 ? 2017 InterDigital, Inc. All Rights Reserved. Self Driving Algorithms for Improved Safety 1. Lane Change Control: The CDP self driving client will detect the presence of a neighboring car(s) and prohibit the car from changing lanes. 2. Collison Control: The CDP self driving client will detect if the car is rapidly approaching a car or obstacle ahead and either guide the car to change lanes or reduce the speed of the car. 3. Speed Control: The CDP self driving client will asses the risk level of the car and adjust speed accordingly. For example, if the risk score is high, the CDP will reduce the speed of the car. 4. Inter-Vehicle Distance Control: The CDP self driving client will process V2V messages and adjust distance between human driven cars and CDP self driving client in order to optimize risk. ? 2017 InterDigital, Inc. All Rights Reserved. SENSOR LAYER ON-BOARD PROCESSING & CROSS-VALIDATION V2Cloud CONTEXTUAL DRIVING PLATFORM CLOUD VIDEO CAMERA RADAR POSITION ESTIMATOR V2V (CO-OP SENSING) ULTRASONIC SENSOR GPS LiDAR Offline Data Fusion Cloud Computing Online Map Data Cloud Streaming Overall System Architecture ? 2017 InterDigital, Inc. All Rights Reserved. Building Blocks APPLICATION PLATFORM Situation awareness ? Positioning ? Objects ? Landmarks ? 360? view model Risk assessment ? Critical Objects ? Crash mitigation strategy Trajectory planning ? Optimal driving pathV2X SUPPORT? Cars ? Infrastructure ? ITS service PERCEPTION/ DATA FUSION Object detection Dynamic maps Trajectories SENSOR LAYER Cameras LiDAR Radars Inertia Unit GPS ? 2017 InterDigital, Inc. All Rights Reserved. Demonstration ? 2017 InterDigital, Inc. All Rights Reserved. Session: ? Mission: Drive a RC car safely and avoid obstacles along a pre-determined route in the track ? Duration: 60 seconds ? Goal: Minimize the risk score Score card: ? The CDP client running in each car is continuously assessing the risk score of cars in its vicinity. As the player drives more aggressively, the other cars detect less spacing and gaps, elevating the risk it is to other cars on the track. ? The leadership board ranks the lowest aggregate risk score, which is sum of risk scores calculated by all the vehicles on the track. Driving Experience Using Virtual Reality Seated in the car Top view ? The demonstration was built using Unity Game Engine and rendering is with the Virtual Reality headset. ? The movements of the cars are carefully calibrated and replicated by overhead OptiTrack cameras to replicate the actions of virtual self driving cars and physical human driven cars. VISUALIZATION USING VIRTUAL REALITY ? Bubble domes over each car indicate the assessed risk at each vehicle. ? Red dome: High risk ? Yellow dome: Medium risk ? Green dome: Low risk ? Lines indicate cars in communication with each other ? Green dots ? V2V messages between cars ? 2017 InterDigital, Inc. All Rights Reserved. 1. 4X8 foot board with three tracks. 2. 2 driven by the human and 4 driven by the AI (self driving cars). Demo Setup Physical Track Scoreboard Human Driven Car AI Driven Car ? 2017 InterDigital, Inc. All Rights Reserved. RC Driven Car Steering Wheel & Pedal Driven Car Contextual Driving Platform Algorithm ? 2017 InterDigital, Inc. All Rights Reserved. Fig 6a - Car 3 (gold) has first detected Car 4 (red) in sensors & assigns it initial risk score (P) of 1. Fig 6b - Car 4 (red) propagates its risk score to the five cars near the red car. Car 3 (gold) also computes encroachment of red car, and assigns it new value of P (3) and sends as a reply. Fig 6c - Each car sums the received risks and propagating a new P to all of its neighbors, reinforcing the risk with in the group. Car1 Car2 Car3 Car4 Car5 Car6 0 0 0 0 0 0 0 0 1 0 0 0 1 1 4 2 1 1 4 4 10 8 4 4 10 10 20 20 10 10 20 20 35 40 20 20 35 35 56 70 35 35 56 56 84 112 56 56 84 84 120 168 84 84 120 120 165 240 120 120 165 165 220 330 165 165 Car 2 Car 1 Car 3 Car 4 Car 5 Car 6 Car 2 Car 1 Car 2 Car 3 Car 4 Car 5 Car 6 Car 6 Car 1 Car 2 Car 3 Car 4 Car 5 ? 2017 InterDigital, Inc. All Rights Reserved.