We’ve all been there: a light turns green and the car in front of you doesn’t budge. No one likes to get stuck behind a vehicle that doesn’t notice when a light changes. Also, a system that can countdown on red light the time remaining until a change to green can save a significant quantities of fuel in city driving (e.g., restart engine five seconds before green) and advise driver to start braking early if it will not make it through a green light. That’s where you get into the picture: in this competition, you will develop a model to recognize traffic-light state in the car driving direction.
Nexar will provide the participants with two datasets, training and testing (dataset was splitted randomly to train and test).
The training-set consists of 18,659 labeled images for training and validating the CNN model (i.e., train and validation set) with the following labels:
The testing-set consists of 500,000 unlabeled images for testing the final solution. Test-set will be available to download by the submission server open date (see section Timeline).
The participants will build a CNN model using the training-set, use it to predict on the testing-set, and create a file with predicted labels for test-set. More specifically, for each image in the test set, participants will predict a label for its id (0=no traffic light in driving direction, 1=Red light in traffic direction, 2=Green light in traffic direction) and the probability of each class -> (image_fname, label_id,p0,p1,p2) .
Performance will be evaluated on classification accuracy (i.e., percentage of correctly labeled images in test-set) while taking into consideration the trained model size to prefer networks with lower model size.
Classification accuracy will be calculated as follows:
classification_accuracy = number of correctly labeled images/number of predictions
Minimum success criteria for submission acceptance: 0.95 (i.e., 95%) classification accuracy.
Then, normalized model size score will be calculated as follow:
model_size_score = exp(-model_size_mb/100)
Finally, challenge score will be calculated as follow:
challenge_score = classification_accuracy*model_size_score
For example, a straightforward fine-tuning of the GoogLeNet to the Nexar train-set using Caffe yielded test classification accuracy of 0.93 (i.e., 93%) with 41MB model size and challenge_score of 0.6172 .
All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. Nexar challenge organizers reserve the right to update the contest timeline if they deem it necessary.