By Dominic Basulto
Computers have already been trained how to write up quick summaries of sporting events after studying the box score, putting the jobs of sports journalists at risk. Now comes the next step: live commentary of professional sporting events as they actually happen, thanks to a combination of machine learning and computer vision.
In India, for example, computers are able to provide text-based commentary of cricket matches with 90 percent accuracy, according to a recently published paper by a group of three Indian researchers, Rahul Anand Sharma and C.V. Jawahar - scientists at the International Institute of Information Technology, Hyderabad - together with Pramod Sankar Kompalli of Xerox Research Center India.
To make that possible, computers analyzed hundreds of hours of cricket videos from the YouTube channel for the Indian Premier League, breaking them into categories based on text descriptions of them that were already available. The next step was breaking down these longer videos into smaller scenes in order to classify each video shot. Then a computer algorithm had to find the right commentary that matched up with what was being shown in each video shot. For that, the researchers used commentary for about 300 matches that already existed in the ESPN Cricinfo database.
As a result, computer algorithms were able to accurately label a batsman's cricketing shot by using visual-recognition techniques for an action that sometimes lasted no more than 1.2 seconds. "To learn such a representation, several examples are needed. A computer program then learns from these examples using machine learning algorithms, and tags parts of the video with these labels," Jawahar told India's NDTV.
For now, NFL and NBA fans in the United States are in the clear - there is no immediate risk that a machine will be able to make sense of the manic action of 5-on-5 or 11-on-11 sporting events. A computer would struggle to make sense of a botched Michigan punt returned to the end zone in the final seconds by Michigan State or the bizarre sequence of events in the final seconds that led to Miami defeating Duke. And forget about it if athletes are wearing "Color Rush" uniforms - a machine that has been trained to recognize certain shapes and colors might just throw in the towel before the start of the game.
However, there is one sport that might be open to real-time computer analysis sooner than you think - tennis. After all, a tennis ball is fairly easy to track on a court, and computers that are able to determine exactly where a ball bounces are a routine part of any live tennis broadcast. Plus, there are only two players, so there is less action to monitor. Other than a few blazing 100 mph serves and a few furious serve-and-volleys, what is so hard, really, about calling a tennis match between two baseline duelers?
The same Indian researchers, for example, have analyzed how a machine might be able to break down the action of a tennis match. In a paper presented earlier to British artificial intelligence researchers, Jawahar and Mohak Sukhwani showed how algorithms could be used to provide text commentary on tennis matches.
Of course, we are still a long way from a live sports broadcast completely narrated by a machine. The Indian researchers see their technique as a way to help out reporters and journalists. And indeed, the Associated Press recently signed up to experiment with computers providing written description of college sporting events.
As machines become ever faster and ever more adept, however, it is possible to think in terms of what machine learning means for the "play-by-play guy" and the "color commentator" - AI probably gets the announcer first, the analyst second. After all, it's harder to replicate so much human knowledge on the fly - a machine may be able to analyze whether Roger Federer just hit a forehand or a backhand, but may not know the full back story of how he prepared for the match or anything about his current physical condition.
However, there is one advantage that machines have that humans do not - the ability to process views of the action from various angles at once. How many times have you watched a sports broadcast in which the broadcasters had to watch - and re-watch - a play before coming up with a definitive assessment about what just happened? Machines able to pull in multiple video feeds from multiple cameras at once might not have that problem.
For now, the play-by-play ability of machines is not expected to take away any jobs. The focus is on using this AI technology for training and coaching. According to the researchers, the annotation of the videos allows computers to search across hundreds of hours of content for specific actions that last only a few seconds.
As a result, imagine preparing for a tennis match with a rival and having an AI companion that can immediately make sense of your opponent's playing style and search for a specific stroke on a specific part of the court - within seconds. Watching hours of tape is already a staple in the professional sports world - but now it appears that machines are becoming just as proficient as humans at making sense of all the nuances hidden in that tape.
For journalists on the sidelines, that could one day mean by the time they have cracked open their laptops, their machine rivals may have already written and posted their article online.
The Washington Post
By Dominic Basulto
Computers have already been trained how to write up quick summaries of sporting events after studying the box score, putting the jobs of sports journalists at risk. Now comes the next step: live commentary of professional sporting events as they actually happen, thanks to a combination of machine learning and computer vision.
In India, for example, computers are able to provide text-based commentary of cricket matches with 90 percent accuracy, according to a recently published paper by a group of three Indian researchers, Rahul Anand Sharma and C.V. Jawahar - scientists at the International Institute of Information Technology, Hyderabad - together with Pramod Sankar Kompalli of Xerox Research Center India.
To make that possible, computers analyzed hundreds of hours of cricket videos from the YouTube channel for the Indian Premier League, breaking them into categories based on text descriptions of them that were already available. The next step was breaking down these longer videos into smaller scenes in order to classify each video shot. Then a computer algorithm had to find the right commentary that matched up with what was being shown in each video shot. For that, the researchers used commentary for about 300 matches that already existed in the ESPN Cricinfo database.
As a result, computer algorithms were able to accurately label a batsman's cricketing shot by using visual-recognition techniques for an action that sometimes lasted no more than 1.2 seconds. "To learn such a representation, several examples are needed. A computer program then learns from these examples using machine learning algorithms, and tags parts of the video with these labels," Jawahar told India's NDTV.
For now, NFL and NBA fans in the United States are in the clear - there is no immediate risk that a machine will be able to make sense of the manic action of 5-on-5 or 11-on-11 sporting events. A computer would struggle to make sense of a botched Michigan punt returned to the end zone in the final seconds by Michigan State or the bizarre sequence of events in the final seconds that led to Miami defeating Duke. And forget about it if athletes are wearing "Color Rush" uniforms - a machine that has been trained to recognize certain shapes and colors might just throw in the towel before the start of the game.
However, there is one sport that might be open to real-time computer analysis sooner than you think - tennis. After all, a tennis ball is fairly easy to track on a court, and computers that are able to determine exactly where a ball bounces are a routine part of any live tennis broadcast. Plus, there are only two players, so there is less action to monitor. Other than a few blazing 100 mph serves and a few furious serve-and-volleys, what is so hard, really, about calling a tennis match between two baseline duelers?
The same Indian researchers, for example, have analyzed how a machine might be able to break down the action of a tennis match. In a paper presented earlier to British artificial intelligence researchers, Jawahar and Mohak Sukhwani showed how algorithms could be used to provide text commentary on tennis matches.
Of course, we are still a long way from a live sports broadcast completely narrated by a machine. The Indian researchers see their technique as a way to help out reporters and journalists. And indeed, the Associated Press recently signed up to experiment with computers providing written description of college sporting events.
As machines become ever faster and ever more adept, however, it is possible to think in terms of what machine learning means for the "play-by-play guy" and the "color commentator" - AI probably gets the announcer first, the analyst second. After all, it's harder to replicate so much human knowledge on the fly - a machine may be able to analyze whether Roger Federer just hit a forehand or a backhand, but may not know the full back story of how he prepared for the match or anything about his current physical condition.
However, there is one advantage that machines have that humans do not - the ability to process views of the action from various angles at once. How many times have you watched a sports broadcast in which the broadcasters had to watch - and re-watch - a play before coming up with a definitive assessment about what just happened? Machines able to pull in multiple video feeds from multiple cameras at once might not have that problem.
For now, the play-by-play ability of machines is not expected to take away any jobs. The focus is on using this AI technology for training and coaching. According to the researchers, the annotation of the videos allows computers to search across hundreds of hours of content for specific actions that last only a few seconds.
As a result, imagine preparing for a tennis match with a rival and having an AI companion that can immediately make sense of your opponent's playing style and search for a specific stroke on a specific part of the court - within seconds. Watching hours of tape is already a staple in the professional sports world - but now it appears that machines are becoming just as proficient as humans at making sense of all the nuances hidden in that tape.
For journalists on the sidelines, that could one day mean by the time they have cracked open their laptops, their machine rivals may have already written and posted their article online.
The Washington Post
