At 14:00 on 13th November my monitoring system showed a dramatic change in the form and quality of the subtitles on the BBC News channel. The subtitles went from a mixture of block and snake subtitles to being entirely snake and the distribution of subtitle delay went from -3 to +15 seconds in the period 13:00 to 14:00 to +3 to +7 seconds in the period from 14:00 to 15:00.
Before 14:00
After 14:00
Now I have been monitoring BBC News subtitles, on and off, for several months and have fed some of the worst problems back to the BBC and RedBee, their subtitle provider. I had seen examples of subtitle delay that looked like they were caused by one or more technical faults along with a few incidents of subtitles turning up early were more likely to have been caused by the subtitler. In one case I saw subtitles being buffered to the extent that ended up over 3 minutes late before catching up and another where pre-prepared blocks of subtitles were triggered around a minute early. I was beginning to suspect that there had been some improvements as a result of my feedback, but this recent change is clearly a complete technology refresh that must have been planned well in advance.
The other noticeable change my monitoring has picked up is the subtitles are now more or less verbatim. Where subtitles are generated using respeaking it is generally the case that the maximum word rate that can be achieved is 180 words per minute, often as low as 160 words per minute. In this plot from 13:00 to 14:00 you can see the subtitle word rate is in places well below the speech rate, indicating that the respeaker has left words out
By contrast between 14:00 and 15:00 the subtitles keep up with the speech. The two gaps where there is speech but no subtitles are programme trails which were not subtitled.
All of this indicates that BBC News is now trialling direct speech to text subtitles. There is no official word yet from the BBC, but having watched the subtitles for a while they are pretty impressive. They have managed to retain the colour changes between speakers, which is an improvement over ITV regional news where the speaker change is indicated by a new line beginning with “>>”, though the colour change is not entirely reliable. One other issue with these subtitles is that the punctuation is not as good as that produced by a respeaker, something that could do with more work. On the positive side, while my system cannot directly measure word errors, from what I have watched so far the accuracy is very high, even managing to get names and places correct, which suggests that the system is being fed with script and other helper data.
This is a really interesting development and it will be fascinating to see if there is any response from the audience as the change is quite obvious. There is has been no official statement about this yet, as it currently has the status of a trial, but given that it would appear to have eliminated many other problems it is likely to continue into a full deployment at which point there should be a press statement or blog post.
46