It will be a shorter and less technical version than the one for IBC2025. The IBC paper is aimed at the broadcast engineering community, while the DARCI presentation is aimed at accessibility experts. You can attend DARCI in person or on-line and registration is open on their web site https://enhancingaudiodescription.com/darci-conference. I am speaking in the 10:45 session on the 11th September, Paper Session 1B: Sign Languages and Captioning in the Creative Industries.
We’re all used to hearing middle class parents complain about the behaviour of their teenage offspring, from the state of their bedroom to the way they talk or the fact they are always after money. However, there is a new complaint I’m hearing these days, even turning up on Radio 4 news programmes. Teenagers insist on turning the TV subtitles on. Infuriated parents are having to learn where the “subs” button on the remote control in order to turn them off and arguments break out if they turn them off whilst their offspring are watching.
This is not some moral outrage story, cooked up by the tabloid press as part of the ongoing culture wars, it is a very real phenomenon. Moreover, it is not confined to teenage children, but younger adults too. A recent YouGov poll confirmed something we’ve been seeing signs of for several years. This poll shows that 61% of 18-24 year olds prefer to watch TV with the subtitles turned on and while it doesn’t give data for people under 18 the level of subtitle use in the younger age group is probably higher. Other, surveys also indicate a clear trend of increasing preference for subtitles with decreasing age. These trends are not new, iPlayer data from 2016 showed subtitle usage of around 30% on children’s services rising to 35% on content classified as learning. Seven years later it is no surprise to see these trends continuing to grow. Meantime in the field of computer games the usage of subtitles is even more widespread with usage figures for subtitles in major computer games of 95% and higher.
So, what is going on? Why are young people so enamoured with subtitles? The short answer is we don’t know. There is a lack of published research on this phenomenon and while broadcasters are interested in answering this question, there hasn’t been sufficient motivation for them to fund further investigation. Part of the problem being that everyone seems to have their pet theory on the topic, so they don’t see it as warranting further expenditure.
Personally, I think that subtitle use has to be seen as part of a much wider set of cultural changes that have taken place in the past 30 years which have changed the relationship between people, media and communication in general. Let’s start with television itself. What has changed in the past 25-30 years to take us from a situation where subtitles were seen as something for older people and those with hearing difficulties to something that is now a majority preference amongst younger audiences?
When I was a teenager the only subtitles available on television were ones that were burnt into the picture. These were provided for foreign language programmes and occasionally for programmes aimed at the Deaf and hard of hearing. Teletext subtitles didn’t arrive on UK Television until 1979, and TV sets that displayed Teletext were very expensive and very few programmes had subtitles, all of which were pre-recorded. Live subtitles didn’t arrive until 1984 and it was only with the arrival of digital television at the end of the 1990s that the ability to display subtitles became mandatory in all new receivers. The ITC (now Ofcom) was given the duty to set for minimum levels of subtitle provision for broadcasters thanks to rebel amendments to the Broadcast Act of 1996 which also mandated minimum levels of audio description and sign language. Quotas for subtitling increased over the years with the BBC unilaterally achieving 100% subtitling for all its main television channels by 2008 and mirroring this on iPlayer by 2012. The result is that for anyone under the age of 24 there have almost always been subtitles available on television. Unlike older generations they have grown up with subtitles, whether they used them or not. The question then is why would they chose to use them?
Another technology that may have had a big impact is the world wide web which became available as a royalty-free technology in 1993. The web, like books and newspapers, was initially text based with images, sound and video coming later. At the same time SMS text messaging became part of the digital mobile phone services and a cheaper method of communicating short messages than phone calls. From these technologies come the smart phones, aps and near ubiquitous connectivity we see today. These, along with the arrival of social media apps, change the way people communicate with each other. Rather than making a phone call, communication is more often text based, giving the advantage of being both asynchronous and recorded. In the past a phone call would rudely interrupt whatever you are doing, and you would have had to write down any important information, but now a text message can be attended to later and the information saved for later use.
As a result, a whole generation has now grown up exchanging text messages and has also become used to handling multiple different exchanges interleaved with each other. There is something unnerving about the way I have seen younger colleagues able to continue typing a report whilst holding a face-to-face conversation. The ability to process more than one stream of language simultaneously is something I definitely don’t have. So how does that relate back to subtitles? Well, unlike speech which vanishes as soon as it is uttered, subtitles stay onscreen for a period of time and can be glanced at if you miss a word or its context. This enables you to switch attention between a TV programme and other things going on like a conversation, text messages or a game. Research is finally catching up with these issues and I hope to see the results published soon.
So as far as I can see subtitles are here to stay as part of our media landscape. Other issues such as not wanting to wear headphones, watching video with the sound turned off also contribute to their popularity amongst the population as a whole. Furthermore, recent improvements in the quality of text to speech has meant that almost any video can have useful, if not entirely accurate, subtitles mean they are now available for anyone posting video content.
The RNID’s 2023 report, “Subtitle It,” highlights the significant challenges viewers face in accessing subtitles through on-demand platforms. While the Media Act 2024 mandates minimum quotas for subtitle provision on “tier 1” services, subtitle quality remains an issue which impacts accessibility and viewer experience.
While AI-based speech-to-text tools cannot provide broadcast-quality subtitles, because they produce different types of errors, they can be used to monitor some of the problems which affect the quality of television subtitles and degrade the audience experience. This presentation will demonstrate how, using a modified version of Whisper, OpenAI’s speech-to-text engine combined with natural language processing and a simple statistical approach, we can usefully quantify problems with timing and word omission in subtitles in broadcast and on-demand content.
We will show how these problems vary across different types of TV programming, including archive programmes with original subtitles that omit a substantial portion of the spoken words and live programmes, where subtitlers re-speak the dialogue or manually cue pre-prepared blocks, leading to subtitles that (usually) lag the speech along with the omission and reordering of words.
These problems will be illustrated with examples from broadcast television and on-demand content, including technical faults and workflow issues. The examples will also highlight the challenges of aligning subtitles with a speech-to-text transcript, given that this work has revealed examples of subtitles omitting around 40% of the spoken words and subtitles arriving between 20 seconds early and 50 seconds late as well as an example of a programme broadcast with the subtitles for a different episode.
We will conclude with some observations on current practices and historical trends in TV subtitling and discuss the need for improved quality control and monitoring of subtitles provided for broadcast and on-demand programmes.
“Using generative-AI speech-to-text output to provide automated monitoring of television subtitles”
My latest paper will be published as part of the IBC Technical Papers conference. This paper will cover the technical details of a research project that uses speech to text software to track timing and word omission in television subtitles.
This is the outcome of several months of independent research carried out in association with Dr Michael Crabb, Head of Computing at the University of Dundee and the UK Subtitling Audiences Network.
The proof of concept software can plot timings for individual subtitles where a match is found between the speech to text transcript and the first word of a subtitle. The resulting chart can give a feel for the amount of delay across the recording.
The software can also plot the number of words in each minute for the transcript and subtitles to give an indication of the proportion of words missing from the subtitles.
While the technique cannot be 100% accurate it gives a good indication of the quality of the subtitles and direct attention to where faults have occurred.
