Research & Development

AI innovation in speech technology and beyond

At Speechmatics, our relentless pursuit of innovation ensures unparalleled accuracy and broad language inclusivity, redefining industry standards.

SSL image
Self-Supervised Learning

Expanding accuracy through innovation

Enabling autonomous learning to enhance our understanding

At Speechmatics, self-supervised learning (SSL) serves as a transformative approach in training speech models, harnessing unlabeled data to enhance our speech recognition systems.

This technique allows us to autonomously identify patterns in vast amounts of data, significantly expanding the diversity of speech variations our models can learn from and improving accuracy across multiple languages.

Tirelessly pushing speech technology forward...

Throughout the years, Speechmatics has remained at the forefront of speech recognition research and innovation. We are consistently pushing the boundaries of what's possible in speech-to-text technology.

2015
Our groundbreaking research, as detailed in the paper Scaling Laws for Neural Language Models, explores how the performance of neural language models improves predictably with scale. By analyzing models ranging from small to very large, we established key scaling laws that guide the efficient design and training of advanced language models, leading to significant enhancements in their accuracy and capabilities.

  • FastCompany
  • National Innovation Awards
  • Go:Tech Awards

Our published research

Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought

Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought

James Chua, Edward Rees, Hunar Batra, Samuel R. Bowman, Julian Michael, Ethan Perez, Miles Turpin. March 8, 2024.
While chain-of-thought prompting (CoT) has the potential to improve the explainability of language model reasoning, it can systematically misrepresent the factors influencing models' behavior--for example, rationalizing answers in line with a user's opinion without mentioning this bias. To mitigate this biased reasoning problem, we introduce bias-augmented consistency training (BCT), an unsupervised fine-tuning scheme that trains models to give consistent reasoning across prompts with and without biasing features.Read the paper in full

Debating with More Persuasive LLMs Leads to More Truthful Answers

Debating with More Persuasive LLMs Leads to More Truthful Answers

Akbir Khan, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Edward Grefenstette, Samuel R. Bowman, Tim Rocktäschel, Ethan Perez. February 9, 2024.
Common methods for aligning large language models (LLMs) with desired behaviour heavily rely on human-labelled data. However, as models grow increasingly sophisticated, they will surpass human expertise, and the role of human evaluation will evolve into non-experts overseeing experts. In anticipation of this, we ask: can weaker models assess the correctness of stronger models?Read the paper in full

Hierarchical Quantized Autoencoders

Hierarchical Quantized Autoencoders

Will Williams, Sam Ringer, Tom Ash, John Hughes, David MacLeod, Jamie Dougherty. February 19, 2020.
Speechmatics’ paper was submitted and accepted to the most prestigious ML conference – NeurIPs. The paper is about a type of lossy image compression algorithm based on discrete representation learning, leading to a system that can reconstruct images of high-perceptual quality and retain semantically meaningful features despite very high compression rates.Read the paper in full

Texture Bias Of CNNs Limits Few-Shot Classification Performance

Texture Bias Of CNNs Limits Few-Shot Classification Performance

Sam Ringer, Will Williams, Tom Ash, Remi Francis, David MacLeod. October 18, 2019.
Speechmatics published the paper at NeurIPS 2019 presenting in the meta-learning workshop.Read the paper in full

Technical spotlight

Technical

Sparse All-Reduce in PyTorch

The All-Reduce collective is ubiquitous in distributed training, but is currently not supported for sparse CUDA tensors in PyTorch.

In the first part of this blog we contrast the existing alternatives available in the Gloo/NCCL backends.

David MacLeodMachine Learning Architect
Speech Intelligence

The.Shed: Speechmatics Capabilities in real-time

Imagine being able to understand and interpret spoken language not only retrospectively, but as it happens. This isn't just a pipe dream — it's a reality we're crafting at Speechmatics.

Our mission is to deliver Speech Intelligence for the AI era, leveraging foundational speech technology and cutting-edge AI.

Aaron NgMachine Learning Engineer
Technical

An Almost Pointless Exercise in GPU Optimization

Not everyone is able to write funky fused operators to make ML models run faster on GPUs using clever quantization tricks. However lots of developers work with algorithms that feel like they should be able to leverage the thousands of cores in a GPU to run faster than using the dozens of cores on a server CPU.

To see what is possible and what is involved, I revisited the first problem I ever considered trying to accelerate with a GPU.

Andrew InnesChief Architect

Be part of our mission

We are actively seeking talented individuals to join our collective team of ambitious, problem solvers and throught-leaders, paving the way for inclusion in speech recognition technology.