Decoding intended speech is one of the central focuses of BCI research and product development in recent years, with several companies developing implantable devices to restore communication for people with conditions such as ALS. While these implants achieve impressive accuracy, the need for brain surgery limits their scalability. Decoding speech from non-invasive recordings of neural activity avoids these risks. However, non-invasive systems have historically struggled to decode language reliably.
On the of 29th June, Meta introduced Brain2Qwerty, the latest version of its non-invasive brain-to-text system. The model, which uses MEG recordings as input, achieved an average word accuracy of 61%, a notable leap from previous non-invasive approaches. The recordings were done in collaboration with the Basque Centre on Cognition, Brain and Language. Brain2Qwerty v2 was released as an arXiv preprint, alongside the publication of Brain2Qwerty v1 in Nature Neuroscience, on the same day.
Meta’s brain-to-text system, Brain2Qwerty, is capable of decoding natural sentences from MEG data with an average word error rate (WER) of 39%. For the best participant, WER was as low as 22%, a more than two-fold improvement from v1’s best participant (WER 52%). v2 was built on a larger dataset than v1, and Meta observed no performance plateau as training data increased. This suggests further improvements are possible as datasets scale.
To train the system, nine participants were recorded using MEG whilst actively typing sentences that were read aloud to them. Each participant was recorded for ten hours in total, resulting in a dataset containing the neural signals produced from 22,000 typed sentences. Meta fed these recordings into an end-to-end learning model.
Brain2Qwerty comprises a three-stage neural network, decoding characters, words, and sentence-level representations, respectively. An LLM leverages semantic context to improve word accuracy if data is noisy, helping to circumvent a major drawback of non-invasive recordings. Unlike earlier systems that decoded individual keystrokes within fixed time windows, Brain2Qwerty analyses continuous neural activity across an entire typing sequence.
The brain-to-text model is not currently intended for use as a commercial product. Senior researcher Jean-Rèmi King describes commercialising the system as ‘too difficult’. Instead, Meta has open-sourced the model code and training data through its Digital BrainProject, allowing other research groups to replicate and build on the work.
Implantable BCIs have transformed speech restoration, delivering high accuracy and real-time decoding of attempted or imagined speech. However, the risk of infection, tissue response management, and hardware degradation limit the scalability of these solutions. Brain2Qwerty v2 represents a major leap in accuracy for non-invasive decoding, where low signal-to-noise ratios have historically constrained performance.
That said, Brain2Qwerty remains far from clinical use. The 306-sensor MEG scanner used is expensive, immobile, and requires a magnetically shielded room. As such, the current Brain2Qwerty method practically replaces one scalability challenge with another. Meta highlighted the potential for optically pumped magnetometers as a route towards wearable MEG, while demonstrating that decoding accuracy remained robust even with fewer MEG sensors. Companies such as Cerca Magnetics are already developing this technology.
The current system also operates at the sentence level rather than in real time, producing text only once a typing trial has been completed. In addition, it has only been tested on healthy participants, decoding motor cortex activity generated during actual typing.While attempted typing in clinical populations may produce similar neural signals, Brain2Qwerty has not been evaluated on attempted or imagined typing, nor does it decode inner speech or silent reading.
Meta’s Brain2Qwerty does not exist in a vacuum. Companies such as Cognixion and Araya BCI are exploring non-invasive speech decoding for clinical use, but both use EEG instead of MEG. The distinction is notable, given Meta’s v1 paper found MEG substantially outperformed EEG for sentence reconstruction. In addition, Sabi is developing an EEG thought-to-text beanie, although with a consumer focus and unproven product thesis.
.svg)
