Grengle #1: generating code-switching audio with Facebook MMS

I previously mentioned my attempts to learn Greek, and I haven't yet talked about my experiments with Grozer (and I won't right now, but they involve flash cards, clozes, and code-switching text). The big problem I've had with flash cards and other techniques is that they require my full attention, and take time out of the day. The result is that I eventually just drop them when something more important (or interesting) comes along.

So I've been wondering for a while whether there was a way to generate audio and make it comprehensible. I have a long commute to work every day, and I tend to use that for listening to the Bible in English. Is there a way to instead listen to the Bible in English+Greek; a code-switching audiobook that over time becomes more and more Greek?

Grengle is my set of experiments to attempt that.

The idea is to have a list of words I "know", and a handful of words that I'm "learning". Then as I listen to a passage, the words I "know" or am "learning" will be played in Greek, and the rest in English. Over time, it will automatically move the "learning" words into the "known" category, and pick new words to learn.

The first thing I tried was to get espeak to read Greek and English for me. I've been using espeak to read chapter headings for the Bible for a while now, and though it sounds like a robot, I can understand it. Greek, on the other hand, was terrible. Robotic Greek was beyond me, and so this experiment reached a dead end.

Recently, Facebook AI research put out a set of models that could (amongst other things) do text-to-speech for over a thousand languages. And Greek was one on the list. I finally got around to giving it a go, and once I got it installed, it generated some audio that at first seemed promising.

Then I plugged it into the Grengle interleaver engine, which slices a text into short phrases like the following:

GRK Καί λέγει αυτοῖς. ουκ οίδατε τήν
ENG parable
GRK ταύτην, καί πῶς πάσας τάς
ENG parables will you understand?
GRK ο
ENG sowing
...

The engine then used the model to render a bunch of audio files meant to be glued together.

The result was disappointing. There were a few dealbreaking issues:

  • Most importantly, the Greek speech was way too fast for me.
  • The Greek audio seemed to miss words occasionally, eg. a lone "ό" like the one above (which, yes, wouldn't normally occur in speech).
  • The English audio was rather muffled.

Thus, this second experiment failed.

I have a few ideas of how else I could get an interleaved audio system working:

  • Generate the text and record it myself. Which, since the words that should be played in Greek vs English is constantly changing, defeats the purpose of getting a model to read it for me.
  • Attempt to train a model myself (possibly a model that can handle both languages at once). I've never done this before.
  • Give up on the code-switching idea. Chop up some existing recordings into verse snippets, and play each one twice as English then Greek (or vice versa). However I suspect this will be too large a gap to get good Comprehensible Input at the word level.

Learning to read the Greek Bible

I want to read the Bible in the original Greek. To be specific, I'd like to be able to sit down and read the New Testament (or LXX) as fluently as I can English. For that, I need to learn Koine Greek.

The typical skill people are taught in ancient language classes is the ability to parse and "decode" those languages (but this is slowly changing). You learn a bunch of stuff about the grammar, and memorise a bunch of vocabulary, and then you're able to pick your way through a text.

This is reasonable given the constraints of those courses. It takes a long time and a lot of effort to really learn a language to fluency — something people eg in seminary don't always want/need to invest. In those cases having the skills to pick through a text and judge between more skilled scholars is enough, and useful.

But I'm a guy with free time, and I'd like to get beyond that stage, working my way asymptotically towards the point of fluency. Also I don't feel like memorising a bunch of vocab and conjugation tables!

So what I need is to internalise the language. Learn it in a similar way people learn living languages today.

There are multiple different paths you can take. Some people go to immersion colleges like Polis, or take online immersion courses like Seumas Macdonald's. While I might eventually do one of these, I want to start off on my own first, and at minimal cost.

The heart of language is Sound associated with Meaning. Even when we read silently, we usually subvocalise the written words into a stream of sound that we interpret as speech. And so the skills of knowing a language are to listen and speak: be able to interpret Sound into Meaning, and then (optionally in my case) be able to encode Meaning in Sound.

Apparently the best way to train you brain to unconsciouly do this is to get Comprehensible Input. This is any kind of input (words written or spoken) that you can understand "somehow", whether directly because you know them, or by some means of a side-channel — you can infer the meaning using something else. Getting lots of this kind of input — and it really needs to be a LOT of it — will over time will train your brain to automatically interpret the Sound.

So that's what I'm trying to do. I've been at this for a few years on and off, using various approaches. None of them have stuck very long, though some for longer than others. My current stab at this led me to build the application Grozer, and take it through multiple generations as my learning theories changed and new ideas came to me.

Next up, I'll talk more about my experiments with Grozer.

An afternoon build: Unravelled

This morning's digest email

I use Raindrop.io to collect links at work to read later. Every now and then I'd go in to the Inbox and read some of those things. But it's outside my normal routine, and for a while now the links have been accumulating.

What I wanted was a service that'd email me a few of the links every week, as if it was a newsletter.

A few days ago, in my holidays, I finally got around to building it, and came up with Unravelled — a script that takes the latest 8 links from a Raindrop.io collection, sends me a newsletter email, and removes them from the collection.

This was the fastest project I've ever built from start to finish. I made a point to start with all the boring deployment bits:

  • writing a Dockerfile
  • building the Docker container in a GitLab CI script
  • getting Podman to run it via a SystemD timer

I hadn't used SystemD to actually run a container before (previously only to call start/stop), and it worked really well.

Once I had my script running, getting it to actually do what it needed to do was straightforward: calling out to the Raindrop api a few times and generating an email body. Sending emails isn't intuitive in Python, but at least it's short.

And now it's running. After a couple hiccoughs with podman, it's now reliably emailing me every week.

Programming language experiments

This details a project called language_design that I started September 2015 and worked on sporadically ever since. It was written in Scala.

The initial aim was to build a language that mixed aspects of Scala and Erlang. It would be strongly & statically typed, have actors, and be roughly functional. Functional in the sense that immutability was encouraged and functions were used everywhere, but purity wasn't a big concern.

Read more…

Battling Maven repository hosting

Write your post here.

title: Battling maven repository hosting date: 2014-08-08 08:53:07 categories: - Software tags: - java - note-to-self


Recently I've been working on a closed-source application for a client. It's an Android implementation of a recently invented card game, with both single player and multiplayer modes. I had to write a custom server for it (I chose the Play framework with Scala). The fact that the system had effectively two server implementations --- one for single player coordinating bots, and one for multiplayer --- combined with my lack of experience building such systems meant I created a lot of duplication when implementing the rules of the game.

Read more…

Rigid-body physics using particles

Towards the end of last year, I worked on a prototype physics engine based on a GPU Gems chapter --- a realtime physics simulation running on the GPU. So far I've only written it for the CPU as I've never done any GPU work before. (judging by the problems I've had with the physics side of things, it was probably a good idea not to heap on an order of magnitude more GPU problems...)

Read more…

Balloon Box

I've recently been working on a game called Balloon Box. It's a 2D desktop platformer in which the player controls a cardboard box, navigates through an abandoned factory which is known for its pneumatic material transport system. (When I implement it) you will have to avoid cardboard-dissolving acid and knife racks, and collect balloons and batteries. Read the story on Github.

Read more…