Wow, that's deep. I should try to explain that better.

Regular expressions are a programming language for defining and running a particular kind of FSM.

But python (and unix/Linux/posix) regexes take some liberties with some of the rules that FSMs must obey, like no "lookahead." So they aren't exactly FSMs, but if you need them to be, you can just not use those features in Python's 're' package. The reason you might want them to be true FSMs, is that FSMs always make a decision, match or not match. But poorly-written regexes can run forever or crash. Kyle Gorman is working on a whole book to explain FSMs with python examples.

Neural Turing Machines are new to me. Until I've implemented and tested some examples (chapter 12) I can't say much, authoritatively, about them. My vague understanding is that they are neural networks that can learn to write algorithms/programs that a Turing machine can run. NTMs are programs trained to write programs based on examples (data) that you, the programmer's trainer, feed into it. Deep.

And that brings us to machine learning, another self-programming algorithm. But machine learning came before NTMs (I think). It's simpler. If you can reduce a problem to a mathematical function, like multiplying and adding a bunch of numbers (the dot product of a matrix of weights with a vector of input values) that makes it much easier to find the right "program" that solves your problem. You just have a computer search for the best possible "weights" to use in that formula based on how close the output is to the examples you train it with (the training dataset, which pairs input vectors with desired output vectors). It's trial and error programming of a very simple program, without loops or recursion any other fancy programming tricks, just math. It's how a computer does TDD on it's own

The trick is writing algorithms that make good guesses at the weights and don't take too much data to figure out a good set of math formula coefficients (weights).

Machine learning is what most of the rest of NLPIA is about. You can see some rough examples in the python package 'nlpia' as we build examples for the book. Just keep pulling the latest source code, because we're updating it weekly. (

http://github.com/totalgood/nlpia). And send us pull requests for NLP machine learning problems or programs you are working on.