morphology: special characters
A digit is one of 1, 2, 3, 4, 5, 6, 7, 8, 9, 0. They can stand for the syllables pa, re, ci, vo, mu, xa, ze, bi, so, no in cmavo, in which case they constitute the whole cmavo, as part of a cmene, anywhere but at the end of it. They cannot appear in brivla.
A pause is any number (at least one) of space-characters, possibly preceded by commas.
A space-character is a blank, a dot, a question mark, an exclamation mark, an end-of-line or an end of string.
A comma is accepted anywhere and absorbed by the following character, treated as if it wasn't there (not a space!).
A non-lojban-word is any string of non-space-characters up to the next pause such that it does not begin with a Lojban-word. Any string that contains an unrecognized character or an impermissible sequence is a non-lojban-word.