Introduction—Historical Context To Show Contrast Between Rule-Based and Stastical-Based Approaches To NLP

Two things distinguish Discovery from other NLP systems:

• Discovery is entirely rule-based, and
• Discovery's grammatical analysis is divided into twelve successive stages, all of which govern every kind of phrase and clause which occur in English sentences.

With regard to rule-based analysis, some historical background is necessary to understand the difference.

From the 1950s until the late 1980s, all NLP systems were made up of small sets of carefully handcrafted rules based on fewer than several hundred examples. They meant to devise grammars with a broad enough coverage to parse any kind of conceivable sentence, but they fell short of their expectations. It wasn't that they weren't competent or lacked effort. One hypothesis for this shortcoming was that while a human's grammar is largely modular, the interaction of the components led the system to be unimaginably complex. Developers accepted this hypothesis, and looked for other approaches.

They eventually settled on systems that analyzed text by conducting statistical analysis of corpora—or large, structured, annotated sets of English text—putting more emphasis the computer's ability to parse sentences and disambiguate words. The computer compares a given sentence to examples using the same words in the corpora.

While the results were better, statistical approaches still have limitations. No matter how extensive corpora are, computers lack awareness of what words represent. The computer essentially makes its best guess, without due judgment, mindlessly following statistical rules. Typically they don't identify true ambiguities that a human would ask for clarification to resolve—and such a system would have to ask, because how can we expect a computer to be better at resolving ambiguity than the human beings who built it?

In contrast to rule-based approaches, statistical analysis takes away control from the developer to ensure that the system mimics or emulates human comprehension of language as closely as possible, for whatever practical and productive purposes it's used. Furthermore, many psychological studies indicate that while examples remain important, rule learning is generally more efficient than exposure to examples.

As such, computerized linguistic analysis is a double-edged sword. Nevertheless, no matter how difficult it would eventually and in fact did turn out to be, I decided to review and devise a vastly improved rule-based system of my own. Overall, I believe that I have succeeded, and wish to make it open for peer review. But now, for you, the prospective investor, here is a brief overview of the entire process:

Fragment Analysis

Fragments are defined as either

• individual identifiers or "signal" words: articles, prepositions, conjunctions and all manner of pronouns or interjections, or
• one or more series of consecutive non-identifiers: nouns, verbs, adjectives or adverbs.

In his book The Structure of English, linguist Charles Carpenter Fries addresses the question of how speakers of a language recognize structural meaning. He argued that structural meaning is "signaled by specific and definite devices...that signal structural meanings which constitute the grammar of a language." Fries called these words function or signal words—words which do not belong to the four major part-of-speech classes of nouns, verbs, adjectives and adverbs. This leaves a limited list of more commonly used words. They include prepositions, conjunctions, articles, interjections, whose meanings are generally unambiguous. In development, we refer to these words as "identifiers". Discovery adopts a strategy of first dividing up sentences into fragments, consisting of any nouns, verbs, adjectives and adverbs between different pairs of identifiers, to more readily assess structural meaning. This helps to make sentence analysis far faster and more efficient, because it eliminates certain part-of-speech fragments—which would otherwise and invariably result in ungrammatical part-of-speech permutations for the entire sentence—before they are ever tested. The remaining fragments are recombined to a more limited list of permutations to test.

This highlights yet another way Discovery is unique among other NLP systems. Instead of the traditional method of part-of-speech tagging, it parses the sentence by determining which part-of-speech permutations result in a grammatically correct sentence.

The process can be compared to a bank vault. Only one combination, or rather permutation of numbers in a specific order will succeed in opening it. But depending on the number of turns there are, and how many possibilities for each turn, there could be a vast number of other permutations which fail. The more turns there are in the permutation, and more possibilities for each turn, the more time must be spent testing all of them. Yet you could save an enormous amount of time if there was a way to reduce the number of permutations you would have to test.

This is the purpose of fragment analysis. The solution is to break up the sentence into these fragments and figure out which ones occur in grammatically correct English sentences. The system extracts each possible part-of-speech permutation of each fragment, and determines the number of words in the fragment, its type, and the combination of any identifiers preceding or following it. Based on these three criteria, the result is a radically smaller number of permutations for the entire sentence that the system would later test in the next stage of the analysis.

If the system can't find a valid sequence of parts of speech for a fragment in a sentence which is grammatically correct, at least in the estimation of the developer, an interface is available to add it easily and continue processing.

Word Sense Analysis

This stage conducts three types of word sense disambiguation (or WSD) techniques for individual words. WSD is the process of identifying which sense of a word (i.e. meaning or definition) applies in the context of a sentence, when the word has more than one.

• Word Pair WSD

The second method governs sense disambiguation of words in pairs, depending on the relationships provided by the sentence diagram Discovery generates. These pairs include:

an adverb modifying an adjective or verb an adjective modifying a verb or noun a direct object noun of a verb, and a subject noun with a predicate verb

There are four different types of rules that apply to each of these pairs. The reason for there being four types is to prevent an excess of such rules, the problem which plagued rule-based NLP systems in the past. This new strategy greatly reduces the number of rules necessary by compelling Discovery to search for rules according to priority, for specific rules that apply to different word properties before searching for more general ones. definition / selectional restriction (provided by VerbNet) definition / hypernym ancestor definition / lexical category lexical category / lexical category

Here again, an interface has been built to easily add these rules where necessary to broaden Discovery's ability to exercise reliable WSD, a feature not found in NLP systems using statistical analysis.